Position: Predictive Uncertainty Is Not Enough — Joint Distribution for Full Uncertainty Representation
Abstract
When AI is deployed in safety-critical domains, erroneous and overconfident predictions can have severe consequences, making comprehensive uncertainty quantification (UQ) essential for responsible decision-making. Current UQ methods based on epistemic and aleatoric decomposition are insufficient for fully understanding the problem, a limitation further compounded by treating these terms in isolation from uncertainty about the domain. Our position claims that any meaningful analysis must account for three sources of uncertainty — domain, epistemic, and aleatoric — and that only the joint distribution $p(x,y|\mathcal{D})$ provides a coherent representation of them. We mirror prior findings showing that information-theoretic UQ methods are suboptimal in ID and OOD settings, mainly due to the difficulty of disentangling epistemic and aleatoric components. We argue that modeling the unconditional distribution $p(x|\mathcal{D})$ is necessary to account for input validity, giving rise to a third class of uncertainty: domain uncertainty. Finally, by jointly considering the domain distribution and the conditional distribution $p(y|x,\mathcal{D})$, we argue that their product, $p(x,y|\mathcal{D})$, captures all relevant sources of uncertainty.