Proof: The p-value follows a uniform distribution under the null hypothesis
Index: The Book of Statistical Proofs ▷ General Theorems ▷ Frequentist statistics ▷ Hypothesis testing ▷ Distribution of p-value under null hypothesis
Metadata: ID: P318 | shortcut: pval-h0 | author: JoramSoch | date: 2022-03-18, 22:37.
Theorem: Under the null hypothesis, the p-value in a statistical test follows a continuous uniform distribution:
\[\label{eq:pval-h0} p \sim \mathcal{U}(0,1) \; .\]Proof: Without loss of generality, consider a left-sided one-tailed hypothesis test. Then, the p-value is a function of the test statistic
\[\label{eq:pval} \begin{split} P &= F_T(T) \\ p &= F_T(t_\mathrm{obs}) \end{split}\]where $t_\mathrm{obs}$ is the observed test statistic and $F_T(t)$ is the cumulative distribution function of the test statistic under the null hypothesis.
Then, we can obtain the cumulative distribution function of the p-value as
\[\label{eq:pval-cdf-s1} \begin{split} F_P(p) &= \mathrm{Pr}(P < p) \\ &= \mathrm{Pr}(F_T(T) < p) \\ &= \mathrm{Pr}(T < F_T^{-1}(p)) \\ &= F_T(F_T^{-1}(p)) \\ &= p \end{split}\]which is the cumulative distribution function of a continuous uniform distribution over the interval $[0,1]$:
\[\label{eq:cuni-cdf} F_X(x) = \int_{-\infty}^{x} \mathcal{U}(z; 0, 1) \, \mathrm{d}z = x \quad \text{where} \quad 0 \leq x \leq 1 \; .\]∎
Sources: - jll (2018): "Why are p-values uniformly distributed under the null hypothesis?"; in: StackExchange CrossValidated, retrieved on 2022-03-18; URL: https://stats.stackexchange.com/a/345763/270304.
Metadata: ID: P318 | shortcut: pval-h0 | author: JoramSoch | date: 2022-03-18, 22:37.