Index: The Book of Statistical Proofs ▷ Statistical Models ▷ Univariate normal data ▷ Bayesian linear regression ▷ Posterior probability of alternative hypothesis

Theorem: Let there be a linear regression model with normally distributed errors:

\[\label{eq:GLM} y = X \beta + \varepsilon, \; \varepsilon \sim \mathcal{N}(0, \sigma^2 V)\]

and assume a normal-gamma prior distribution over the model parameters $\beta$ and $\tau = 1/\sigma^2$:

\[\label{eq:GLM-NG-prior} p(\beta,\tau) = \mathcal{N}(\beta; \mu_0, (\tau \Lambda_0)^{-1}) \cdot \mathrm{Gam}(\tau; a_0, b_0) \; .\]

Then, the posterior probability of the alternative hypothesis

\[\label{eq:GLM-H1} \mathrm{H}_1: \, c^\mathrm{T} \beta > 0\]

is given by

\[\label{eq:GLM-NG-PP} \mathrm{Pr}\left( \mathrm{H}_1 | y \right) = 1 - \mathrm{T}\left( -\frac{c^\mathrm{T} \mu}{\sqrt{c^\mathrm{T} \Sigma c}}; \nu \right)\]

where $c$ is a $p \times 1$ contrast vector, $\mathrm{T}(x; \nu)$ is the cumulative distribution function of the t-distribution with $\nu$ degrees of freedom and $\mu$, $\Sigma$ and $\nu$ can be obtained from the posterior hyperparameters of Bayesian linear regression.

Proof: The posterior distribution for Bayesian linear regression is given by a normal-gamma distribution over $\beta$ and $\tau = 1/\sigma^2$

\[\label{eq:GLM-NG-post} p(\beta,\tau|y) = \mathcal{N}(\beta; \mu_n, (\tau \Lambda_n)^{-1}) \cdot \mathrm{Gam}(\tau; a_n, b_n)\]

with the posterior hyperparameters

\[\label{eq:GLM-NG-post-par} \begin{split} \mu_n &= \Lambda_n^{-1} (X^\mathrm{T} P y + \Lambda_0 \mu_0) \\ \Lambda_n &= X^\mathrm{T} P X + \Lambda_0 \\ a_n &= a_0 + \frac{n}{2} \\ b_n &= b_0 + \frac{1}{2} (y^\mathrm{T} P y + \mu_0^\mathrm{T} \Lambda_0 \mu_0 - \mu_n^\mathrm{T} \Lambda_n \mu_n) \; . \end{split}\]

The marginal distribution of a normal-gamma distribution is a multivariate t-distribution, such that the marginal posterior distribution of $\beta$ is

\[\label{eq:GLM-NG-post-beta} p(\beta|y) = t(\beta; \mu, \Sigma, \nu)\]

with the posterior hyperparameters

\[\label{eq:GLM-NG-post-par-beta} \begin{split} \mu &= \mu_n \\ \Sigma &= \left( \frac{a_n}{b_n} \Lambda_n \right)^{-1} \\ \nu &= 2 \, a_n \; . \end{split}\]

Define the quantity $\gamma = c^\mathrm{T} \beta$. According to the linear transformation theorem for the multivariate t-distribution, $\gamma$ also follows a multivariate t-distribution:

\[\label{eq:GLM-NG-post-gamma} p(\gamma|y) = t(\gamma; c^\mathrm{T} \mu, c^\mathrm{T} \Sigma \, c, \nu) \; .\]

Because $c^\mathrm{T}$ is a $1 \times p$ vector, $\gamma$ is a scalar and actually has a non-standardized t-distribution. Therefore, the posterior probability of $H_1$ can be calculated using a one-dimensional integral:

\[\label{eq:GLM-NG-post-prob-H0-s1} \begin{split} \mathrm{Pr}\left( \mathrm{H}_1 | y \right) &= p(\gamma > 0|y) \\ &= \int_{0}^{+\infty} p(\gamma|y) \, \mathrm{d}\gamma \\ &= 1 - \int_{-\infty}^{0} p(\gamma|y) \, \mathrm{d}\gamma \\ &= 1 - \mathrm{T}_\mathrm{nst}(0; c^\mathrm{T} \mu, c^\mathrm{T} \Sigma \, c, \nu) \; . \end{split}\]

Using the relation between non-standardized t-distribution and standard t-distribution, we can finally write:

\[\label{eq:GLM-NG-post-prob-H0-s2} \begin{split} \mathrm{Pr}\left( \mathrm{H}_1 | y \right) &= 1 - \mathrm{T}\left( \frac{(0 - c^\mathrm{T} \mu)}{\sqrt{c^\mathrm{T} \Sigma c}}; \nu \right) \\ &= 1 - \mathrm{T}\left( -\frac{c^\mathrm{T} \mu}{\sqrt{c^\mathrm{T} \Sigma c}}; \nu \right) \; . \end{split}\]
∎
Sources:

Metadata: ID: P133 | shortcut: blr-pp | author: JoramSoch | date: 2020-07-17, 17:03.