Index: The Book of Statistical ProofsProbability Distributions ▷ Multivariate continuous distributions ▷ Normal-gamma distribution ▷ Conditional distributions

Theorem: Let $x$ and $y$ follow a normal-gamma distribution:

$\label{eq:ng} x,y \sim \mathrm{NG}(\mu, \Lambda, a, b) \; .$

Then,

1) the conditional distribution of $x$ given $y$ is a multivariate normal distribution

$\label{eq:ng-cond-x-y} x|y \sim \mathcal{N}(\mu, (y \Lambda)^{-1}) \; ;$

2) the conditional distribution of a subset vector $x_1$, given the complement vector $x_2$ and $y$, is also a multivariate normal distribution

$\label{eq:ng-cond-x1-x2-y} x_1|x_2,y \sim \mathcal{N}(\mu_{1|2}(y), \Sigma_{1|2}(y))$

with the conditional mean and covariance

$\label{eq:ng-cond-x1-x2-y-hyp} \begin{split} \mu_{1|2}(y) &= \mu_1 + \Sigma_{12} \Sigma_{22}^{-1} (x_2 - \mu_2) \\ \Sigma_{1|2}(y) &= \Sigma_{11} - \Sigma_{12} \Sigma_{22}^{-1} \Sigma_{12} \end{split}$

where $\mu_1$, $\mu_2$ and $\Sigma_{11}$, $\Sigma_{12}$, $\Sigma_{22}$, $\Sigma_{21}$ are block-wise components of $\mu$ and $\Sigma(y) = (y \Lambda)^{-1}$;

3) the conditional distribution of $y$ given $x$ is a gamma distribution

$\label{eq:ng-cond-y-x} y|x \sim \mathrm{Gam}\left( a + \frac{n}{2}, b + \frac{1}{2} (x-\mu)^\mathrm{T} \Lambda (x-\mu) \right)$

where $n$ is the dimensionality of $x$.

Proof:

1) This follows from the definition of the normal-gamma distribution:

$\label{eq:ng-pdf} \begin{split} p(x,y) &= p(x|y) \cdot p(y) \\ &= \mathcal{N}(x; \mu, (y \Lambda)^{-1}) \cdot \mathrm{Gam}(y; a, b) \; . \end{split}$

2) This follows from \eqref{eq:ng-cond-x-y} and the conditional distributions of the multivariate normal distribution:

$\label{eq:mvn-cond} \begin{split} x &\sim \mathcal{N}(\mu, \Sigma) \\ \Rightarrow x_1|x_2 &\sim \mathcal{N}(\mu_{1|2}, \Sigma_{1|2}) \\ \mu_{1|2} &= \mu_1 + \Sigma_{12} \Sigma_{22}^{-1} (x_2 - \mu_2) \\ \Sigma_{1|2} &= \Sigma_{11} - \Sigma_{12} \Sigma_{22}^{-1} \Sigma_{21} \; . \end{split}$

3) The conditional density of $y$ given $x$ follows from Bayes’ theorem as

$\label{eq:ng-cond-y-x-s1} p(y|x) = \frac{p(x|y) \cdot p(y)}{p(x)} \; .$

The conditional distribution of $x$ given $y$ is a multivariate normal distribution

$\label{eq:ng-x-y-pdf} p(x|y) = \mathcal{N}(x; \mu, (y \Lambda)^{-1}) = \sqrt{\frac{|y \Lambda|}{(2 \pi)^n}} \exp \left[ -\frac{1}{2} (x-\mu)^\mathrm{T} (y \Lambda) (x-\mu) \right] \; ,$ $\label{eq:ng-y-pdf} p(y) = \mathrm{Gam}(y; a, b) = \frac{b^a}{\Gamma(a)} y^{a-1} \exp\left[ -by \right]$ $\label{eq:ng-x-pdf} \begin{split} p(x) &= t\left( x; \mu, \left(\frac{a}{b} \Lambda \right)^{-1}, 2a \right) \\ &= \sqrt{\frac{\left| \frac{a}{b}\,\Lambda \right|}{(2a\,\pi)^n}} \cdot \frac{\Gamma\left( \frac{2a+n}{2} \right)}{\Gamma\left( \frac{2a}{2} \right)} \cdot \left( 1 + \frac{1}{2a} (x-\mu)^\mathrm{T} \left( \frac{a}{b}\Lambda \right) (x-\mu) \right)^{-\frac{2a+n}{2}} \\ &= \sqrt{\frac{|\Lambda|}{(2 \pi)^n}} \cdot \frac{\Gamma\left( a+\frac{n}{2} \right)}{\Gamma(a)} \cdot b^a \cdot \left( b + \frac{1}{2} (x-\mu)^\mathrm{T} \Lambda (x-\mu) \right)^{-\left( a+\frac{n}{2} \right)} \; . \end{split}$

Plugging \eqref{eq:ng-x-y-pdf}, \eqref{eq:ng-y-pdf} and \eqref{eq:ng-x-pdf} into \eqref{eq:ng-cond-y-x-s1}, we obtain

$\label{eq:ng-cond-y-x-s2} \begin{split} p(y|x) &= \frac{\sqrt{\frac{|y \Lambda|}{(2 \pi)^n}} \exp \left[ -\frac{1}{2} (x-\mu)^\mathrm{T} (y \Lambda) (x-\mu) \right] \cdot \frac{b^a}{\Gamma(a)} y^{a-1} \exp\left[ -by \right]}{\sqrt{\frac{|\Lambda|}{(2 \pi)^n}} \cdot \frac{\Gamma\left( a+\frac{n}{2} \right)}{\Gamma(a)} \cdot b^a \cdot \left( b + \frac{1}{2} (x-\mu)^\mathrm{T} \Lambda (x-\mu) \right)^{-\left( a+\frac{n}{2} \right)}} \\ &= y^{\frac{n}{2}} \cdot \exp \left[ -\frac{1}{2} (x-\mu)^\mathrm{T} (y \Lambda) (x-\mu) \right] \cdot y^{a-1} \cdot \exp\left[ -by \right] \cdot \frac{1}{\Gamma\left( a+\frac{n}{2} \right)} \cdot \left( b + \frac{1}{2} (x-\mu)^\mathrm{T} \Lambda (x-\mu) \right)^{a+\frac{n}{2}} \\ &= \frac{\left( b + \frac{1}{2} (x-\mu)^\mathrm{T} \Lambda (x-\mu) \right)^{a+\frac{n}{2}}}{\Gamma\left( a+\frac{n}{2} \right)} \cdot y^{a+\frac{n}{2}-1} \cdot \exp \left[ -\left( b + \frac{1}{2} (x-\mu)^\mathrm{T} \Lambda (x-\mu) \right) \right] \end{split}$

which is the probability density function of a gamma distribution with shape and rate parameters

$\label{eq:ng-cond-y-x-hyp} a + \frac{n}{2} \quad \text{and} \quad b + \frac{1}{2} (x-\mu)^\mathrm{T} \Lambda (x-\mu) \; ,$

such that

$\label{eq:ng-cond-y-x-qed} p(y|x) = \mathrm{Gam}\left( y; a + \frac{n}{2}, b + \frac{1}{2} (x-\mu)^\mathrm{T} \Lambda (x-\mu) \right) \; .$
Sources:

Metadata: ID: P146 | shortcut: ng-cond | author: JoramSoch | date: 2020-08-05, 06:54.