Index: The Book of Statistical Proofs ▷ Probability Distributions ▷ Multivariate continuous distributions ▷ Normal-gamma distribution ▷ Drawing samples

Theorem: Let $Z_1 \in \mathbb{R}^n$ be a random vector with all entries independently following a standard normal distribution and let $Z_2 \in \mathbb{R}$ be a random variable following a standard gamma distribution with shape $a$. Moreover, let $A \in \mathbb{R}^{n \times n}$ be a matrix, such that $A A^\mathrm{T} = \Lambda^{-1}$.

Then, $X = \mu + A Z_1 / \sqrt{Z_2/b}$ and $Y = Z_2/b$ jointly follow a normal-gamma distribution with mean vector $\mu$, precision matrix $\Lambda$, shape parameter $a$ and rate parameter $b$:

\[\label{eq:ng-samp} \left( X = \mu + A Z_1 / \sqrt{Z_2/b}, \; Y = Z_2/b \right) \sim \mathrm{NG}(\mu, \Lambda, a, b) \; .\]

Proof: If all entries of $Z_1$ are independent and standard normally distributed

\[\label{eq:zi-dist} z_{1i} \overset{\mathrm{i.i.d.}}{\sim} \mathcal{N}(0, 1) \quad \text{for all} \quad i = 1,\ldots,n \; ,\]

this implies a multivariate normal distribution with diagonal covariance matrix:

\[\label{eq:Z1-dist} Z_1 \sim \mathcal{N}\left(0_n, I_n \right)\]

where $0_n$ is an $n \times 1$ matrix of zeros and $I_n$ is the $n \times n$ identity matrix.

If the distribution of $Z_2$ is a standard gamma distribution

\[\label{eq:Z2-dist} Z_2 \sim \mathrm{Gam}(a, 1) \; ,\]

then due to the relationship between gamma and standard gamma distribution distribution, we have:

\[\label{eq:Y-dist} Y = \frac{Z_2}{b} \sim \mathrm{Gam}(a,b) \; .\]

Moreover, using the linear transformation theorem for the multivariate normal distribution, it follows that:

\[\label{eq:X-dist} \begin{split} Z_1 &\sim \mathcal{N}\left(0_n, I_n \right) \\ X = \mu + \frac{1}{\sqrt{Z_2/b}} A Z_1 &\sim \mathcal{N}\left(\mu + \frac{1}{\sqrt{Z_2/b}} A \, 0_n, \left( \frac{1}{\sqrt{Z_2/b}} A \right) I_n \left( \frac{1}{\sqrt{Z_2/b}} A \right)^\mathrm{T} \right) \\ X &\sim \mathcal{N}\left(\mu + 0_n, \left( \frac{1}{\sqrt{Y}} \right)^2 A A^\mathrm{T} \right) \\ X &\sim \mathcal{N}\left(\mu, \left( Y \Lambda \right)^{-1} \right) \; . \end{split}\]

Thus, $Y$ follows a gamma distribution and the distribution of $X$ conditional on $Y$ is a multivariate normal distribution:

\[\label{eq:mvn-gam} \begin{split} X \vert Y &\sim \mathcal{N}(\mu, (Y \Lambda)^{-1}) \\ Y &\sim \mathrm{Gam}(a, b) \; . \end{split}\]

This means that, by definition, $X$ and $Y$ jointly follow a normal-gamma distribution:

\[\label{eq:ng-samp-qed} X,Y \sim \mathrm{NG}(\mu, \Lambda, a, b) \; ,\]

Thus, given $Z_1$ defined by \eqref{eq:zi-dist} and $Z_2$ defined by \eqref{eq:Z2-dist}, $X$ and $Y$ defined by \eqref{eq:ng-samp} are a sample from $\mathrm{NG}(\mu, \Lambda, a, b)$.

∎
Sources:

Metadata: ID: P346 | shortcut: ng-samp | author: JoramSoch | date: 2022-09-22, 11:10.