Index: The Book of Statistical Proofs ▷ Probability Distributions ▷ Multivariate continuous distributions ▷ Multivariate normal distribution ▷ Conditions for independence

Theorem: Let $x$ be an $n \times 1$ random vector following a multivariate normal distribution:

\[\label{eq:mvn} x \sim \mathcal{N}(\mu, \Sigma) \; .\]

Then, the components of $x$ are statistically independent, if and only if the covariance matrix is a diagonal matrix:

\[\label{eq:mvn-ind} p(x) = p(x_1) \cdot \ldots \cdot p(x_n) \quad \Leftrightarrow \quad \Sigma = \mathrm{diag}\left( \left[ \sigma^2_1, \ldots, \sigma^2_n \right] \right) \; .\]

Proof: The marginal distribution of one entry from a multivariate normal random vector is a univariate normal distribution where mean and variance are equal to the corresponding entries of the mean vector and covariance matrix:

\[\label{eq:mvn-marg} x \sim \mathcal{N}(\mu, \Sigma) \quad \Rightarrow \quad x_i \sim \mathcal{N}(\mu_i, \sigma^2_{ii}) \; .\]

The probability density function of the multivariate normal distribution is

\[\label{eq:mvn-pdf} p(x) = \frac{1}{\sqrt{(2 \pi)^n |\Sigma|}} \cdot \exp \left[ -\frac{1}{2} (x-\mu)^\mathrm{T} \Sigma^{-1} (x-\mu) \right]\]

and the probability density function of the univariate normal distribution is

\[\label{eq:norm-pdf} p(x_i) = \frac{1}{\sqrt{2 \pi \sigma^2_i}} \cdot \exp \left[ -\frac{1}{2} \left( \frac{x_i-\mu_i}{\sigma_i} \right)^2 \right] \; .\]


1) Let

\[\label{eq:x-ind} p(x) = p(x_1) \cdot \ldots \cdot p(x_n) \; .\]

Then, we have

\[\label{eq:x-ind-dev} \begin{split} \frac{1}{\sqrt{(2 \pi)^n |\Sigma|}} \cdot \exp \left[ -\frac{1}{2} (x-\mu)^\mathrm{T} \Sigma^{-1} (x-\mu) \right] &\overset{\eqref{eq:mvn-pdf},\eqref{eq:norm-pdf}}{=} \prod_{i=1}^{n} \frac{1}{\sqrt{2 \pi \sigma^2_i}} \cdot \exp \left[ -\frac{1}{2} \left( \frac{x_i-\mu_i}{\sigma_i} \right)^2 \right] \\ \frac{1}{\sqrt{(2 \pi)^n |\Sigma|}} \cdot \exp \left[ -\frac{1}{2} (x-\mu)^\mathrm{T} \Sigma^{-1} (x-\mu) \right] &= \frac{1}{\sqrt{(2 \pi)^n \prod_{i=1}^{n} \sigma^2_i}} \cdot \exp \left[ -\frac{1}{2} \sum_{i=1}^{n} (x_i-\mu_i) \frac{1}{\sigma^2_i} (x_i-\mu_i) \right] \\ - \frac{1}{2} \log |\Sigma| - \frac{1}{2} (x-\mu)^\mathrm{T} \Sigma^{-1} (x-\mu) &= - \frac{1}{2} \sum_{i=1}^{n} \log \sigma^2_i - \frac{1}{2} \sum_{i=1}^{n} (x_i-\mu_i) \frac{1}{\sigma^2_i} (x_i-\mu_i) \end{split}\]

which is only fulfilled by a diagonal covariance matrix

\[\label{eq:Sigma-diag-qed} \Sigma = \mathrm{diag}\left( \left[ \sigma^2_1, \ldots, \sigma^2_n \right] \right) \; ,\]

because the determinant of a diagonal matrix is a product

\[\label{eq:diag-det} | \mathrm{diag}\left( \left[ a_1, \ldots, a_n \right] \right) | = \prod_{i=1}^n a_i \; ,\]

the inverse of a diagonal matrix is a diagonal matrix

\[\label{eq:diag-inv} \mathrm{diag}\left( \left[ a_1, \ldots, a_n \right] \right)^{-1} = \mathrm{diag}\left( \left[ 1/a_1, \ldots, 1/a_n \right] \right)\]

and the squared form with a diagonal matrix is

\[\label{eq:diag-sqr} x^\mathrm{T} \mathrm{diag}\left( \left[ a_1, \ldots, a_n \right] \right) x = \sum_{i=1}^n a_i x_i^2 \; .\]


2) Let

\[\label{eq:Sigma-diag} \Sigma = \mathrm{diag}\left( \left[ \sigma^2_1, \ldots, \sigma^2_n \right] \right) \; .\]

Then, we have

\[\label{eq:Sigma-diag-dev} \begin{split} p(x) &\overset{\eqref{eq:mvn-pdf}}{=} \frac{1}{\sqrt{(2 \pi)^n |\mathrm{diag}\left( \left[ \sigma^2_1, \ldots, \sigma^2_n \right] \right)|}} \cdot \exp \left[ -\frac{1}{2} (x-\mu)^\mathrm{T} \mathrm{diag}\left( \left[ \sigma^2_1, \ldots, \sigma^2_n \right] \right)^{-1} (x-\mu) \right] \\ &= \frac{1}{\sqrt{(2 \pi)^n \prod_{i=1}^{n} \sigma^2_i}} \cdot \exp \left[ -\frac{1}{2} (x-\mu)^\mathrm{T} \mathrm{diag}\left( \left[ 1/\sigma^2_1, \ldots, 1/\sigma^2_n \right] \right) (x-\mu) \right] \\ &= \frac{1}{\sqrt{(2 \pi)^n \prod_{i=1}^{n} \sigma^2_i}} \cdot \exp \left[ -\frac{1}{2} \sum_{i=1}^{n} \frac{(x_i-\mu_i)^2}{\sigma^2_i} \right] \\ &= \prod_{i=1}^n \frac{1}{\sqrt{2 \pi \sigma^2_i}} \cdot \exp \left[ -\frac{1}{2} \left( \frac{x_i-\mu_i}{\sigma_i} \right)^2 \right] \end{split}\]

which implies that

\[\label{eq:x-ind-qed} p(x) = p(x_1) \cdot \ldots \cdot p(x_n) \; .\]
∎
Sources:

Metadata: ID: P236 | shortcut: mvn-ind | author: JoramSoch | date: 2021-06-02, 09:22.