Index: The Book of Statistical Proofs ▷ Probability Distributions ▷ Univariate continuous distributions ▷ Normal distribution ▷ Normal and uncorrelated does not imply independent

Theorem: Consider two random variables $X$ and $Y$. If each of them is normally distributed and both are uncorrelated, then $X$ and $Y$ are not necessarily independent.

Proof: As an example, let $V$ follow a Bernoulli distribution with success probability $1/2$ and let $W$ be defined as a transformation of $V$:

\[\label{eq:V-W} \begin{split} V &\sim \mathrm{Bern}\left( \frac{1}{2} \right) \\ W &= 2V-1 \; . \end{split}\]

By definition of the Bernoulli distribution, it follows that

\[\label{eq:V-W-dist} p(V=0) = p(V=1) = \frac{1}{2} \quad \Rightarrow \quad p(W=-1) = p(W=+1) = \frac{1}{2} \; .\]

Moreover, let $X$ follow a standard normal distribution and let $Y$ be defined as a combination of $X$ and $W$:

\[\label{eq:X-Y} \begin{split} X &\sim \mathcal{N}(0,1) \\ Y &= WX \; . \end{split}\]

Then, by the nature of the random variable $W$, it follows that

\[\label{eq:X-Y-dist} p(W=-1) = p(W=+1) = \frac{1}{2} \quad \Rightarrow \quad p(Y=-X) = p(Y=+X) = \frac{1}{2} \; .\]

Since the negative of a standard normal random variable is also standard normally distributed,

\[\label{eq:X-dist} X \sim \mathcal{N}(0,1) \quad \Rightarrow \quad -X \sim \mathcal{N}(0,1) \; ,\]

we can calculate the probability density function belonging to the mixture distribution of $Y$ as follows:

\[\label{eq:Y-pdf} \begin{split} p(y) &= p(y|Y=-X) \cdot p(Y=-X) + p(y|Y=+X) \cdot p(Y=+X) \\ &\overset{\eqref{eq:X-Y-dist}}{=} \mathcal{N}(y; 0, 1) \cdot \frac{1}{2} + \mathcal{N}(y; 0, 1) \cdot \frac{1}{2} \\ &= \mathcal{N}(y; 0, 1) \end{split}\]

where we have used the law of marginal probability in the first line and $\mathcal{N}(x; \mu, \sigma^2)$ denotes the probability density function of the normal distribution. Thus, $Y$ is also standard normally distributed:

\[\label{eq:Y-dist} Y \sim \mathcal{N}(0,1) \; .\]

This means that both $X$ and $Y$ have expected value zero:

\[\label{eq:X-Y-mean} \mathrm{E}(X) = \mathrm{E}(Y) = 0 \; .\]

With that, we can start to work out the covariance of $X$ and $Y$:

\[\label{eq:X-Y-cov-s1} \begin{split} \mathrm{Cov}(X,Y) &= \mathrm{E}\left[ \left( X-\mathrm{E}(X) \right) \left( Y-\mathrm{E}(Y) \right) \right] \\ &\overset{\eqref{eq:X-Y-mean}}{=} \mathrm{E}\left[ XY \right] \\ &\overset{\eqref{eq:X-Y}}{=} \mathrm{E}\left[ XWX \right] \\ &= \mathrm{E}\left[ WX^2 \right] \; . \end{split}\]

Since $W$ and $X$ are independent by construction, their expected values factorize:

\[\label{eq:X-Y-cov-s2} \begin{split} \mathrm{Cov}(X,Y) &= \mathrm{E}[W] \cdot \mathrm{E}[X^2] \\ &= \left( (-1) \cdot p(W=-1) + (+1) \cdot p(W=+1) \right) \cdot \mathrm{E}[X^2] \\ &\overset{\eqref{eq:V-W-dist}}{=} \left( (-1) \cdot \frac{1}{2} + (+1) \cdot \frac{1}{2} \right) \cdot \mathrm{E}[X^2] \\ &= 0 \cdot \mathrm{E}[X^2] \\ &= 0 \; . \end{split}\]

Thus, $X$ and $Y$ are uncorrelated:

\[\label{eq:X-Y-corr} \mathrm{Corr}(X,Y) = \frac{\mathrm{Cov}(X,Y)}{\sqrt{\mathrm{Var}(X)} \sqrt{\mathrm{Var}(Y)}} = 0 \; .\]

Yet, $X$ and $Y$ are not independent, since the marginal density of $Y$ is

\[\label{eq:Y-dist-marg} p(y) = \mathcal{N}(y; 0, 1) \; ,\]

but the conditional density of $Y$ given $X$ is

\[\label{eq:Y-dist-cond} p(y|x) = \left\{ \begin{array}{rl} 1/2 \; , & \text{if} \; y = -x \\ 1/2 \; , & \text{if} \; y = +x \\ 0 \; , & \text{otherwise} \end{array} \right. \; ,\]

thus violating the behavior of probability under independence:

\[\label{eq:X-Y-dep} p(Y) \neq p(Y|X) \; .\]

Therefore, $X$ and $Y$ defined by \eqref{eq:X-Y} and \eqref{eq:V-W} constitute an example for two random variables that are normally distributed and uncorrelated, but not independent.

∎
Sources:

Metadata: ID: P473 | shortcut: norm-corrind | author: JoramSoch | date: 2024-10-04, 10:59.