Index: The Book of Statistical ProofsGeneral Theorems ▷ Probability theory ▷ Variance ▷ Law of total variance

Theorem: (law of total variance, also called “conditional variance formula”) Let $X$ and $Y$ be random variables defined on the same probability space and assume that the variance of $Y$ is finite. Then, the sum of the expectation of the conditional variance and the variance of the conditional expectation of $Y$ given $X$ is equal to the variance of $Y$:

$\label{eq:var-tot} \mathrm{Var}(Y) = \mathrm{E}[\mathrm{Var}(Y \vert X)] + \mathrm{Var}[\mathrm{E}(Y \vert X)] \; .$

Proof: The variance can be decomposed into expected values as follows:

$\label{eq:var-tot-s1} \mathrm{Var}(Y) = \mathrm{E}(Y^2) - \mathrm{E}(Y)^2 \; .$

This can be rearranged into:

$\label{eq:var-tot-s2} \mathrm{E}(Y^2) = \mathrm{Var}(Y) + \mathrm{E}(Y)^2 \; .$

Applying the law of total expectation, we have:

$\label{eq:var-tot-s3} \mathrm{E}(Y^2) = \mathrm{E}\left[ \mathrm{Var}(Y \vert X) + \mathrm{E}(Y \vert X)^2 \right] \; .$

Now subtract the second term from \eqref{eq:var-tot-s1}:

$\label{eq:var-tot-s4} \mathrm{E}(Y^2) - \mathrm{E}(Y)^2 = \mathrm{E}\left[ \mathrm{Var}(Y \vert X) + \mathrm{E}(Y \vert X)^2 \right] - \mathrm{E}(Y)^2 \; .$

Again applying the law of total expectation, we have:

$\label{eq:var-tot-s5} \mathrm{E}(Y^2) - \mathrm{E}(Y)^2 = \mathrm{E}\left[ \mathrm{Var}(Y \vert X) + \mathrm{E}(Y \vert X)^2 \right] - \mathrm{E}\left[ \mathrm{E}(Y \vert X) \right]^2 \; .$

With the linearity of the expected value, the terms can be regrouped to give:

$\label{eq:var-tot-s6} \mathrm{E}(Y^2) - \mathrm{E}(Y)^2 = \mathrm{E}\left[ \mathrm{Var}(Y \vert X) \right] + \left( \mathrm{E}\left[ \mathrm{E}(Y \vert X)^2 \right] - \mathrm{E}\left[ \mathrm{E}(Y \vert X) \right]^2 \right) \; .$

Using the decomposition of variance into expected values, we finally have:

$\label{eq:var-tot-s7} \mathrm{Var}(Y) = \mathrm{E}[\mathrm{Var}(Y \vert X)] + \mathrm{Var}[\mathrm{E}(Y \vert X)] \; .$
Sources:

Metadata: ID: P292 | shortcut: var-tot | author: JoramSoch | date: 2021-11-26, 11:20.