Proof: Law of total variance
Theorem: (law of total variance, also called “conditional variance formula”) Let $X$ and $Y$ be random variables defined on the same probability space and assume that the variance of $Y$ is finite. Then, the sum of the expectation of the conditional variance and the variance of the conditional expectation of $Y$ given $X$ is equal to the variance of $Y$:
\[\label{eq:var-tot} \mathrm{Var}(Y) = \mathrm{E}[\mathrm{Var}(Y \vert X)] + \mathrm{Var}[\mathrm{E}(Y \vert X)] \; .\]Proof: The variance can be decomposed into expected values as follows:
\[\label{eq:var-tot-s1} \mathrm{Var}(Y) = \mathrm{E}(Y^2) - \mathrm{E}(Y)^2 \; .\]This can be rearranged into:
\[\label{eq:var-tot-s2} \mathrm{E}(Y^2) = \mathrm{Var}(Y) + \mathrm{E}(Y)^2 \; .\]Applying the law of total expectation, we have:
\[\label{eq:var-tot-s3} \mathrm{E}(Y^2) = \mathrm{E}\left[ \mathrm{Var}(Y \vert X) + \mathrm{E}(Y \vert X)^2 \right] \; .\]Now subtract the second term from \eqref{eq:var-tot-s1}:
\[\label{eq:var-tot-s4} \mathrm{E}(Y^2) - \mathrm{E}(Y)^2 = \mathrm{E}\left[ \mathrm{Var}(Y \vert X) + \mathrm{E}(Y \vert X)^2 \right] - \mathrm{E}(Y)^2 \; .\]Again applying the law of total expectation, we have:
\[\label{eq:var-tot-s5} \mathrm{E}(Y^2) - \mathrm{E}(Y)^2 = \mathrm{E}\left[ \mathrm{Var}(Y \vert X) + \mathrm{E}(Y \vert X)^2 \right] - \mathrm{E}\left[ \mathrm{E}(Y \vert X) \right]^2 \; .\]With the linearity of the expected value, the terms can be regrouped to give:
\[\label{eq:var-tot-s6} \mathrm{E}(Y^2) - \mathrm{E}(Y)^2 = \mathrm{E}\left[ \mathrm{Var}(Y \vert X) \right] + \left( \mathrm{E}\left[ \mathrm{E}(Y \vert X)^2 \right] - \mathrm{E}\left[ \mathrm{E}(Y \vert X) \right]^2 \right) \; .\]Using the decomposition of variance into expected values, we finally have:
\[\label{eq:var-tot-s7} \mathrm{Var}(Y) = \mathrm{E}[\mathrm{Var}(Y \vert X)] + \mathrm{Var}[\mathrm{E}(Y \vert X)] \; .\]- Wikipedia (2021): "Law of total variance"; in: Wikipedia, the free encyclopedia, retrieved on 2021-11-26; URL: https://en.wikipedia.org/wiki/Law_of_total_variance#Proof.
Metadata: ID: P292 | shortcut: var-tot | author: JoramSoch | date: 2021-11-26, 11:20.