Index: The Book of Statistical Proofs ▷ General Theorems ▷ Probability theory ▷ Covariance ▷ Law of total covariance

Theorem: (law of total covariance, also called “conditional covariance formula”) Let $X$, $Y$ and $Z$ be random variables defined on the same probability space and assume that the covariance of $X$ and $Y$ is finite. Then, the sum of the expectation of the conditional covariance and the covariance of the conditional expectations of $X$ and $Y$ given $Z$ is equal to the covariance of $X$ and $Y$:

\[\label{eq:cov-tot} \mathrm{Cov}(X,Y) = \mathrm{E}[\mathrm{Cov}(X,Y \vert Z)] + \mathrm{Cov}[\mathrm{E}(X \vert Z),\mathrm{E}(Y \vert Z)] \; .\]

Proof: The covariance can be decomposed into expected values as follows:

\[\label{eq:cov-tot-s1} \mathrm{Cov}(X,Y) = \mathrm{E}(XY) - \mathrm{E}(X) \mathrm{E}(Y) \; .\]

Then, conditioning on $Z$ and applying the law of total expectation, we have:

\[\label{eq:cov-tot-s2} \mathrm{Cov}(X,Y) = \mathrm{E}\left[ \mathrm{E}(XY \vert Z) \right] - \mathrm{E}\left[ \mathrm{E}(X \vert Z ) \right] \mathrm{E}\left[ \mathrm{E}(Y \vert Z) \right] \; .\]

Applying the decomposition of covariance into expected values to the first term gives:

\[\label{eq:cov-tot-s3} \mathrm{Cov}(X,Y) = \mathrm{E}\left[ \mathrm{Cov}(X,Y \vert Z) + \mathrm{E}(X \vert Z) \mathrm{E}(Y \vert Z) \right] - \mathrm{E}\left[ \mathrm{E}(X \vert Z ) \right] \mathrm{E}\left[ \mathrm{E}(Y \vert Z) \right] \; .\]

With the linearity of the expected value, the terms can be regrouped to give:

\[\label{eq:cov-tot-s4} \mathrm{Cov}(X,Y) = \mathrm{E}\left[ \mathrm{Cov}(X,Y \vert Z) \right] + \left( \mathrm{E}\left[ \mathrm{E}(X \vert Z) \mathrm{E}(Y \vert Z) \right] - \mathrm{E}\left[ \mathrm{E}(X \vert Z ) \right] \mathrm{E}\left[ \mathrm{E}(Y \vert Z) \right] \right) \; .\]

Once more using the decomposition of covariance into expected values, we finally have:

\[\label{eq:var-tot-s5} \mathrm{Cov}(X,Y) = \mathrm{E}[\mathrm{Cov}(X,Y \vert Z)] + \mathrm{Cov}[\mathrm{E}(X \vert Z),\mathrm{E}(Y \vert Z)] \; .\]
∎
Sources:

Metadata: ID: P293 | shortcut: cov-tot | author: JoramSoch | date: 2021-11-26, 11:38.