Proof: Covariance matrix of the sum of two random vectors
Index: The Book of Statistical Proofs ▷ General Theorems ▷ Probability theory ▷ Covariance ▷ Covariance matrix of a sum
Metadata: ID: P349 | shortcut: covmat-sum | author: JoramSoch | date: 2022-09-26, 10:37.
Theorem: The covariance matrix of the sum of two random vectors of the same dimension equals the sum of the covariances of those random vectors, plus the sum of their cross-covariances:
\[\label{eq:covmat-sum} \Sigma(X+Y) = \Sigma_{XX} + \Sigma_{YY} + \Sigma_{XY} + \Sigma_{YX} \; .\]Proof: The covariance matrix of $X$ can be expressed in terms of expected values as follows
\[\label{eq:covmat} \Sigma_{XX} = \Sigma(X) = \mathrm{E}\left[ (X-\mathrm{E}[X]) (X-\mathrm{E}[X])^\mathrm{T} \right]\]and the cross-covariance matrix of $X$ and $Y$ can similarly be written as
\[\label{eq:covmat-cross} \Sigma_{XY} = \Sigma(X,Y) = \mathrm{E}\left[ (X-\mathrm{E}[X]) (Y-\mathrm{E}[Y])^\mathrm{T} \right]\]Using this and the linearity of the expected value as well as the definitions of covariance matrix and cross-covariance matrix, we can derive \eqref{eq:covmat-sum} as follows:
\[\label{eq:covmat-sum-qed} \begin{split} \Sigma(X+Y) &\overset{\eqref{eq:covmat}}{=} \mathrm{E}\left[ ([X+Y]-\mathrm{E}[X+Y]) ([X+Y]-\mathrm{E}[X+Y])^\mathrm{T} \right] \\ &= \mathrm{E}\left[ ([X-\mathrm{E}(X)] + [Y-\mathrm{E}(Y)]) ([X-\mathrm{E}(X)] + [Y-\mathrm{E}(Y)])^\mathrm{T} \right] \\ &= \mathrm{E}\left[ (X-\mathrm{E}[X]) (X-\mathrm{E}[X])^\mathrm{T} + (X-\mathrm{E}[X]) (Y-\mathrm{E}[Y])^\mathrm{T} + (Y-\mathrm{E}[Y]) (X-\mathrm{E}[X])^\mathrm{T} + (Y-\mathrm{E}[Y]) (Y-\mathrm{E}[Y])^\mathrm{T} \right] \\ &= \mathrm{E}\left[ (X-\mathrm{E}[X]) (X-\mathrm{E}[X])^\mathrm{T} \right] + \mathrm{E}\left[ (X-\mathrm{E}[X]) (Y-\mathrm{E}[Y])^\mathrm{T} \right] + \mathrm{E}\left[ (Y-\mathrm{E}[Y]) (X-\mathrm{E}[X])^\mathrm{T} \right] + \mathrm{E}\left[ (Y-\mathrm{E}[Y]) (Y-\mathrm{E}[Y])^\mathrm{T} \right] \\ &\overset{\eqref{eq:covmat}}{=} \Sigma_{XX} + \Sigma_{YY} + \mathrm{E}\left[ (X-\mathrm{E}[X]) (Y-\mathrm{E}[Y])^\mathrm{T} \right] + \mathrm{E}\left[ (Y-\mathrm{E}[Y]) (X-\mathrm{E}[X])^\mathrm{T} \right] \\ &\overset{\eqref{eq:covmat-cross}}{=} \Sigma_{XX} + \Sigma_{YY} + \Sigma_{XY} + \Sigma_{YX} \; . \end{split}\]∎
Sources: - Wikipedia (2022): "Covariance matrix"; in: Wikipedia, the free encyclopedia, retrieved on 2022-09-26; URL: https://en.wikipedia.org/wiki/Covariance_matrix#Basic_properties.
Metadata: ID: P349 | shortcut: covmat-sum | author: JoramSoch | date: 2022-09-26, 10:37.