Index: The Book of Statistical Proofs ▷ General Theorems ▷ Probability theory ▷ Covariance ▷ Covariance matrix and correlation matrix

Theorem: Let $X$ be a random vector. Then, the covariance matrix $\Sigma_{XX}$ of $X$ can be expressed in terms of its correlation matrix $\mathrm{C}_{XX}$ as follows

\[\label{eq:covmat-corrmat} \Sigma_{XX} = \mathrm{D}_X \cdot \mathrm{C}_{XX} \cdot \mathrm{D}_X \; ,\]

where $\mathrm{D}_X$ is a diagonal matrix with the standard deviations of $X_1, \ldots, X_n$ as entries on the diagonal:

\[\label{eq:diagmat} \mathrm{D}_X = \mathrm{diag}(\sigma_{X_1},\ldots,\sigma_{X_n}) = \begin{bmatrix} \sigma_{X_1} & \ldots & 0 \\ \vdots & \ddots & \vdots \\ 0 & \ldots & \sigma_{X_n} \end{bmatrix} \; .\]

Proof: Reiterating \eqref{eq:covmat-corrmat} and applying \eqref{eq:diagmat}, we have:

\[\label{eq:covmat-corrmat-s1} \Sigma_{XX} = \begin{bmatrix} \sigma_{X_1} & \ldots & 0 \\ \vdots & \ddots & \vdots \\ 0 & \ldots & \sigma_{X_n} \end{bmatrix} \cdot \mathrm{C}_{XX} \cdot \begin{bmatrix} \sigma_{X_1} & \ldots & 0 \\ \vdots & \ddots & \vdots \\ 0 & \ldots & \sigma_{X_n} \end{bmatrix} \; .\]

Together with the definition of the correlation matrix, this gives

\[\label{eq:covmat-corrmat-s2} \begin{split} \Sigma_{XX} &= \begin{bmatrix} \sigma_{X_1} & \ldots & 0 \\ \vdots & \ddots & \vdots \\ 0 & \ldots & \sigma_{X_n} \end{bmatrix} \cdot \begin{bmatrix} \frac{\mathrm{E}\left[(X_1-\mathrm{E}[X_1]) (X_1-\mathrm{E}[X_1])\right]}{\sigma_{X_1} \, \sigma_{X_1}} & \ldots & \frac{\mathrm{E}\left[(X_1-\mathrm{E}[X_1]) (X_n-\mathrm{E}[X_n])\right]}{\sigma_{X_1} \, \sigma_{X_n}} \\ \vdots & \ddots & \vdots \\ \frac{\mathrm{E}\left[(X_n-\mathrm{E}[X_n]) (X_1-\mathrm{E}[X_1])\right]}{\sigma_{X_n} \, \sigma_{X_1}} & \ldots & \frac{\mathrm{E}\left[(X_n-\mathrm{E}[X_n]) (X_n-\mathrm{E}[X_n])\right]}{\sigma_{X_n} \, \sigma_{X_n}} \end{bmatrix} \cdot \begin{bmatrix} \sigma_{X_1} & \ldots & 0 \\ \vdots & \ddots & \vdots \\ 0 & \ldots & \sigma_{X_n} \end{bmatrix} \\ &= \begin{bmatrix} \frac{\sigma_{X_1} \cdot \mathrm{E}\left[(X_1-\mathrm{E}[X_1]) (X_1-\mathrm{E}[X_1])\right]}{\sigma_{X_1} \, \sigma_{X_1}} & \ldots & \frac{\sigma_{X_1} \cdot \mathrm{E}\left[(X_1-\mathrm{E}[X_1]) (X_n-\mathrm{E}[X_n])\right]}{\sigma_{X_1} \, \sigma_{X_n}} \\ \vdots & \ddots & \vdots \\ \frac{\sigma_{X_n} \cdot \mathrm{E}\left[(X_n-\mathrm{E}[X_n]) (X_1-\mathrm{E}[X_1])\right]}{\sigma_{X_n} \, \sigma_{X_1}} & \ldots & \frac{\sigma_{X_n} \cdot \mathrm{E}\left[(X_n-\mathrm{E}[X_n]) (X_n-\mathrm{E}[X_n])\right]}{\sigma_{X_n} \, \sigma_{X_n}} \end{bmatrix} \cdot \begin{bmatrix} \sigma_{X_1} & \ldots & 0 \\ \vdots & \ddots & \vdots \\ 0 & \ldots & \sigma_{X_n} \end{bmatrix} \\ &= \begin{bmatrix} \frac{\sigma_{X_1} \cdot \mathrm{E}\left[(X_1-\mathrm{E}[X_1]) (X_1-\mathrm{E}[X_1])\right] \cdot \sigma_{X_1}}{\sigma_{X_1} \, \sigma_{X_1}} & \ldots & \frac{\sigma_{X_1} \cdot \mathrm{E}\left[(X_1-\mathrm{E}[X_1]) (X_n-\mathrm{E}[X_n])\right] \cdot \sigma_{X_n}}{\sigma_{X_1} \, \sigma_{X_n}} \\ \vdots & \ddots & \vdots \\ \frac{\sigma_{X_n} \cdot \mathrm{E}\left[(X_n-\mathrm{E}[X_n]) (X_1-\mathrm{E}[X_1])\right] \cdot \sigma_{X_1}}{\sigma_{X_n} \, \sigma_{X_1}} & \ldots & \frac{\sigma_{X_n} \cdot \mathrm{E}\left[(X_n-\mathrm{E}[X_n]) (X_n-\mathrm{E}[X_n])\right] \cdot \sigma_{X_n}}{\sigma_{X_n} \, \sigma_{X_n}} \end{bmatrix} \\ &= \begin{bmatrix} \mathrm{E}\left[ (X_1-\mathrm{E}[X_1]) (X_1-\mathrm{E}[X_1]) \right] & \ldots & \mathrm{E}\left[ (X_1-\mathrm{E}[X_1]) (X_n-\mathrm{E}[X_n]) \right] \\ \vdots & \ddots & \vdots \\ \mathrm{E}\left[ (X_n-\mathrm{E}[X_n]) (X_1-\mathrm{E}[X_1]) \right] & \ldots & \mathrm{E}\left[ (X_n-\mathrm{E}[X_n]) (X_n-\mathrm{E}[X_n]) \right] \end{bmatrix} \end{split}\]

which is nothing else than the definition of the covariance matrix.

∎
Sources:

Metadata: ID: P121 | shortcut: covmat-corrmat | author: JoramSoch | date: 2020-06-06, 06:02.