Index: The Book of Statistical ProofsGeneral Theorems ▷ Probability theory ▷ Expected value ▷ Expectation of a quadratic form

Theorem: Let $X$ be an $n \times 1$ random vector with mean $\mu$ and covariance $\Sigma$ and let $A$ be a symmetric $n \times n$ matrix. Then, the expectation of the quadratic form $X^\mathrm{T} A X$ is

$\label{eq:mean-qf} \mathrm{E}\left[ X^\mathrm{T} A X \right] = \mu^\mathrm{T} A \mu + \mathrm{tr}(A \Sigma) \; .$

Proof: Note that $X^\mathrm{T} A X$ is a $1 \times 1$ matrix. We can therefore write

$\label{eq:mean-qf-s1} \mathrm{E}\left[ X^\mathrm{T} A X \right] = \mathrm{E}\left[ \mathrm{tr} \left( X^\mathrm{T} A X \right) \right] \; .$

Using the trace property $\mathrm{tr}(ABC) = \mathrm{tr}(BCA)$, this becomes

$\label{eq:mean-qf-s2} \mathrm{E}\left[ X^\mathrm{T} A X \right] = \mathrm{E}\left[ \mathrm{tr} \left( A X X^\mathrm{T} \right) \right] \; .$

Because mean and trace are linear operators, we have

$\label{eq:mean-qf-s3} \mathrm{E}\left[ X^\mathrm{T} A X \right] = \mathrm{tr} \left( A \; \mathrm{E}\left[ X X^\mathrm{T} \right] \right) \; .$ $\label{eq:covmat-mean} \mathrm{Cov}(X,X) = \mathrm{E}(X X^\mathrm{T}) - \mathrm{E}(X) \mathrm{E}(X)^\mathrm{T} \; ,$

such that the expected value of the quadratic form becomes

$\label{eq:mean-qf-s4} \mathrm{E}\left[ X^\mathrm{T} A X \right] = \mathrm{tr} \left( A \left[ \mathrm{Cov}(X,X) + \mathrm{E}(X) \mathrm{E}(X)^\mathrm{T} \right] \right) \; .$

Finally, applying mean and covariance of $X$, we have

$\label{eq:mean-qf-s5} \begin{split} \mathrm{E}\left[ X^\mathrm{T} A X \right] &= \mathrm{tr} \left( A \left[ \Sigma + \mu \mu^\mathrm{T} \right] \right) \\ &= \mathrm{tr} \left( A \Sigma + A \mu \mu^\mathrm{T} \right) \\ &= \mathrm{tr}(A \Sigma) + \mathrm{tr}(A \mu \mu^\mathrm{T}) \\ &= \mathrm{tr}(A \Sigma) + \mathrm{tr}(\mu^\mathrm{T} A \mu) \\ &= \mu^\mathrm{T} A \mu + \mathrm{tr}(A \Sigma) \; . \end{split}$
Sources:

Metadata: ID: P131 | shortcut: mean-qf | author: JoramSoch | date: 2020-07-13, 21:59.