Expectation of a quadratic form

Index: The Book of Statistical Proofs ▷ General Theorems ▷ Probability theory ▷ Expected value ▷ Expectation of a quadratic form

Theorem: Let $X$ be an $n \times 1$ random vector with mean $\mu$ and covariance $\Sigma$ and let $A$ be a symmetric $n \times n$ matrix. Then, the expectation of the quadratic form $X^\mathrm{T} A X$ is

\[\label{eq:mean-qf} \mathrm{E}\left[ X^\mathrm{T} A X \right] = \mu^\mathrm{T} A \mu + \mathrm{tr}(A \Sigma) \; .\]

Proof: Note that $X^\mathrm{T} A X$ is a $1 \times 1$ matrix. We can therefore write

\[\label{eq:mean-qf-s1} \mathrm{E}\left[ X^\mathrm{T} A X \right] = \mathrm{E}\left[ \mathrm{tr} \left( X^\mathrm{T} A X \right) \right] \; .\]

Using the trace property $\mathrm{tr}(ABC) = \mathrm{tr}(BCA)$, this becomes

\[\label{eq:mean-qf-s2} \mathrm{E}\left[ X^\mathrm{T} A X \right] = \mathrm{E}\left[ \mathrm{tr} \left( A X X^\mathrm{T} \right) \right] \; .\]

Because mean and trace are linear operators, we have

\[\label{eq:mean-qf-s3} \mathrm{E}\left[ X^\mathrm{T} A X \right] = \mathrm{tr} \left( A \; \mathrm{E}\left[ X X^\mathrm{T} \right] \right) \; .\]

Note that the covariance matrix can be partitioned into expected values

\[\label{eq:covmat-mean} \mathrm{Cov}(X,X) = \mathrm{E}(X X^\mathrm{T}) - \mathrm{E}(X) \mathrm{E}(X)^\mathrm{T} \; ,\]

such that the expected value of the quadratic form becomes

\[\label{eq:mean-qf-s4} \mathrm{E}\left[ X^\mathrm{T} A X \right] = \mathrm{tr} \left( A \left[ \mathrm{Cov}(X,X) + \mathrm{E}(X) \mathrm{E}(X)^\mathrm{T} \right] \right) \; .\]

Finally, applying mean and covariance of $X$, we have

\[\label{eq:mean-qf-s5} \begin{split} \mathrm{E}\left[ X^\mathrm{T} A X \right] &= \mathrm{tr} \left( A \left[ \Sigma + \mu \mu^\mathrm{T} \right] \right) \\ &= \mathrm{tr} \left( A \Sigma + A \mu \mu^\mathrm{T} \right) \\ &= \mathrm{tr}(A \Sigma) + \mathrm{tr}(A \mu \mu^\mathrm{T}) \\ &= \mathrm{tr}(A \Sigma) + \mathrm{tr}(\mu^\mathrm{T} A \mu) \\ &= \mu^\mathrm{T} A \mu + \mathrm{tr}(A \Sigma) \; . \end{split}\]

∎

Sources:

Kendrick, David (1981): "Expectation of a quadratic form"; in: Stochastic Control for Economic Models, pp. 170-171.
Wikipedia (2020): "Multivariate random variable"; in: Wikipedia, the free encyclopedia, retrieved on 2020-07-13; URL: https://en.wikipedia.org/wiki/Multivariate_random_variable#Expectation_of_a_quadratic_form.
Halvorsen, Kjetil B. (2012): "Expected value and variance of trace function"; in: StackExchange CrossValidated, retrieved on 2020-07-13; URL: https://stats.stackexchange.com/questions/34477/expected-value-and-variance-of-trace-function.
Sarwate, Dilip (2013): "Expected Value of Quadratic Form"; in: StackExchange CrossValidated, retrieved on 2020-07-13; URL: https://stats.stackexchange.com/questions/48066/expected-value-of-quadratic-form.

Metadata: ID: P131 | shortcut: mean-qf | author: JoramSoch | date: 2020-07-13, 21:59.