Proof: Projection matrix and residual-forming matrix are idempotent
Index: The Book of Statistical Proofs ▷ Statistical Models ▷ Univariate normal data ▷ Multiple linear regression ▷ Idempotence of projection and residual-forming matrix
Metadata: ID: P135 | shortcut: mlr-idem | author: JoramSoch | date: 2020-07-22, 06:28.
Theorem: The projection matrix and the residual-forming matrix are idempotent:
\[\label{eq:P^2-R^2} \begin{split} P^2 &= P \\ R^2 &= R \; . \end{split}\]Proof:
1) The projection matrix for ordinary least squares is given by
\[\label{eq:P} P = X (X^\mathrm{T} X)^{-1} X^\mathrm{T} \; ,\]such that
\[\label{eq:P^2} \begin{split} P^2 &= X (X^\mathrm{T} X)^{-1} X^\mathrm{T} X (X^\mathrm{T} X)^{-1} X^\mathrm{T} \\ &= X (X^\mathrm{T} X)^{-1} X^\mathrm{T} \\ &\overset{\eqref{eq:P}}{=} P \; . \end{split}\]
2) The residual-forming matrix for ordinary least squares is given by
such that
\[\label{eq:R^2} \begin{split} R^2 &= (I_n - P) (I_n - P) \\ &= I_n - P - P + P^2 \\ &\overset{\eqref{eq:P^2}}{=} I_n - 2 P + P \\ &= I_n - P \\ &\overset{\eqref{eq:R}}{=} R \; . \end{split}\]∎
Sources: - Wikipedia (2020): "Projection matrix"; in: Wikipedia, the free encyclopedia, retrieved on 2020-07-22; URL: https://en.wikipedia.org/wiki/Projection_matrix#Properties.
Metadata: ID: P135 | shortcut: mlr-idem | author: JoramSoch | date: 2020-07-22, 06:28.