Index: The Book of Statistical Proofs ▷ Statistical Models ▷ Univariate normal data ▷ Simple linear regression ▷ Sum of residuals is zero

Theorem: In simple linear regression, the sum of the residuals is zero when estimated using ordinary least squares.

Proof: The residuals are defined as the estimated error terms

\[\label{eq:slr-res} \hat{\varepsilon}_i = y_i - \hat{\beta}_0 - \hat{\beta}_1 x_i\]

where $\hat{\beta}_0$ and $\hat{\beta}_1$ are parameter estimates obtained using ordinary least squares:

\[\label{eq:slr-ols} \hat{\beta}_0 = \bar{y} - \hat{\beta}_1 \bar{x} \quad \text{and} \quad \hat{\beta}_1 = \frac{s_{xy}}{s_x^2} \; .\]

With that, we can calculate the sum of the residuals:

\[\label{eq:slr-ressum} \begin{split} \sum_{i=1}^n \hat{\varepsilon}_i &= \sum_{i=1}^n (y_i - \hat{\beta}_0 - \hat{\beta}_1 x_i) \\ &= \sum_{i=1}^n (y_i - \bar{y} + \hat{\beta}_1 \bar{x} - \hat{\beta}_1 x_i) \\ &= \sum_{i=1}^n y_i - n \bar{y} + \hat{\beta}_1 n \bar{x} - \hat{\beta}_1 \sum_{i=1}^n x_i \\ &= n \bar{y} - n \bar{y} + \hat{\beta}_1 n \bar{x} - \hat{\beta}_1 n \bar{x} \\ &= 0 \; . \end{split}\]

Thus, the sum of the residuals is zero under ordinary least squares, if the model includes an intercept term $\beta_0$.

∎
Sources:

Metadata: ID: P276 | shortcut: slr-ressum | author: JoramSoch | date: 2021-10-27, 13:07.