Proof: F-statistic in terms of ordinary least squares estimates in one-way analysis of variance
Index: The Book of Statistical Proofs ▷ Statistical Models ▷ Univariate normal data ▷ Analysis of variance ▷ F-statistic in terms of OLS estimates
Metadata: ID: P377 | shortcut: anova1-fols | author: JoramSoch | date: 2022-11-15, 17:35.
Theorem: Given the one-way analysis of variance assumption
\[\label{eq:anova1} y_{ij} = \mu_i + \varepsilon_{ij}, \; \varepsilon_{ij} \overset{\mathrm{i.i.d.}}{\sim} \mathcal{N}(0, \sigma^2),\]1) the F-statistic for the main effect can be expressed in terms of ordinary least squares parameter estimates as
\[\label{eq:anova1-fols-v1} F = \frac{\frac{1}{k-1} \sum_{i=1}^{k} n_i (\hat{\mu}_i - \bar{y})^2}{\frac{1}{n-k} \sum_{i=1}^{k} \sum_{j=1}^{n_i} (y_{ij} - \hat{\mu}_i)^2}\]2) or, when using the reparametrized version of one-way ANOVA, the F-statistic can be expressed as
\[\label{eq:anova1-fols-v2} F = \frac{\frac{1}{k-1} \sum_{i=1}^{k} n_i \hat{\delta}_i^2}{\frac{1}{n-k} \sum_{i=1}^{k} \sum_{j=1}^{n_i} (y_{ij} - \hat{\mu} - \hat{\delta}_i)^2} \; .\]Proof: The F-statistic for the main effect in one-way ANOVA is given in terms of the sample means as
\[\label{eq:anova1-f} F = \frac{\frac{1}{k-1} \sum_{i=1}^{k} n_i (\bar{y}_i - \bar{y})^2}{\frac{1}{n-k} \sum_{i=1}^{k} \sum_{j=1}^{n_i} (y_{ij} - \bar{y}_i)^2}\]where $\bar{y}_i$ is the average of all values $y_{ij}$ from category $i$ and $\bar{y}$ is the grand mean of all values $y_{ij}$ from all categories $i = 1, \ldots, k$.
1) The ordinary least squares estimates for one-way ANOVA are
\[\label{eq:anova1-ols} \hat{\mu}_i = \bar{y}_i \; ,\]such that
\[\label{eq:anova1-fols-v1-qed} F = \frac{\frac{1}{k-1} \sum_{i=1}^{k} n_i (\hat{\mu}_i - \bar{y})^2}{\frac{1}{n-k} \sum_{i=1}^{k} \sum_{j=1}^{n_i} (y_{ij} - \hat{\mu}_i)^2} \; .\]2) The OLS estimates for reparametrized one-way ANOVA are
\[\label{eq:anova1-repara-ols} \begin{split} \hat{\mu} &= \bar{y} \\ \hat{\delta}_i &= \bar{y}_i - \bar{y} \; , \end{split}\]such that
\[\label{eq:anova1-fols-v2-qed} F = \frac{\frac{1}{k-1} \sum_{i=1}^{k} n_i \hat{\delta}_i^2}{\frac{1}{n-k} \sum_{i=1}^{k} \sum_{j=1}^{n_i} (y_{ij} - \hat{\mu} - \hat{\delta}_i)^2} \; .\]∎
Sources: Metadata: ID: P377 | shortcut: anova1-fols | author: JoramSoch | date: 2022-11-15, 17:35.