Index: The Book of Statistical Proofs ▷ Statistical Models ▷ Univariate normal data ▷ Analysis of variance ▷ F-statistic in terms of OLS estimates

Theorem: Given the one-way analysis of variance assumption

\[\label{eq:anova1} y_{ij} = \mu_i + \varepsilon_{ij}, \; \varepsilon_{ij} \overset{\mathrm{i.i.d.}}{\sim} \mathcal{N}(0, \sigma^2),\]

1) the F-statistic for the main effect can be expressed in terms of ordinary least squares parameter estimates as

\[\label{eq:anova1-fols-v1} F = \frac{\frac{1}{k-1} \sum_{i=1}^{k} n_i (\hat{\mu}_i - \bar{y})^2}{\frac{1}{n-k} \sum_{i=1}^{k} \sum_{j=1}^{n_i} (y_{ij} - \hat{\mu}_i)^2}\]

2) or, when using the reparametrized version of one-way ANOVA, the F-statistic can be expressed as

\[\label{eq:anova1-fols-v2} F = \frac{\frac{1}{k-1} \sum_{i=1}^{k} n_i \hat{\delta}_i^2}{\frac{1}{n-k} \sum_{i=1}^{k} \sum_{j=1}^{n_i} (y_{ij} - \hat{\mu} - \hat{\delta}_i)^2} \; .\]

Proof: The F-statistic for the main effect in one-way ANOVA is given in terms of the sample means as

\[\label{eq:anova1-f} F = \frac{\frac{1}{k-1} \sum_{i=1}^{k} n_i (\bar{y}_i - \bar{y})^2}{\frac{1}{n-k} \sum_{i=1}^{k} \sum_{j=1}^{n_i} (y_{ij} - \bar{y}_i)^2}\]

where $\bar{y}_i$ is the average of all values $y_{ij}$ from category $i$ and $\bar{y}$ is the grand mean of all values $y_{ij}$ from all categories $i = 1, \ldots, k$.

1) The ordinary least squares estimates for one-way ANOVA are

\[\label{eq:anova1-ols} \hat{\mu}_i = \bar{y}_i \; ,\]

such that

\[\label{eq:anova1-fols-v1-qed} F = \frac{\frac{1}{k-1} \sum_{i=1}^{k} n_i (\hat{\mu}_i - \bar{y})^2}{\frac{1}{n-k} \sum_{i=1}^{k} \sum_{j=1}^{n_i} (y_{ij} - \hat{\mu}_i)^2} \; .\]

2) The OLS estimates for reparametrized one-way ANOVA are

\[\label{eq:anova1-repara-ols} \begin{split} \hat{\mu} &= \bar{y} \\ \hat{\delta}_i &= \bar{y}_i - \bar{y} \; , \end{split}\]

such that

\[\label{eq:anova1-fols-v2-qed} F = \frac{\frac{1}{k-1} \sum_{i=1}^{k} n_i \hat{\delta}_i^2}{\frac{1}{n-k} \sum_{i=1}^{k} \sum_{j=1}^{n_i} (y_{ij} - \hat{\mu} - \hat{\delta}_i)^2} \; .\]
∎
Sources:

Metadata: ID: P377 | shortcut: anova1-fols | author: JoramSoch | date: 2022-11-15, 17:35.