Proof: Expectation of the log Bayes factor for the univariate Gaussian with known variance
Theorem: Let
\[\label{eq:ugkv} y = \left\lbrace y_1, \ldots, y_n \right\rbrace, \quad y_i \sim \mathcal{N}(\mu, \sigma^2), \quad i = 1, \ldots, n\]be a univariate Gaussian data set with unknown mean $\mu$ and known variance $\sigma^2$. Moreover, assume two statistical models, one assuming that $\mu$ is zero (null model), the other imposing a normal distribution as the prior distribution on the model parameter $\mu$ (alternative):
\[\label{eq:UGkv-m01} \begin{split} m_0&: \; y_i \sim \mathcal{N}(\mu, \sigma^2), \; \mu = 0 \\ m_1&: \; y_i \sim \mathcal{N}(\mu, \sigma^2), \; \mu \sim \mathcal{N}(\mu_0, \lambda_0^{-1}) \; . \end{split}\]Then, under the null hypothesis that $m_0$ generated the data, the expectation of the log Bayes factor in favor of $m_1$ with $\mu_0 = 0$ against $m_0$ is
\[\label{eq:UGkv-LBF} \left\langle \mathrm{LBF}_{10} \right\rangle = \frac{1}{2} \log\left( \frac{\lambda_0}{\lambda_n} \right) + \frac{1}{2} \left( \frac{\lambda_n - \lambda_0}{\lambda_n} \right)\]where $\lambda_n$ is the posterior precision for the univariate Gaussian with known variance.
Proof: The log Bayes factor for the univariate Gaussian with known variance is
\[\label{eq:UGkv-LBF-m10-s1} \mathrm{LBF}_{10} = \frac{1}{2} \log\left( \frac{\lambda_0}{\lambda_n} \right) - \frac{1}{2} \left( \lambda_0 \mu_0^2 - \lambda_n \mu_n^2 \right)\]where the posterior hyperparameters are given by
\[\label{eq:UGkv-post-par} \begin{split} \mu_n &= \frac{\lambda_0 \mu_0 + \tau n \bar{y}}{\lambda_0 + \tau n} \\ \lambda_n &= \lambda_0 + \tau n \end{split}\]with the sample mean $\bar{y}$ and the inverse variance or precision $\tau = 1/\sigma^2$. Plugging $\mu_n$ from \eqref{eq:UGkv-post-par} into \eqref{eq:UGkv-LBF-m10-s1}, we obtain:
\[\label{eq:UGkv-LBF-m10-s2} \begin{split} \mathrm{LBF}_{10} &= \frac{1}{2} \log\left( \frac{\lambda_0}{\lambda_n} \right) - \frac{1}{2} \left( \lambda_0 \mu_0^2 - \lambda_n \, \frac{(\lambda_0 \mu_0 + \tau n \bar{y})^2}{\lambda_n^2} \right) \\ &= \frac{1}{2} \log\left( \frac{\lambda_0}{\lambda_n} \right) - \frac{1}{2} \left( \lambda_0 \mu_0^2 - \frac{1}{\lambda_n} (\lambda_0^2 \mu_0^2 - 2 \tau n \lambda_0 \mu_0 \bar{y} + \tau^2 (n \bar{y})^2) \right) \end{split}\]Because $m_1$ uses a zero-mean prior distribution with prior mean $\mu_0 = 0$ per construction, the log Bayes factor simplifies to:
\[\label{eq:UGkv-LBF-m10-s3} \mathrm{LBF}_{10} = \frac{1}{2} \log\left( \frac{\lambda_0}{\lambda_n} \right) + \frac{1}{2} \left( \frac{\tau^2 (n \bar{y})^2}{\lambda_n} \right) \; .\]From \eqref{eq:ugkv}, we know that the data are distributed as $y_i \sim \mathcal{N}(\mu, \sigma^2)$, such that we can derive the expectation of $(n \bar{y})^2$ as follows:
\[\label{eq:UGkv-E(ny2)} \begin{split} \left\langle (n \bar{y})^2 \right\rangle = \left\langle \sum_{i=1}^n \sum_{j=1}^n y_i y_j \right\rangle &= \left\langle n y_i^2 + (n^2-n) [y_i y_j]_{i \neq j} \right\rangle \\ &= n (\mu^2 + \sigma^2) + (n^2 - n) \mu^2 \\ &= n^2 \mu^2 + n \sigma^2 \; . \end{split}\]Applying this expected value to \eqref{eq:UGkv-LBF-m10-s3}, the expected LBF emerges as:
\[\label{eq:UGkv-LBF-m10-s4} \begin{split} \left\langle \mathrm{LBF}_{10} \right\rangle &= \frac{1}{2} \log\left( \frac{\lambda_0}{\lambda_n} \right) + \frac{1}{2} \left( \frac{\tau^2 (n^2 \mu^2 + n \sigma^2)}{\lambda_n} \right) \\ &= \frac{1}{2} \log\left( \frac{\lambda_0}{\lambda_n} \right) + \frac{1}{2} \left( \frac{(\tau n \mu)^2 + \tau n}{\lambda_n} \right) \end{split}\]Under the null hypothesis that $m_0$ generated the data, the unknown mean is $\mu = 0$, such that the log Bayes factor further simplifies to:
\[\label{eq:UGkv-LBF-m10-s5} \left\langle \mathrm{LBF}_{10} \right\rangle = \frac{1}{2} \log\left( \frac{\lambda_0}{\lambda_n} \right) + \frac{1}{2} \left( \frac{\tau n}{\lambda_n} \right) \; .\]Finally, plugging $\lambda_n$ from \eqref{eq:UGkv-post-par} into \eqref{eq:UGkv-LBF-m10-s5}, we obtain:
\[\label{eq:UGkv-LBF-m10-s6} \left\langle \mathrm{LBF}_{10} \right\rangle = \frac{1}{2} \log\left( \frac{\lambda_0}{\lambda_n} \right) + \frac{1}{2} \left( \frac{\lambda_n - \lambda_0}{\lambda_n} \right) \; .\]Metadata: ID: P216 | shortcut: ugkv-lbfmean | author: JoramSoch | date: 2021-03-24, 10:03.