Index: The Book of Statistical Proofs ▷ Model Selection ▷ Bayesian model selection ▷ Bayes factor ▷ Derivation of the group log Bayes factor

Theorem: Let there be two generative models $m_1$ and $m_2$ for a set of $N$ conditionally independent data sets $Y = \left\lbrace y_1, \ldots, y_N \right\rbrace$ with model evidences $p(y_i \vert m_1)$ and $p(y_i \vert m_2)$ for $i = 1,\ldots,N$. Then, the group log Bayes factor

\[\label{eq:glbf-gbf} \mathrm{GLBF}_{12} = \log \mathrm{GBF}_{12}\]

can be expressed as

\[\label{eq:glbf-lme} \mathrm{GLBF}_{12} = \sum_{i=1}^N \left[ \log p(y_i \vert m_1) - \log p(y_i \vert m_2) \right] \; .\]

or

\[\label{eq:glbf-lbf} \mathrm{GLBF}_{12} = \sum_{i=1}^N \mathrm{LBF}_{12}^{(i)}\]

where $\mathrm{LBF}_{12}^{(i)}$ is the log Bayes factor for $y_i$ in favor of $m_1$ over $m_2$.

Proof:

1) The group Bayes factor for the “group data set” $Y$ is equal to

\[\label{eq:gbf-me} \mathrm{GBF}_{12} = \prod_{i=1}^N \frac{p(y_i \vert m_1)}{p(y_i \vert m_2)} \; .\]

Logarithmizing both sides of \eqref{eq:gbf-me}, we get:

\[\label{eq:glbf-lme-qed} \begin{split} \log \mathrm{GBF}_{12} &= \log \prod_{i=1}^N \frac{p(y_i \vert m_1)}{p(y_i \vert m_2)} \\ \mathrm{GLBF}_{12} &= \sum_{i=1}^N \log \frac{p(y_i \vert m_1)}{p(y_i \vert m_2)} \\ \mathrm{GLBF}_{12} &= \sum_{i=1}^N \left[ \log p(y_i \vert m_1) - \log p(y_i \vert m_2) \right] \; . \end{split}\]

2) The log Bayes factor for a “single data set” $y_i$ evaluates to

\[\label{eq:lbf-lme-yi} \mathrm{LBF}_{12}^{(i)} = \log p(y_i \vert m_1) - \log p(y_i \vert m_2)\]

and substituting \eqref{eq:lbf-lme-yi} into \eqref{eq:glbf-lme-qed}, we obtain:

\[\label{eq:glbf-lbf-qed} \mathrm{GLBF}_{12} = \sum_{i=1}^N \mathrm{LBF}_{12}^{(i)} \; .\]
∎
Sources:

Metadata: ID: P544 | shortcut: glbf-der | author: JoramSoch | date: 2026-06-14, 16:13.