Index: The Book of Statistical Proofs ▷ Model Selection ▷ Bayesian model selection ▷ Bayes factor ▷ Definition

Definition: Consider two competing generative models $m_1$ and $m_2$ for observed data $y$. Then the Bayes factor in favor $m_1$ over $m_2$ is the ratio of marginal likelihoods of $m_1$ and $m_2$:

\[\label{eq:BF} \text{BF}_{12} = \frac{p(y\mid m_1)}{p(y\mid m_2)}.\]

Note that by Bayes’ theorem, the ratio of posterior model probabilities (i.e., the posterior model odds) can be written as

\[\label{eq:odds} \frac{p(m_1 \mid y)}{p(m_2 \mid y)} = \frac{p(m_1)}{p(m_2)} \cdot \frac{p(y\mid m_1)}{p(y\mid m_2)},\]

or equivalently by \eqref{eq:BF},

\[\label{eq:odds2} \frac{p(m_1 \mid y)}{p(m_2 \mid y)} = \frac{p(m_1)}{p(m_2)} \cdot \text{BF}_{12}.\]

In other words, the Bayes factor can be viewed as the factor by which the prior model odds are updated (after observing data $y$) to posterior model odds – which is also expressed by Bayes’ rule.

 
Sources:

Metadata: ID: D92 | shortcut: bf | author: tomfaulkenberry | date: 2020-08-26, 12:00.