Definition: Bayes factor
Index:
The Book of Statistical Proofs ▷
Model Selection ▷
Bayesian model selection ▷
Bayes factor ▷
Definition
Sources:
Metadata: ID: D92 | shortcut: bf | author: tomfaulkenberry | date: 2020-08-26, 12:00.
Definition: Consider two competing generative models $m_1$ and $m_2$ for observed data $y$. Then the Bayes factor in favor $m_1$ over $m_2$ is the ratio of marginal likelihoods of $m_1$ and $m_2$:
\[\label{eq:BF} \text{BF}_{12} = \frac{p(y\mid m_1)}{p(y\mid m_2)}.\]Note that by Bayes’ theorem, the ratio of posterior model probabilities (i.e., the posterior model odds) can be written as
\[\label{eq:odds} \frac{p(m_1 \mid y)}{p(m_2 \mid y)} = \frac{p(m_1)}{p(m_2)} \cdot \frac{p(y\mid m_1)}{p(y\mid m_2)},\]or equivalently by \eqref{eq:BF},
\[\label{eq:odds2} \frac{p(m_1 \mid y)}{p(m_2 \mid y)} = \frac{p(m_1)}{p(m_2)} \cdot \text{BF}_{12}.\]In other words, the Bayes factor can be viewed as the factor by which the prior model odds are updated (after observing data $y$) to posterior model odds – which is also expressed by Bayes’ rule.
- Kass, Robert E. and Raftery, Adrian E. (1995): "Bayes Factors"; in: Journal of the American Statistical Association, vol. 90, no. 430, pp. 773-795; URL: https://dx.doi.org/10.1080/01621459.1995.10476572; DOI: 10.1080/01621459.1995.10476572.
Metadata: ID: D92 | shortcut: bf | author: tomfaulkenberry | date: 2020-08-26, 12:00.