Definition: Variational Bayesian log model evidence
Index:
The Book of Statistical Proofs ▷
Model Selection ▷
Bayesian model selection ▷
Model evidence ▷
Variational Bayesian log model evidence
Sources:
Metadata: ID: D115 | shortcut: vblme | author: JoramSoch | date: 2020-11-25, 08:10.
Definition: Let $m$ be a generative model with model parameters $\theta$ implying the likelihood function $p(y \vert \theta, m)$ and prior distribution $p(\theta \vert m)$. Moreover, assume an approximate posterior distribution $q(\theta)$. Then, the Variational Bayesian log model evidence, also referred to as the “negative free energy”, is the expectation of the log-likelihood function with respect to the approximate posterior, minus the Kullback-Leibler divergence between approximate posterior and the prior distribution:
\[\label{eq:vbLME} \mathrm{vbLME}(m) = \left\langle \log p(y \vert \theta, m) \right\rangle_{q(\theta)} - \mathrm{KL}\left[q(\theta) || p(\theta \vert m)\right]\]where
\[\label{eq:ELL} \left\langle \log p(y \vert \theta, m) \right\rangle_{q(\theta)} = \int q(\theta) \log p(y \vert \theta, m) \, \mathrm{d}\theta\]and
\[\label{eq:KL} \mathrm{KL}\left[q(\theta) || p(\theta \vert m)\right] = \int q(\theta) \log \frac{q(\theta)}{p(\theta \vert m)} \, \mathrm{d}\theta \; .\]- Wikipedia (2020): "Variational Bayesian methods"; in: Wikipedia, the free encyclopedia, retrieved on 2020-11-25; URL: https://en.wikipedia.org/wiki/Variational_Bayesian_methods#Evidence_lower_bound.
- Penny W, Flandin G, Trujillo-Barreto N (2007): "Bayesian Comparison of Spatially Regularised General Linear Models"; in: Human Brain Mapping, vol. 28, pp. 275–293, eqs. 2-9; URL: https://onlinelibrary.wiley.com/doi/full/10.1002/hbm.20327; DOI: 10.1002/hbm.20327.
Metadata: ID: D115 | shortcut: vblme | author: JoramSoch | date: 2020-11-25, 08:10.