Index: The Book of Statistical Proofs ▷ Model Selection ▷ Bayesian model selection ▷ Model evidence ▷ Variational Bayesian log model evidence

Definition: Let $m$ be a generative model with model parameters $\theta$ implying the likelihood function $p(y \vert \theta, m)$ and prior distribution $p(\theta \vert m)$. Moreover, assume an approximate posterior distribution $q(\theta)$. Then, the Variational Bayesian log model evidence, also referred to as the “negative free energy”, is the expectation of the log-likelihood function with respect to the approximate posterior, minus the Kullback-Leibler divergence between approximate posterior and the prior distribution:

\[\label{eq:vbLME} \mathrm{vbLME}(m) = \left\langle \log p(y \vert \theta, m) \right\rangle_{q(\theta)} - \mathrm{KL}\left[q(\theta) || p(\theta \vert m)\right]\]

where

\[\label{eq:ELL} \left\langle \log p(y \vert \theta, m) \right\rangle_{q(\theta)} = \int q(\theta) \log p(y \vert \theta, m) \, \mathrm{d}\theta\]

and

\[\label{eq:KL} \mathrm{KL}\left[q(\theta) || p(\theta \vert m)\right] = \int q(\theta) \log \frac{q(\theta)}{p(\theta \vert m)} \, \mathrm{d}\theta \; .\]
 
Sources:

Metadata: ID: D115 | shortcut: vblme | author: JoramSoch | date: 2020-11-25, 08:10.