Index: The Book of Statistical Proofs ▷ Model Selection ▷ Bayesian model selection ▷ Model evidence ▷ Variational Bayesian log model evidence

Definition: Let $m$ be a generative model with model parameters $\theta \in \Theta$ implying the likelihood function $p(y \vert \theta, m)$ and prior distribution $p(\theta \vert m)$. Moreover, assume an approximate posterior distribution $q(\theta)$. Then, the Variational Bayesian log model evidence, also referred to as the “variational free energy”, is defined as the expected logarithm of the likelihood function, divided by the approximate posterior:

\[\label{eq:vbLME} \mathrm{vbLME}(m) = \mathrm{F}_m[q(\theta)] = \int_{\Theta} q(\theta) \log \frac{p(\theta \vert y, m)}{q(\theta)} \, \mathrm{d}\theta \; .\]

The variational free energy can be decomposed into the difference between log model evidence and KL divergence of approximate from true posterior or, alternatively, as the difference of expected log-likelihood and KL divergence of approximate posterior from prior.

 
Sources:

Metadata: ID: D115 | shortcut: vblme | author: JoramSoch | date: 2020-11-25, 08:10.