Index: The Book of Statistical Proofs ▷ Model Selection ▷ Bayesian model selection ▷ Model evidence ▷ Derivation

Theorem: Let $p(y \vert \theta,m)$ be a likelihood function of a generative model $m$ for making inferences on model parameters $\theta$ given measured data $y$. Moreover, let $p(\theta \vert m)$ be a prior distribution on model parameters $\theta$ in the parameter space $\Theta$. Then, the model evidence (ME) can be expressed in terms of likelihood and prior as

\[\label{eq:ME-marg} \mathrm{ME}(m) = \int_{\Theta} p(y|\theta,m) \, p(\theta|m) \, \mathrm{d}\theta \; .\]

Proof: This a consequence of the law of marginal probability for continuous variables

\[\label{eq:prob-marg} p(y|m) = \int_{\Theta} p(y,\theta|m) \, \mathrm{d}\theta\]

and the law of conditional probability according to which

\[\label{eq:prob-cond} p(y,\theta|m) = p(y|\theta,m) \, p(\theta|m) \; .\]

Plugging \eqref{eq:prob-cond} into \eqref{eq:prob-marg}, we obtain:

\[\label{eq:ME-marg-qed} \mathrm{ME}(m) = p(y|m) = \int_{\Theta} p(y|\theta,m) \, p(\theta|m) \, \mathrm{d}\theta \; .\]
∎
Sources:

Metadata: ID: P367 | shortcut: me-der | author: JoramSoch | date: 2022-10-20, 10:11.