Index: The Book of Statistical Proofs ▷ General Theorems ▷ Bayesian statistics ▷ Probabilistic modeling ▷ Posterior predictive distribution is marginal of joint likelihood

Theorem: The posterior predictive distribution is the marginal distribution of the joint likelihood of the new data $y_{\mathrm{new}}$, conditional on the measured data $y$:

\[\label{eq:postpred-jl} p(y_{\mathrm{new}} \vert y) = \int p(y_{\mathrm{new}}, \theta \vert y) \, \mathrm{d}\theta\]

Proof: The posterior predictive distribution is defined as the marginal distribution of new data $y_{\mathrm{new}}$, predicted based on the posterior distribution obtained from the measured data $y$:

\[\label{eq:post-pred-s1} p(y_{\mathrm{new}} \vert y) = \int p(y_{\mathrm{new}} \vert \theta) \, p(\theta \vert y) \, \mathrm{d}\theta \; .\]

We notice that $y_{\text{new}}$ is independent of $y$, so we can write:

\[\label{eq:post-pred-s2} p(y_{\mathrm{new}} \vert y) = \int p(y_{\mathrm{new}} \vert \theta, y) \, p(\theta \vert y) \, \mathrm{d}\theta \; .\]

By using the law of conditional probability, we can write the integrand as:

\[\label{eq:jl-post} p(y_{\text{new}} \vert \theta, y) \, p(\theta \vert y) = p(y_{\text{new}}, \theta \vert y)\]

This is the posterior joint likelihood. Thus, expression \eqref{eq:postpred-jl} can be written as:

\[\label{eq:postpred-marg-qed} p(y_{\mathrm{new}} \vert y) = \int p(y_{\mathrm{new}}, \theta \vert y) \, \mathrm{d}\theta \; .\]
∎
Sources:

Metadata: ID: P467 | shortcut: postpred-jl | author: aloctavodia | date: 2024-09-11, 15:30.