Index: The Book of Statistical ProofsStatistical Models ▷ Categorical data ▷ Binomial observations ▷ Conjugate prior distribution

Theorem: Let $y$ be the number of successes resulting from $n$ independent trials with unknown success probability $p$, such that $y$ follows a binomial distribution:

$\label{eq:Bin} y \sim \mathrm{Bin}(n,p) \; .$

Then, the conjugate prior for the model parameter $p$ is a beta distribution:

$\label{eq:Beta} \mathrm{p}(p) = \mathrm{Bet}(p; \alpha_0, \beta_0) \; .$

Proof: With the probability mass function of the binomial distribution, the likelihood function implied by \eqref{eq:Bin} is given by

$\label{eq:Bin-LF} \mathrm{p}(y|p) = {n \choose y} \, p^y \, (1-p)^{n-y} \; .$

In other words, the likelihood function is proportional to a power of $p$ times a power of $(1-p)$:

$\label{eq:Bin-LF-prop} \mathrm{p}(y|p) \propto p^y \, (1-p)^{n-y} \; .$

The same is true for a beta distribution over $p$

$\label{eq:Bin-prior-s1} \mathrm{p}(p) = \mathrm{Bet}(p; \alpha_0, \beta_0)$ $\label{eq:Bin-prior-s2} \mathrm{p}(p) = \frac{1}{B(\alpha_0,\beta_0)} \, p^{\alpha_0-1} \, (1-p)^{\beta_0-1}$

exhibits the same proportionality

$\label{eq:Bin-prior-s3} \mathrm{p}(p) \propto p^{\alpha_0-1} \, (1-p)^{\beta_0-1}$

and is therefore conjugate relative to the likelihood.

Sources:

Metadata: ID: P29 | shortcut: bin-prior | author: JoramSoch | date: 2020-01-23, 23:38.