Index: The Book of Statistical Proofs ▷ Statistical Models ▷ Count data ▷ Binomial observations ▷ Conjugate prior distribution

Theorem: Let $y$ be the number of successes resulting from $n$ independent trials with unknown success probability $p$, such that $y$ follows a binomial distribution:

\[\label{eq:Bin} y \sim \mathrm{Bin}(n,p) \; .\]

Then, the conjugate prior for the model parameter $p$ is a beta distribution:

\[\label{eq:Beta} \mathrm{p}(p) = \mathrm{Bet}(p; \alpha_0, \beta_0) \; .\]

Proof: With the probability mass function of the binomial distribution, the likelihood function implied by \eqref{eq:Bin} is given by

\[\label{eq:Bin-LF} \mathrm{p}(y|p) = {n \choose y} \, p^y \, (1-p)^{n-y} \; .\]

In other words, the likelihood function is proportional to a power of $p$ times a power of $(1-p)$:

\[\label{eq:Bin-LF-prop} \mathrm{p}(y|p) \propto p^y \, (1-p)^{n-y} \; .\]

The same is true for a beta distribution over $p$

\[\label{eq:Bin-prior-s1} \mathrm{p}(p) = \mathrm{Bet}(p; \alpha_0, \beta_0)\]

the probability density function of which

\[\label{eq:Bin-prior-s2} \mathrm{p}(p) = \frac{1}{B(\alpha_0,\beta_0)} \, p^{\alpha_0-1} \, (1-p)^{\beta_0-1}\]

exhibits the same proportionality

\[\label{eq:Bin-prior-s3} \mathrm{p}(p) \propto p^{\alpha_0-1} \, (1-p)^{\beta_0-1}\]

and is therefore conjugate relative to the likelihood.

∎
Sources:

Metadata: ID: P29 | shortcut: bin-prior | author: JoramSoch | date: 2020-01-23, 23:38.