Index: The Book of Statistical Proofs ▷ Statistical Models ▷ Count data ▷ Binomial observations ▷ Maximum-a-posteriori estimation

Theorem: Let $y$ be the number of successes resulting from $n$ independent trials with unknown success probability $p$, such that $y$ follows a binomial distribution:

\[\label{eq:Bin} y \sim \mathrm{Bin}(n,p) \; .\]

Moreover, assume a beta prior distribution over the model parameter $p$:

\[\label{eq:Bin-prior} \mathrm{p}(p) = \mathrm{Bet}(p; \alpha_0, \beta_0) \; .\]

Then, the maximum-a-posteriori estimate of $p$ is

\[\label{eq:Bin-MAP} \hat{p}_\mathrm{MAP} = \frac{\alpha_0+y-1}{\alpha_0+\beta_0+n-2} \; .\]

Proof: Given the prior distribution in \eqref{eq:Bin-prior}, the posterior distribution for binomial observations is also a beta distribution

\[\label{eq:Bin-post} \mathrm{p}(p|y) = \mathrm{Bet}(p; \alpha_n, \beta_n)\]

where the posterior hyperparameters are equal to

\[\label{eq:Bin-post-par} \begin{split} \alpha_n &= \alpha_0 + y \\ \beta_n &= \beta_0 + (n-y) \; . \end{split}\]

The mode of the beta distribution is given by:

\[\label{eq:Beta-mode} X \sim \mathrm{Bet}(\alpha, \beta) \quad \Rightarrow \quad \mathrm{mode}(X) = \frac{\alpha-1}{\alpha+\beta-2} \; .\]

Applying \eqref{eq:Beta-mode} to \eqref{eq:Bin-post} with \eqref{eq:Bin-post-par}, the maximum-a-posteriori estimate of $p$ follows as:

\[\label{eq:Bin-MAP-qed} \begin{split} \hat{p}_\mathrm{MAP} &= \frac{\alpha_n-1}{\alpha_n+\beta_n-2} \\ &\overset{\eqref{eq:Bin-post-par}}{=} \frac{\alpha_0+y-1}{\alpha_0+y+\beta_0+(n-y)-2} \\ &= \frac{\alpha_0+y-1}{\alpha_0+\beta_0+n-2} \; . \end{split}\]
∎
Sources:

Metadata: ID: P427 | shortcut: bin-map | author: JoramSoch | date: 2023-12-01, 14:36.