Index: The Book of Statistical Proofs ▷ Statistical Models ▷ Count data ▷ Multinomial observations ▷ Maximum likelihood estimation

Theorem: Let $y = [y_1, \ldots, y_k]$ be the number of observations in $k$ categories resulting from $n$ independent trials with unknown category probabilities $p = [p_1, \ldots, p_k]$, such that $y$ follows a multinomial distribution:

\[\label{eq:Mult} y \sim \mathrm{Mult}(n,p) \; .\]

Then, the maximum likelihood estimator of $p$ is

\[\label{eq:Mult-MLE} \hat{p} = \frac{1}{n} y , \quad \text{i.e.} \quad \hat{p}_j = \frac{y_j}{n} \quad \text{for all} \quad j = 1, \ldots, k \; .\]

Proof: Note that the marginal distribution of each element in a multinomial random vector is a binomial distribution

\[\label{eq:Mult-marg} X \sim \mathrm{Mult}(n,p) \quad \Rightarrow \quad X_j \sim \mathrm{Bin}(n, p_j) \quad \text{for all} \quad j = 1, \ldots, k \; .\]

Thus, combining \eqref{eq:Mult} with \eqref{eq:Mult-marg}, we have

\[\label{eq:Mult-Bin} y_j \sim \mathrm{Bin}(n,p_j)\]

which implies the likelihood function

\[\label{eq:Bin-LF} \mathrm{p}(y|p_j) = \mathrm{Bin}(y_j; n, p_j) = {n \choose y_j} \, p_j^{y_j} \, (1-p_j)^{n-y_j} \; .\]

To this, we can apply maximum likelihood estimation for binomial observations, such that the MLE for each $p_j$ is

\[\label{eq:Mult-MLE-qed} \hat{p}_j = \frac{y_j}{n} \; .\]
∎
Sources:

Metadata: ID: P385 | shortcut: mult-mle | author: JoramSoch | date: 2022-12-02, 17:00.