Index: The Book of Statistical Proofs ▷ Probability Distributions ▷ Multivariate discrete distributions ▷ Categorical distribution ▷ Covariance

Theorem: Let $X$ be a random vector following a categorical distribution:

\[\label{eq:cat} X \sim \mathrm{Cat}(n,p) \; .\]

Then, the covariance matrix of $X$ is

\[\label{eq:cat-cov} \mathrm{Cov}(X) = \mathrm{diag}(p) - pp^\mathrm{T} \; .\]

Proof: The categorical distribution is a special case of the multinomial distribution in which $n = 1$:

\[\label{eq:cat-mult} X \sim \mathrm{Mult}(n,p) \quad \text{and} \quad n = 1 \quad \Rightarrow \quad X \sim \mathrm{Cat}(p) \; .\]

The covariance matrix of the multinomial distribution is

\[\label{eq:mult-cov} \mathrm{Cov}(X) = n \left(\mathrm{diag}(p) - pp^\mathrm{T} \right) \; ,\]

thus the covariance matrix of the categorical distribution is

\[\label{eq:cat-cov-qed} \mathrm{Cov}(X) = \mathrm{diag}(p) - pp^\mathrm{T} \; .\]
∎
Sources:

Metadata: ID: P338 | shortcut: cat-cov | author: JoramSoch | date: 2022-09-09, 16:57.