Index: The Book of Statistical ProofsGeneral Theorems ▷ Bayesian statistics ▷ Prior distributions ▷ Maximum entropy priors

Definition: Let $m$ be a generative model with likelihood function $p(y \vert \theta, m)$ and prior distribution $p(\theta \vert \lambda, m)$ using prior hyperparameters $\lambda$. Then, the prior distribution is called a “maximum entropy prior”, if

1) when $\theta$ is a discrete random variable, it maximizes the entropy of the prior probability mass function:

$\label{eq:prior-maxent-disc} \lambda_{\mathrm{maxent}} = \operatorname*{arg\,max}_{\lambda} \mathrm{H}\left[ p(\theta \vert \lambda, m) \right] \; ;$

2) when $\theta$ is a continuous random variable, it maximizes the differential entropy of the prior probability density function:

$\label{eq:prior-maxent-cont} \lambda_{\mathrm{maxent}} = \operatorname*{arg\,max}_{\lambda} \mathrm{h}\left[ p(\theta \vert \lambda, m) \right] \; .$

Sources:

Metadata: ID: D121 | shortcut: prior-maxent | author: JoramSoch | date: 2020-12-02, 18:13.