Definition: Maximum entropy prior distribution
Index:
The Book of Statistical Proofs ▷
General Theorems ▷
Bayesian statistics ▷
Prior distributions ▷
Maximum entropy priors
Sources:
Metadata: ID: D121 | shortcut: prior-maxent | author: JoramSoch | date: 2020-12-02, 18:13.
Definition: Let $m$ be a generative model with likelihood function $p(y \vert \theta, m)$ and prior distribution $p(\theta \vert \lambda, m)$ using prior hyperparameters $\lambda$. Then, the prior distribution is called a “maximum entropy prior”, if
1) when $\theta$ is a discrete random variable, it maximizes the entropy of the prior probability mass function:
\[\label{eq:prior-maxent-disc} \lambda_{\mathrm{maxent}} = \operatorname*{arg\,max}_{\lambda} \mathrm{H}\left[ p(\theta \vert \lambda, m) \right] \; ;\]2) when $\theta$ is a continuous random variable, it maximizes the differential entropy of the prior probability density function:
\[\label{eq:prior-maxent-cont} \lambda_{\mathrm{maxent}} = \operatorname*{arg\,max}_{\lambda} \mathrm{h}\left[ p(\theta \vert \lambda, m) \right] \; .\]- Wikipedia (2020): "Prior probability"; in: Wikipedia, the free encyclopedia, retrieved on 2020-12-02; URL: https://en.wikipedia.org/wiki/Prior_probability#Uninformative_priors.
Metadata: ID: D121 | shortcut: prior-maxent | author: JoramSoch | date: 2020-12-02, 18:13.