Index: The Book of Statistical Proofs ▷ Probability Distributions ▷ Multivariate continuous distributions ▷ Multivariate normal distribution ▷ Moment-generating function

Theorem: Let $x$ follow a multivariate normal distribution:

\[\label{eq:mvn} x \sim \mathcal{N}(\mu, \Sigma) \; .\]

Then, the moment-generating function of $x$ is

\[\label{eq:mvn-mgf} M_x(t) = \exp \left[ t^\mathrm{T} \mu + \frac{1}{2} t^\mathrm{T} \Sigma t \right] \; .\]

Proof: The moment-generating function of a random vector $X$ is defined as:

\[\label{eq:mgf} M_X(t) = \mathrm{E} \left[ e^{t^\mathrm{T}X} \right], \quad t \in \mathbb{R}^n \; .\]

Applying the law of the unconscious statistician, we have:

\[\label{eq:mvn-mgf-s1} M_x(t) = \int_{\mathcal{X}} e^{t^\mathrm{T}x} \cdot f_X(x) \, \mathrm{d}x \; .\]

With the probability density function of the multivariate normal distribution, we have:

\[\label{eq:mvn-mgf-s2} M_x(t) = \int_{\mathbb{R}^n} \exp \left[ t^\mathrm{T}x \right] \cdot \frac{1}{\sqrt{(2 \pi)^n |\Sigma|}} \cdot \exp \left[ -\frac{1}{2} (x-\mu)^\mathrm{T} \Sigma^{-1} (x-\mu) \right] \, \mathrm{d}x \; .\]

Now we summarize the two exponential functions inside the integral:

\[\label{eq:mvn-mgf-s3} \begin{split} M_x(t) &= \int_{\mathbb{R}^n} \frac{1}{\sqrt{(2 \pi)^n |\Sigma|}} \cdot \exp \left[ -\frac{1}{2} (x-\mu)^\mathrm{T} \Sigma^{-1} (x-\mu) + t^\mathrm{T}x \right] \, \mathrm{d}x \\ &= \int_{\mathbb{R}^n} \frac{1}{\sqrt{(2 \pi)^n |\Sigma|}} \cdot \exp \left[ -\frac{1}{2} \left( x^\mathrm{T} \Sigma^{-1} x - 2 \mu^\mathrm{T} \Sigma^{-1} x + \mu^\mathrm{T} \Sigma^{-1} \mu - 2 t^\mathrm{T}x \right) \right] \, \mathrm{d}x \\ &= \int_{\mathbb{R}^n} \frac{1}{\sqrt{(2 \pi)^n |\Sigma|}} \cdot \exp \left[ -\frac{1}{2} \left( x^\mathrm{T} \Sigma^{-1} x - 2 (\mu + \Sigma t)^\mathrm{T} \Sigma^{-1} x + \mu^\mathrm{T} \Sigma^{-1} \mu \right) \right] \, \mathrm{d}x \\ &= \int_{\mathbb{R}^n} \frac{1}{\sqrt{(2 \pi)^n |\Sigma|}} \cdot \exp \left[ -\frac{1}{2} \left( (x - [\mu + \Sigma t])^\mathrm{T} \Sigma^{-1} (x - [\mu + \Sigma t]) - 2 t^\mathrm{T} \mu - t^\mathrm{T} \Sigma t \right) \right] \, \mathrm{d}x \\ &= \exp \left[ t^\mathrm{T} \mu + t^\mathrm{T} \Sigma t \right] \int_{\mathbb{R}^n} \frac{1}{\sqrt{(2 \pi)^n |\Sigma|}} \cdot \exp \left[ -\frac{1}{2} (x - [\mu + \Sigma t])^\mathrm{T} \Sigma^{-1} (x - [\mu + \Sigma t]) \right] \, \mathrm{d}x \; . \end{split}\]

The integrand is equal to the probability density function of a multivariate normal distribution:

\[\label{eq:mvn-mgf-s4} M_x(t) = \exp \left[ t^\mathrm{T} \mu + t^\mathrm{T} \Sigma t \right] \int_{\mathbb{R}^n} \mathcal{N}(x; \mu + \Sigma t, \Sigma) \, \mathrm{d}x \; .\]

Because the entire probability density integrates to one, we finally have:

\[\label{eq:mvn-mgf-s5} M_x(t) = \exp \left[ t^\mathrm{T} \mu + t^\mathrm{T} \Sigma t \right] \; .\]
∎
Sources:

Metadata: ID: P436 | shortcut: mvn-mgf | author: JoramSoch | date: 2024-02-16, 15:03.