Normal-Wishart distribution

Short description: Multivariate probability distribution

Normal-Wishart

Notation	$(\mathit{\mu },\mathrm{\Lambda })\sim \mathrm{N}\mathrm{W}({\mathit{\mu }}_0,\lambda ,\mathbf{𝐖},\nu )$
Parameters	${\mathit{\mu }}_0\in {\mathbb{ℝ}}^D$ location (vector of real) $\lambda >0$ (real) $\mathbf{𝐖}\in {\mathbb{ℝ}}^{D\times D}$ scale matrix (pos. def.) $\nu >D-1$ (real)
Support	$\mathit{\mu }\in {\mathbb{ℝ}}^D;\mathrm{\Lambda }\in {\mathbb{ℝ}}^{D\times D}$ covariance matrix (pos. def.)
PDF	$f(\mathit{\mu },\mathrm{\Lambda }|{\mathit{\mu }}_0,\lambda ,\mathbf{𝐖},\nu )={𝒩}(\mathit{\mu }|{\mathit{\mu }}_0,(\lambda \mathrm{\Lambda }{)}^{-1}){𝒲}(\mathrm{\Lambda }|\mathbf{𝐖},\nu )$

In probability theory and statistics, the normal-Wishart distribution (or Gaussian-Wishart distribution) is a multivariate four-parameter family of continuous probability distributions. It is the conjugate prior of a multivariate normal distribution with unknown mean and precision matrix (the inverse of the covariance matrix).

Definition

Suppose

$\mathit{\mu }|{\mathit{\mu }}_0,\lambda ,\mathrm{\Lambda }\sim {𝒩}({\mathit{\mu }}_0,(\lambda \mathrm{\Lambda }{)}^{-1})$

has a multivariate normal distribution with mean ${\mathit{\mu }}_0$ and covariance matrix $(\lambda \mathrm{\Lambda }{)}^{-1}$, where

$\mathrm{\Lambda }|\mathbf{𝐖},\nu \sim {𝒲}(\mathrm{\Lambda }|\mathbf{𝐖},\nu )$

has a Wishart distribution. Then $(\mathit{\mu },\mathrm{\Lambda })$ has a normal-Wishart distribution, denoted as

$(\mathit{\mu },\mathrm{\Lambda })\sim \mathrm{N}\mathrm{W}({\mathit{\mu }}_0,\lambda ,\mathbf{𝐖},\nu ).$

Characterization

Probability density function

$f(\mathit{\mu },\mathrm{\Lambda }|{\mathit{\mu }}_0,\lambda ,\mathbf{𝐖},\nu )={𝒩}(\mathit{\mu }|{\mathit{\mu }}_0,(\lambda \mathrm{\Lambda }{)}^{-1}){𝒲}(\mathrm{\Lambda }|\mathbf{𝐖},\nu )$

Properties

Marginal distributions

By construction, the marginal distribution over $\mathrm{\Lambda }$ is a Wishart distribution, and the conditional distribution over $\mathit{\mu }$ given $\mathrm{\Lambda }$ is a multivariate normal distribution. The marginal distribution over $\mathit{\mu }$ is a multivariate t-distribution.

Posterior distribution of the parameters

After making $n$ observations ${\mathit{x}}_1,\dots ,{\mathit{x}}_n$, the posterior distribution of the parameters is

$(\mathit{\mu },\mathrm{\Lambda })\sim \mathrm{N}\mathrm{W}({\mathit{\mu }}_n,{\lambda }_n,{\mathbf{𝐖}}_n,{\nu }_n),$

where

${\lambda }_n=\lambda +n,$

${\mathit{\mu }}_n=\frac{\lambda {\mathit{\mu }}_0+n\overline{\mathit{x}}}{\lambda +n},$

${\nu }_n=\nu +n,$

${\mathbf{𝐖}}_n^{-1}={\mathbf{𝐖}}^{-1}+\sum _{i=1}^n({\mathit{x}}_i-\overline{\mathit{x}})({\mathit{x}}_i-\overline{\mathit{x}}{)}^T+\frac{n\lambda }{n+\lambda }(\overline{\mathit{x}}-{\mathit{\mu }}_0)(\overline{\mathit{x}}-{\mathit{\mu }}_0{)}^T.$

Generating normal-Wishart random variates

Generation of random variates is straightforward:

1. Sample $\mathrm{\Lambda }$ from a Wishart distribution with parameters $\mathbf{𝐖}$ and $\nu$
2. Sample $\mathit{\mu }$ from a multivariate normal distribution with mean ${\mathit{\mu }}_0$ and variance $(\lambda \mathrm{\Lambda }{)}^{-1}$

Related distributions

• The normal-inverse Wishart distribution is essentially the same distribution parameterized by variance rather than precision.
• The normal-gamma distribution is the one-dimensional equivalent.
• The multivariate normal distribution and Wishart distribution are the component distributions out of which this distribution is made.

References

• Bishop, Christopher M. (2006). Pattern Recognition and Machine Learning. Springer Science+Business Media.