Search for probability and statistics terms on Statlect
StatLect
Index > Probability distributions

Wishart distribution

by , PhD

The Wishart distribution is a multivariate continuous distribution which generalizes the Gamma distribution.

Table of Contents

How the distribution is derived

In previous lectures we have explained that:

  1. a Chi-square random variable with n degrees of freedom can be seen as a sum of squares of n independent normal random variables having mean 0 and variance 1;

  2. a Gamma random variable with parameters n and $h$ can be seen as a sum of squares of n independent normal random variables having mean 0 and variance $h/n$.

A Wishart random matrix with parameters n and H can be seen as a sum of outer products of n independent multivariate normal random vectors having mean 0 and covariance matrix $frac{1}{n}H$.

In this sense, the Wishart distribution can be considered a generalization of the Gamma distribution (take point 2 above and substitute normal random variables with multivariate normal random vectors, squares with outer products and the variance $h/n$ with the covariance matrix $frac{1}{n}H$).

At the bottom of this page you can find a brief review of some basic concepts in matrix algebra that will be helpful in understanding the remainder of this lecture.

Definition

Wishart random matrices are characterized as follows.

Definition Let $W$ be a $K imes K$ continuous random matrix. Let its support be the set of all $K imes K$ symmetric and positive definite real matrices:[eq1]Let H be a symmetric and positive definite matrix and $n>K-1$. We say that $W$ has a Wishart distribution with parameters n and H if its joint probability density function is[eq2]where[eq3]and [eq4] is the Gamma function.

The parameter n needs not be an integer, but, when n is not an integer, $W$ can no longer be interpreted as a sum of outer products of multivariate normal random vectors.

Relation to the multivariate normal distribution

The following proposition provides the link between the multivariate normal distribution and the Wishart distribution.

Proposition Let [eq5] be n independent Kx1 random vectors all having a multivariate normal distribution with mean 0 and covariance matrix $frac{1}{n}H$. Let $Kleq n$. Define[eq6]Then $W$ has a Wishart distribution with parameters n and H.

Proof

The proof of this proposition is quite lengthy and complicated. The interested reader might have a look at Ghosh and Sinha (2002).

Expected value

The expected value of a Wishart random matrix $W$ is[eq7]

Proof

We do not provide a fully general proof, but we prove this result only for the special case in which n is integer and $W$ can be written as[eq8](see subsection above). In this case, we have that[eq9]where we have used the fact that the covariance matrix of X can be written as[eq10](see the lecture entitled Covariance matrix).

Covariance matrix

The concept of covariance matrix is well-defined only for random vectors. However, when dealing with a random matrix, one might want to compute the covariance matrix of its associated vectorization (if you are not familiar with the concept of vectorization, see the review of matrix algebra below for a definition). Therefore, in the case of a Wishart random matrix $W $, we might want to compute the following covariance matrix:[eq11]

Since [eq12], the vectorization of $W$, is a $K^{2} imes 1$ random vector, V is a $K^{2} imes K^{2}$ matrix.

It is possible to prove that[eq13]where $otimes $ denotes the Kronecker product and [eq14] is the transposition-permutation matrix associated to [eq15] (see the review of matrix algebra below for a definition).

Proof

The proof of this formula can be found in Muirhead (2005).

There is a simpler expression for the covariances between the diagonal entries of $W$: [eq16]

Proof

Again, we do not provide a fully general proof, but we prove this result only for the special case in which n is integer and $W$ can be written as[eq8](see above). To compute this covariance, we first need to compute the following fourth cross-moment:[eq18]where $X_{si}$ denotes the i-th component ($i=1,ldots ,K$) of the random vector $X_{s}$ ($s=1,ldots ,n$). This cross-moment can be computed by taking the fourth cross-partial derivative of the joint moment generating function of $X_{si}$ and $X_{sj}$ and evaluating it at zero (see the lecture entitled Joint moment generating function). While this is not complicated, the algebra is quite tedious. I recommend doing it with computer algebra, for example utilizing the Matlab Symbolic Toolbox and the following four commands:

syms t1 t2 s1 s2 s12;

f=exp(0.5*(s1^2)*(t1^2)+0.5*(s2^2)*(t2^2)+s12*t1*t2);

d4f=diff(diff(f,t1,2),t2,2);

subs(d4f,{t1,t2},{0,0})

The result of the computations is[eq19]Using this result, the covariance between $W_{ii}$ and $W_{jj}$ is derived as follows:[eq20]

Review of matrix algebra

This section reviews some results from matrix algebra that are used to deal with the Wishart distribution.

Outer products

As the Wishart distribution involves outer products of multivariate normal random vectors, we briefly review here the concept of outer product.

If X is a Kx1 column vector, the outer product of X with itself is the $K imes K$ matrix A obtained from the multiplication of X with its transpose:[eq21]

Example If X is the $2 imes 1$ random vector[eq22]then its outer product $XX^{ op }$ is the $2 imes 2$ random matrix[eq23]

Symmetric matrices

A $K imes K$ matrix A is symmetric if and only if[eq24]i.e. if and only if A equals its transpose.

Positive definite matrices

A $K imes K$ matrix A is said to be positive definite if and only if [eq25]for any Kx1 real vector x such that $x eq 0$.

All positive definite matrices are also invertible.

Proof

The proof is by contradiction. Suppose a positive definite matrix A were not invertible. Then A would not be full rank, i.e. there would be a vector $x eq 0$ such that[eq26]which, premultiplied by $x^{ op }$, would yield[eq27]But this is a contradiction.

Trace of a matrix

Let A be a $K imes K$ matrix and denote by $A_{ij}$ the $left( i,j ight) $-th entry of A (i.e. the entry at the intersection of the i-th row and the $j$-th column). The trace of A, denoted by [eq28], is the sum of all the diagonal entries of A:[eq29]

Vectorization of a matrix

Given a $K imes L$ matrix A, its vectorization, denoted by [eq30], is the $KL imes 1$ vector obtained by stacking the columns of A on top of each other.

Example If A is a $2 imes 2$ matrix[eq31]the vectorization of A is the $4 imes 1$ random vector[eq32]

For a given matrix A, the vectorization of A will in general be different from the vectorization of its transpose $A^{ op }$. The transposition permutation matrix associated to [eq33] is the $KL imes KL$ matrix [eq34] such that[eq35]

Kronecker product

Given a $K imes L$ matrix A and a $M imes N$ matrix $B$, the Kronecker product of A and $B$, denoted by $Aotimes B$, is a $KM imes LN $ matrix having the following structure:[eq36]where $A_{ij}$ is the $left( i,j ight) $-th entry of A.

References

Ghosh, M. and Sinha, B. K. (2002) "A simple derivation of the Wishart distribution", The American Statistician, 56, 100-101.

Muirhead, R.J. (2005) Aspects of multivariate statistical theory, Wiley.

How to cite

Please cite as:

Taboga, Marco (2021). "Wishart distribution", Lectures on probability theory and mathematical statistics. Kindle Direct Publishing. Online appendix. https://www.statlect.com/probability-distributions/wishart-distribution.

The books

Most of the learning materials found on this website are now available in a traditional textbook format.

Featured pages
Explore
Main sections
About
Glossary entries
Share
  • To enhance your privacy,
  • we removed the social buttons,
  • but don't forget to share.