Search for probability and statistics terms on Statlect
StatLect
Index > Fundamentals of probability

Random vectors

by , PhD

The concept of random vector is a multidimensional generalization of the concept of random variable.

Table of Contents

Definition

Suppose that we conduct a probabilistic experiment and that the possible outcomes of the experiment are described by a sample space Omega.

A random vector is a vector whose value depends on the outcome of the experiment, as stated by the following definition.

Definition Let Omega be a sample space. A random vector X is a function from the sample space Omega to the set of K-dimensional real vectors $U{211d} ^{K}$:[eq1]

In rigorous probability theory, the function X is also required to be measurable (a concept found in measure theory - see a more rigorous definition of random vector).

The real vector [eq2] associated to a sample point omega in Omega is called a realization of the random vector.

The set of all possible realizations is called support and is denoted by R_X.

Notation

Denote by [eq3] the probability of an event $Esubseteq Omega $. When dealing with random vectors, the following conventions are used:

Example

The following example shows how a random vector can be defined on a sample space.

Example Two coins are tossed. The possible outcomes of each toss can be either tail ($T$) or head (H). The sample space is[eq11]The four possible outcomes are assigned equal probabilities:[eq12]If tail ($T$) is the outcome, we win one dollar, if head (H) is the outcome we lose one dollar. A 2-dimensional random vector X indicates the amount we win (or lose) on each toss:[eq13]The probability of winning one dollar on both tosses is[eq14]The probability of losing one dollar on the second toss is[eq15]

Discrete random vectors

This section and the next one deal with discrete and continuous vectors, two kinds of random vectors that have special properties and are often found in applications.

Discrete vectors are defined as follows.

Definition A random vector X is discrete if and only if

  1. its support R_X is a countable set;

  2. there is a function [eq16], called the joint probability mass function (or joint pmf, or joint probability function) of X, such that, for any $xin U{211d} ^{K}$:[eq17]

The following notations are used interchangeably to indicate the joint probability mass function:[eq18]

In the second and third notation the K components of X are explicitly indicated.

Example Suppose X is a $2$-dimensional random vector whose components (X_1 and X_2) can take only two values: 1 or 0. Furthermore, the four possible combinations of 0 and 1 are all equally likely. X is an example of a discrete vector. Its support is [eq19]Its probability mass function is[eq20]

Continuous random vectors

Continuous vectors are defined as follows.

Definition A random vector X is continuous (or absolutely continuous) if and only if

  1. its support R_X is not countable;

  2. there is a function [eq21], called the joint probability density function (or joint pdf or joint density function) of X, such that, for any set [eq7] where[eq23]the probability that X belongs to A can be calculated as follows:[eq24]provided the above multiple integral is well defined.

The following notations are used interchangeably to indicate the joint probability density function:[eq25]

In the second and third notation the K components of the random vector X are explicitly indicated.

Example Suppose X is a $2$-dimensional random vector whose components (X_1 and X_2) are independent uniform random variables (on the interval $left[ 0,1 ight] $). Then, X is an example of a continuous vector. Its support is[eq26]Its joint probability density function is[eq27]The probability that the realization of X falls in the rectangle [eq28] is[eq29]

Random vectors in general

Random vectors, also those that are neither discrete nor continuous, are often described using their joint distribution function.

Definition Let X be a random vector. The joint distribution function (or joint df, or joint cumulative distribution function, or joint cdf) of X is a function [eq30] such that[eq31]where the components of X and x are denoted by $X_{k}$ and $x_{k}$ respectively, for $k=1,ldots ,K$.

The following notations are used interchangeably to indicate the joint distribution function:[eq32]

In the second and third notation the K components of the random vector X are explicitly indicated.

Joint distribution

Sometimes, we talk about the joint distribution of a random vector, without specifying whether we are referring to

This ambiguity is legitimate, since

  1. the joint pmf completely determines (and is completely determined by) the joint distribution function of a discrete vector;

  2. the joint pdf completely determines (and is completely determined by) the joint distribution function of a continuous vector.

In the remainder of this lecture, we use the term joint distribution when we are making statements that apply both to the distribution function and to the probability mass (or density) function of a random vector.

More details

The following subsections contain more details about random vectors.

Random matrices

A random matrix is a matrix whose entries are random variables.

It is not necessary to develop a separate theory for random matrices because a random matrix can always be written as a random vector.

Given a $K imes L$ random matrix A, its vectorization, denoted by [eq33], is the $KL imes 1$ random vector obtained by stacking the columns of A on top of each other.

Example Let A be the following $2 imes 2$ random matrix:[eq34]The vectorization of A is the following $4 imes 1$ random vector:[eq35]

When [eq36] is a discrete vector, then we say that $A $ is a discrete random matrix and the joint pmf of A is just the joint pmf of [eq36].

By the same token, when [eq36] is a continuous vector, then we say that A is a continuous random matrix and the joint pdf of A is just the joint pdf of [eq36].

The marginal distribution of a random vector

Let X_i be the i-th component of a K-dimensional random vector X.

The distribution function [eq40] of X_i is called marginal distribution function of X_i.

If X is discrete, then X_i is a discrete random variable and its probability mass function [eq41] is called marginal probability mass function of X_i.

If X is continuous, then X_i is a continuous random variable and its probability density function [eq42] is called marginal probability density function of X_i.

Marginalization of a joint distribution

The process of deriving the distribution of a component X_i of a random vector X from the joint distribution of X is known as marginalization.

Marginalization can also have a broader meaning: it can refer to the act of deriving the joint distribution of a subset of the set of components of X from the joint distribution of X.

For example, if X is a random vector having three components (X_1, X_2 and $X_{3}$), we can marginalize the joint distribution of X_1, X_2 and $X_{3}$ to find the joint distribution of X_1 and X_2 (in this case we say that $X_{3}$ is marginalized out of the joint distribution of X_1, X_2 and $X_{3}$).

The marginal distribution of a discrete vector

Let X_i be the i-th component of a K-dimensional discrete random vector X. The marginal probability mass function of X_i can be derived from the joint probability mass function of X as follows:[eq43]where the sum is over the set[eq44]

In other words, the probability that $X_{i}=x$ is obtained as the sum of the probabilities of all the vectors in R_X such that their i-th component is equal to x.

Marginalization of a discrete distribution

Let X_i be the i-th component of a discrete random vector X. By marginalizing X_i out of the joint distribution of X, we obtain the joint distribution of the remaining components of X, that is, we obtain the joint distribution of the random vector $X_{-i}$ defined as follows:[eq45]

The joint probability mass function of $X_{-i}$ is computed as follows:[eq46]where the sum is over the set[eq47]

In other words, the joint probability mass function of $X_{-i}$ can be computed by summing the joint probability mass function of X over all values of $x_{i}$ that belong to the support of X_i.

The marginal distribution of a continuous vector

Let X_i be the i-th component of a K-dimensional continuous random vector X. The marginal probability density function of X_i can be derived from the joint probability density function of X as follows:[eq48]

In other words, the joint probability density function, evaluated at $x_{i}=x $, is integrated with respect to all variables except $x_{i}$ (so it is integrated a total of $K-1$ times).

Marginalization of a continuous distribution

Let X_i be the i-th component of a continuous random vector X. By marginalizing X_i out of the joint distribution of X, we obtain the joint distribution of the remaining components of X, that is, we get the joint distribution of the random vector $X_{-i}$ defined as follows:[eq49]

The joint probability density function of $X_{-i}$ is computed as follows:[eq50]

In other words, the joint probability density function of $X_{-i}$ can be computed by integrating the joint probability density function of X with respect to $x_{i}$.

Partial derivatives of the distribution function of a continuous vector

Note that, if X is continuous, then[eq51]

Hence, by taking the K-th order cross-partial derivative with respect to [eq52] of both sides of the above equation, we obtain[eq53]

A more rigorous definition of random vector

We report here a more rigorous definition of random vector by using the formalism of measure theory. This definition is analogous to the measure-theoretic definition given in the lecture on random variables, to which you should refer for a more detailed explanation.

Definition Let [eq54] be a probability space. Let [eq55] be the Borel sigma-algebra of $U{211d} ^{K}$ (i.e., the smallest sigma-algebra containing all open hyper-rectangles in $U{211d} ^{K}$). A function [eq56] such that [eq57]for any [eq58] is said to be a random vector on Omega.

This definition ensures that the probability that the realization of the random vector X will belong to a set [eq58] can be defined as [eq60]because the set [eq61] belongs to the sigma-algebra [eq62] and, as a consequence, its probability is well-defined.

Solved exercises

Some solved exercises on random vectors can be found below.

Exercise 1

Let X be a $2 imes 1$ discrete random vector and denote its components by X_1 and X_2.

Let the support of X be the set of all $2 imes 1$ vectors such that their entries belong to the set of the first three natural numbers, that is, [eq63]where[eq64]

Let the joint probability mass function of X be[eq65]

Find [eq66].

Solution

Trivially, we need to evaluate the joint probability mass function at the point $left( 2,3 ight) $, that is,[eq67]

Exercise 2

Let X be a $2 imes 1$ discrete random vector and denote its components by X_1 and X_2.

Let the support of X be the set of all $2 imes 1$ vectors such that their entries belong to the set of the first three natural numbers, that is,[eq68]where[eq64]

Let the joint probability mass function of X be[eq70]

Find [eq71].

Solution

There are only two possible cases that give rise to the occurrence $X_{1}+X_{2}=3$. These cases are[eq72]and[eq73]Therefore, since these two cases are disjoint events, we can use the additivity of probability:[eq74]

Exercise 3

Let X be a $2 imes 1$ discrete random vector and denote its components by X_1 and X_2.

Let the support of X be[eq75]and its joint probability mass function be[eq76]

Derive the marginal probability mass functions of X_1 and X_2.

Solution

The support of X_1 is[eq77]We need to compute the probability of each element of the support of X_1:[eq78]Thus, the probability mass function of X_1 is[eq79]The support of X_2 is[eq80]We need to compute the probability of each element of the support of X_2:[eq81]Thus, the probability mass function of X_2 is[eq82]

Exercise 4

Let X be a $2 imes 1$ continuous random vector and denote its components by X_1 and X_2.

Let the support of X be [eq83]that is, the set of all $2 imes 1$ vectors such that the first component belongs to the interval $left[ 0,2 ight] $ and the second component belongs to the interval $left[ 0,3 ight] $.

Let the joint probability density function of X be[eq84]

Compute [eq85].

Solution

By the very definition of joint probability density function:[eq86]

Exercise 5

Let X be a $2 imes 1$ continuous random vector and denote its components by X_1 and X_2.

Let the support of X be [eq87]that is, the set of all $2 imes 1$ vectors such that the first component belongs to the interval [eq88] and the second component belongs to the interval $left[ 0,2 ight] $.

Let the joint probability density function of X be[eq89]

Compute [eq90].

Solution

First of all note that $X_{1}+X_{2}leq 3$ if and only if $X_{2}leq 3-X_{1}$. By using the definition of joint probability density function, we obtain[eq91]Now, note that, when [eq92], the inner integral is[eq93]Therefore,[eq94]

Exercise 6

Let X be a $2 imes 1$ continuous random vector and denote its components by X_1 and X_2.

Let the support of X be [eq95] (i.e., the set of all $2$-dimensional vectors with positive entries) and its joint probability density function be[eq96]

Derive the marginal probability density functions of X_1 and X_2.

Solution

The support of X_1 is[eq97](recall that [eq98] and [eq99]). We can find the marginal density by integrating the joint density with respect to $x_{2}$:[eq100]When $x<0$, then [eq101] and the above integral is trivially equal to 0. Thus, when $x<0$, then [eq102]. When $x>0$, then[eq103]but the first of the two integrals is zero since [eq104] when $x_{2}<0$; as a consequence,[eq105]So, by putting pieces together, we get the marginal density function of X_1:[eq106]By symmetry, the marginal density function of X_2 is[eq107]

How to cite

Please cite as:

Taboga, Marco (2021). "Random vectors", Lectures on probability theory and mathematical statistics. Kindle Direct Publishing. Online appendix. https://www.statlect.com/fundamentals-of-probability/random-vectors.

The books

Most of the learning materials found on this website are now available in a traditional textbook format.

Featured pages
Explore
Main sections
About
Glossary entries
Share
  • To enhance your privacy,
  • we removed the social buttons,
  • but don't forget to share.