Search for probability and statistics terms on Statlect
StatLect
Index > Fundamentals of probability

Expected value

by , PhD

The concept of expected value of a random variable is one of the most important concepts in probability theory.

Table of Contents

Intuition

The concept was first devised in the 17th century to analyze gambling games and answer questions such as:

If the possible outcomes of the game (or the bet) and their associated probabilities are described by a random variable, then these questions can be answered by computing its expected value.

The expected value is a weighted average of the possible realizations of the random variable (the possible outcomes of the game). Each realization is weighted by its probability.

For example, if you play a game where you gain 2$ with probability 1/2 and you lose 1$ with probability 1/2, then the expected value of the game is half a dollar:[eq1]

What does this mean? Roughly speaking, it means that if you play this game many times, and the number of times each of the two possible outcomes occurs is proportional to its probability, then on average you gain 1/2$ each time you play the game.

For instance, if you play the game 100 times, win 50 times and lose the remaining 50, then your average winning is equal to the expected value:[eq2]

In general, giving a rigorous definition of expected value requires quite a heavy mathematical apparatus. To keep things simple, we provide an informal definition of expected value and we discuss its computation in this lecture, while we relegate a more rigorous definition to the (optional) lecture entitled Expected value and the Lebesgue integral.

Definition

The following is an informal definition of expected value.

Definition (informal) The expected value of a random variable X is the weighted average of the values that X can take on, where each possible value is weighted by its respective probability.

The expected value of a random variable X is denoted by [eq3] and it is often called the expectation of X or the mean of X.

The following sections discuss how the expected value of a random variable is computed.

Expected value of a discrete random variable

When X is a discrete random variable having support R_X and probability mass function [eq4], the formula for computing its expected value is a straightforward implementation of the informal definition given above: the expected value of X is the weighted average of the values that X can take on (the elements of R_X), where each possible value $xin R_{X}$ is weighted by its respective probability [eq5].

Definition Let X be a discrete random variable with support R_X and probability mass function [eq6]. The expected value of X is[eq7]provided that[eq8]

The symbol [eq9]indicates summation over all the elements of the support R_X.

For example, if [eq10]then[eq11]

The requirement that [eq12]is called absolute summability and ensures that the summation [eq13]is well-defined also when the support R_X contains infinitely many elements.

When summing infinitely many terms, the order in which you sum them can change the result of the sum. However, if the terms are absolutely summable, then the order in which you sum becomes irrelevant.

In the above definition of expected value, the order of the sum[eq14]is not specified, therefore the requirement of absolute summability is introduced in order to ensure that the expected value is well-defined.

When the absolute summability condition is not satisfied, we say that the expected value of X is not well-defined or that it does not exist.

Example Let X be a random variable with support [eq15] and probability mass function[eq16]Its expected value is[eq17]

Expected value of a continuous random variable

When X is a continuous random variable with probability density function [eq18], the formula for computing its expected value involves an integral, which can be thought of as the limiting case of the summation [eq19] found in the discrete case above.

Definition Let X be a continuous random variable with probability density function [eq20]. The expected value of X is[eq21]provided that[eq22]

Roughly speaking, this integral is the limiting case of the formula for the expected value of a discrete random variable [eq23]

Here, [eq24] is replaced by [eq25] (the infinitesimal probability of x) and the integral sign [eq26] replaces the summation sign $sum_{xin R_{X}}$.

The requirement that [eq27]is called absolute integrability and ensures that the improper integral [eq28]is well-defined.

This improper integral is a shorthand for[eq29]and it is well-defined only if both limits are finite. Absolute integrability guarantees that the latter condition is met and that the expected value is well-defined.

When the absolute integrability condition is not satisfied, we say that the expected value of X is not well-defined or that it does not exist.

Example Let X be a continuous random variable with support [eq30] and probability density function[eq31]where $lambda >0$. Its expected value is[eq32]

Expected value of a random variable in general: the Riemann-Stieltjes integral

This section introduces a general formula for computing the expected value of a random variable X. The formula, which does not require X to be discrete or continuous and is applicable to any random variable, involves an integral called Riemann-Stieltjes integral. While we briefly discuss this formula for the sake of completeness, no deep understanding of this formula or of the Riemann-Stieltjes integral is required to understand the other lectures.

Definition Let X be a random variable having distribution function [eq33]. The expected value of X is[eq34]where the integral is a Riemann-Stieltjes integral and the expected value exists and is well-defined only as long as the integral is well-defined.

Roughly speaking, this integral is the limiting case of the formula for the expected value of a discrete random variable [eq35] Here [eq36] replaces [eq37] (the probability of x) and the integral sign [eq38] replaces the summation sign $sum_{xin R_{X}}$.

The following section contains a brief and informal introduction to the Riemann-Stieltjes integral and an explanation of the above formula. Less technically oriented readers can safely skip it: when they encounter a Riemann-Stieltjes integral, they can just think of it as a formal notation which allows for a unified treatment of discrete and continuous random variables and can be treated as a sum in one case and as an ordinary Riemann integral in the other.

The Riemann-Stieltjes integral: intuition

As we have already seen above, the expected value of a discrete random variable is straightforward to compute: the expected value of a discrete variable X is the weighted average of the values that X can take on (the elements of the support R_X), where each possible value x is weighted by its respective probability [eq39]:[eq40]or, written in a slightly different fashion,[eq41]

When X is not discrete the above summation does not make any sense. However, there is a workaround that allows us to extend the formula to random variables that are not discrete. The workaround entails approximating X with discrete variables that can take on only finitely many values.

Let $x_{0}$,$x_{1}$,...,$x_{n}$ be $n+1$ real numbers ($nin U{2115} $) such that:[eq42]

Define a new random variable X_n (function of X) as follows:[eq43]

As the number n of points increases and the points become closer and closer (the maximum distance between two successive points tends to zero), X_n becomes a very good approximation of X, until, in the limit, it is indistinguishable from X.

The expected value of X_n is easy to compute:[eq44]where [eq45] is the distribution function of X.

The expected value of X is then defined as the limit of [eq46] when n tends to infinity (i.e., when the approximation becomes better and better):[eq47]

When the latter limit exists and is well-defined, it is called the Riemann-Stieltjes integral of x with respect to [eq48] and it is indicated as follows:[eq49]

Roughly speaking, the integral notation [eq50] can be thought of as a shorthand for [eq51] and the differential notation [eq52] can be thought of as a shorthand for [eq53].

If you are not familiar with the Riemann-Stieltjes integral, make sure you also read the lecture entitled Computing the Riemann-Stieltjes integral: some rules, before reading the next example.

Example Let X be a random variable with support [eq54] and distribution function[eq55]Its expected value is[eq56]

Expected value of a random variable in general: the Lebesgue integral

A completely general and rigorous definition of expected value is based on the Lebesgue integral. We report it below without further comments. Less technically inclined readers can safely skip it, while interested readers can read more about it in the lecture entitled Expected value and the Lebesgue integral.

Definition Let Omega be a sample space, $QTR{rm}{P}$ a probability measure defined on the events of Omega and X a random variable defined on Omega. The expected value of X is[eq57]provided $int XdQTR{rm}{P}$ (the Lebesgue integral of X with respect to $QTR{rm}{P}$) exists and is well-defined.

More details

The next sections contain more details about the expected value.

The transformation theorem

An important property of the expected value, known as transformation theorem, allows us to easily compute the expected value of a function of a random variable.

Let X be a random variable. Let [eq58] be a real function. Define a new random variable Y as follows:[eq59]

Then,[eq60]provided the above integral exists.

This is an important property. It says that, if you need to compute the expected value of [eq61], you do not need to know the support of Y and its distribution function [eq62]: you can compute it just by replacing x with g(x) in the formula for the expected value of X.

For discrete random variables the formula becomes [eq63]while for continuous random variables it is[eq64]It is possible (albeit non-trivial) to prove that the above two formulae hold also when X is a K-dimensional random vector, [eq65] is a real function of K variables and [eq66].

When X is a discrete random vector and [eq67] is its joint probability function, then[eq68]

When X is an continuous random vector and [eq69] is its joint density function, then[eq70]

Linearity of the expected value

If X is a random variable and Y is another random variable such that[eq71]where $ain U{211d} $ and $bin U{211d} $ are two constants, then the following holds:[eq72]

Proof

For discrete random variables this is proved as follows:[eq73]For continuous random variables the proof is[eq74]In general, the linearity property is a consequence of the transformation theorem and of the fact that the Riemann-Stieltjes integral is a linear operator:[eq75]

A stronger linearity property holds, which involves two (or more) random variables. The property can be proved only using the Lebesgue integral (see the lecture entitled Expected value and the Lebesgue integral).

The property is as follows: let X_1 and X_2 be two random variables and let $c_{1}in U{211d} $ and $c_{2}in U{211d} $ be two constants; then[eq76]

Expected value of random vectors

Let X be a K-dimensional random vector and denote its components by X_1, ..., $X_{K}$. The expected value of X, denoted by [eq77], is just the vector of the expected values of the K components of X. Suppose, for example, that X is a row vector; then[eq78]

Expected value of random matrices

Let Sigma be a $K imes L$ random matrix, that is, a $K imes L$ matrix whose entries are random variables. Denote its $left( i,j ight) $-th entry by $Sigma _{ij}$. The expected value of Sigma, denoted by [eq79], is just the matrix of the expected values of the entries of Sigma:[eq80]

Integrability

Denote the absolute value of a random variable X by [eq81]. If [eq82] exists and is finite, we say that X is an integrable random variable, or just that X is integrable.

Lp spaces

Let $1leq p<infty $. The space of all random variables X such that [eq83] exists and is finite is denoted by $L^{p}$ or [eq84], where the triple [eq85] makes the dependence on the underlying probability space explicit.

If X belongs to $L^{p}$, we write [eq86].

Hence, if X is integrable, we write [eq87].

Related lectures

The following lectures contain more material about the expected value.

Conditional expectation

Introduces the conditional version of the expected value operator

Properties of the expected value

Statements, proofs and examples of the main properties of the expected value operator

Expected value and the Lebesgue integral

Provides a rigorous definition of expected value, based on the Lebesgue integral

Solved exercises

Some solved exercises on expected value can be found below.

Exercise 1

Let X be a discrete random variable. Let its support R_X be[eq88]

Let its probability mass function [eq89] be[eq90]

Compute the expected value of X.

Solution

Since X is discrete, its expected value is computed as a sum over the support of X:[eq91]

Exercise 2

Let X be a discrete variable with support[eq92]

and probability mass function[eq93]

Compute its expected value.

Solution

Since X is discrete, its expected value is computed as a sum over the support of X:[eq94]

Exercise 3

Let X be a discrete variable. Let its support be[eq95]

Let its probability mass function be[eq96]

Compute the expected value of X.

Solution

Since X is discrete, its expected value is computed as a sum over the support of X:[eq97]

Exercise 4

Let X be a continuous random variable with uniform distribution on the interval $left[ 1,3 ight] $.

Its support is[eq98]

Its probability density function is[eq99]

Compute the expected value of X.

Solution

Since X is continuous, its expected value can be computed as an integral:[eq100]

Note that the trick is to: 1) subdivide the interval of integration to isolate the sub-intervals where the density is zero; 2) split up the integral among the various sub-intervals.

Exercise 5

Let X be a continuous random variable. Its support is[eq101]

Its probability density function is[eq102]

Compute the expected value of X.

Solution

Since X is continuous, its expected value can be computed as an integral:[eq103]

Exercise 6

Let X be a continuous random variable. Its support is[eq104]

Its probability density function is[eq105]

Compute the expected value of X.

Solution

Since X is continuous, its expected value can be computed as an integral:[eq106]

How to cite

Please cite as:

Taboga, Marco (2021). "Expected value", Lectures on probability theory and mathematical statistics. Kindle Direct Publishing. Online appendix. https://www.statlect.com/fundamentals-of-probability/expected-value.

The books

Most of the learning materials found on this website are now available in a traditional textbook format.

Featured pages
Explore
Main sections
About
Glossary entries
Share
  • To enhance your privacy,
  • we removed the social buttons,
  • but don't forget to share.