Search for probability and statistics terms on Statlect
StatLect
Index > Fundamentals of probability

Properties of the expected value

by , PhD

This lecture discusses some fundamental properties of the expected value operator.

Some of these properties can be proved using the material presented in previous lectures. Others are gathered here for convenience, but can be fully understood only after reading the material presented in subsequent lectures.

It may be a good idea to memorize these properties as they provide essential rules for performing computations that involve the expected value.

Table of Contents

Scalar multiplication of a random variable

If X is a random variable and $ain U{211d} $ is a constant, then[eq1]

Proof

This property has been discussed in the lecture on the Expected value. It can be proved in several different ways, for example, by using the transformation theorem or the linearity of the Riemann-Stieltjes integral.

Example Let X be a random variable with expectation[eq2]and define[eq3]Then,[eq4]

Sums of random variables

If X_1, X_2, ..., $X_{K}$ are K random variables, then[eq5]

Proof

See the lecture on the Expected value. The same comments made for the previous property apply.

Example Let X and Y be two random variables with expected values[eq6]and define[eq7]Then,[eq8]

Linear combinations of random variables

If X_1, X_2, ..., $X_{K}$ are K random variables and [eq9] are K constants, then[eq10]

Proof

This can be trivially obtained by combining the two properties above (scalar multiplication and sum).

Consider [eq11] as the K entries of a $1 imes K$ vector a and X_1, X_2, ..., $X_{K}$ as the K entries of a Kx1 random vector X.

Then, we can also write[eq12]which is a multivariate generalization of the Scalar multiplication property above.

Example Let X and Y be two random variables with expected values[eq13]and define[eq14]Then,[eq15]

Expected value of a constant

A perhaps obvious property is that the expected value of a constant is equal to the constant itself:[eq16]for any constant $ain U{211d} $.

Proof

This rule is again a consequence of the fact that the expected value is a Riemann-Stieltjes integral and the latter is linear.

Expectation of a product of random variables

Let X and Y be two random variables. In general, there is no easy rule or formula for computing the expected value of their product.

However, if X and Y are statistically independent, then[eq17]

Proof

See the lecture on statistical independence.

Non-linear transformations

Let $g$ be a non-linear function. In general,[eq18]

However, Jensen's inequality tells us that[eq19]if $g$ is convex and [eq20]if $g$ is concave.

Example Since [eq21] is a convex function, we have[eq22]

Addition of a constant matrix and a matrix with random entries

Let Sigma be a $K imes L$ random matrix, that is, a $K imes L$ matrix whose entries are random variables.

If A is a $K imes L$ matrix of constants, then[eq23]

Proof

This is easily proved by applying the linearity properties above to each entry of the random matrix $A+Sigma $.

Note that a random vector is just a particular instance of a random matrix. So, if X is a Kx1 random vector and a is a Kx1 vector of constants, then[eq24]

Example Let X be a $2 imes 1$ random vector such that its two entries X_1 and X_2 have expected values[eq25]Let A be the following $2 imes 1$ constant vector:[eq26]Define[eq27]Then,[eq28]

Multiplication of a constant matrix and a matrix with random entries

Let Sigma be a $K imes L$ random matrix.

If $B$ is a $M imes K$ matrix of constants, then[eq29]

If $C$ is a a $L imes N$ matrix of constants, then[eq30]

Proof

These are immediate consequences of the linearity properties above.

By iteratively applying these properties, if $B$ is a $M imes K$ matrix of constants and $C$ is a a $L imes N$ matrix of constants, we obtain[eq31]

Example Let X be a $1 imes 2$ random vector such that[eq32]where X_1 and X_2 are the two components of X. Let A be the following $2 imes 2$ matrix of constants:[eq33]Define[eq34]Then,[eq35]

Expectation of a positive random variable

Let X be an integrable random variable defined on a sample space Omega.

Let [eq36] for all omega in Omega (i.e., X is a positive random variable).

Then,[eq37]

Proof

Intuitively, this is obvious. The expected value of X is a weighted average of the values that X can take on. But X can take on only positive values. Therefore, also its expectation must be positive. Formally, the expected value is the Lebesgue integral of X, and X can be approximated to any degree of accuracy by positive simple random variables whose Lebesgue integral is positive. Therefore, also the Lebesgue integral of X must be positive.

Preservation of almost sure inequalities

Let X and Y be two integrable random variables defined on a sample space Omega.

Let X and Y be such that $Xleq Y$ almost surely. In other words, there exists a zero-probability event E such that [eq38]

Then,[eq39]

Proof

Let E be a zero-probability event such that [eq40]First, note that[eq41]where $1_{E}$ is the indicator of the event E and $1_{E^{c}}$ is the indicator of the complement of E. As a consequence, we can write [eq42]By the properties of indicators of zero-probability events, we have [eq43]Thus, we can write[eq44]Now, when $omega in E^{c}$, then [eq45] and [eq46]. On the contrary, when $omega in E$, then $1_{E^{c}}=0$ and [eq47]. Therefore, [eq48] for all omega in Omega (i.e., [eq49] is a positive random variable). Thus, by the previous property (expectation of a positive random variable), we have [eq50]which implies [eq51]By the linearity of the expected value, we get[eq52]Therefore,[eq39]

Solved exercises

Below you can find some exercises with explained solutions.

Exercise 1

Let X and Y be two random variables, having expected values:[eq54]

Compute the expected value of the random variable Z defined as follows:[eq55]

Solution

Using the linearity of the expected value operator, we obtain[eq56]

Exercise 2

Let X be a $2 imes 1$ random vector such that its two entries X_1 and X_2 have expected values[eq57]

Let A be the following $2 imes 2$ matrix of constants:[eq58]

Compute the expected value of the random vector Y defined as follows:[eq59]

Solution

The linearity property of the expected value applies to the multiplication of a constant matrix and a random vector:[eq60]

Exercise 3

Let Sigma be a $2 imes 2$ matrix with random entries, such that all its entries have expected value equal to 1.

Let A be the following $1 imes 2$ constant vector:[eq61]

Compute the expected value of the random vector Y defined as follows:[eq62]

Solution

The linearity property of the expected value operator applies to the multiplication of a constant vector and a matrix with random entries:[eq63]

How to cite

Please cite as:

Taboga, Marco (2021). "Properties of the expected value", Lectures on probability theory and mathematical statistics. Kindle Direct Publishing. Online appendix. https://www.statlect.com/fundamentals-of-probability/expected-value-properties.

The books

Most of the learning materials found on this website are now available in a traditional textbook format.

Featured pages
Explore
Main sections
About
Glossary entries
Share
  • To enhance your privacy,
  • we removed the social buttons,
  • but don't forget to share.