Almost sure convergence

This lecture introduces the concept of almost sure (a.s.) convergence, first for sequences of random variables and then for sequences of random vectors.

Table of contents

What you need to know before starting
Weakening pointwise convergence
Definition for sequences of random variables
Example
How to generalize the definition to the multivariate case
Definition for sequences of random vectors
Relation between univariate and multivariate convergence
Solved exercises

What you need to know before starting

In order to understand this lecture, you should understand the concepts of:

almost sure property and almost sure event, explained in the lecture on Zero-probability events;
pointwise convergence of a sequence of random variables, explained in the lecture on Pointwise convergence.

We anyway quickly review both of these concepts below.

Weakening pointwise convergence

Almost sure convergence is defined by weakening the requirements for pointwise convergence.

Let be a sequence of random variables defined on a sample space .

Remember that is pointwise convergent if and only if the sequence of real numbers is convergent for all .

Achieving convergence for all is a very stringent requirement. We weaken it by requiring the convergence of for a large enough subset of , and not necessarily for all .

In particular, we require to be a convergent sequence almost surely: if is the set of all sample points for which the sequence is convergent, its complement $F^{c}$ must be included in a zero-probability event, that is, [eq7]

In other words, almost sure convergence requires that the sequences converge for all sample points , except, possibly, for a very small set $F^{c}$ of sample points.

The set $F^{c}$ is so small that must be included in a zero-probability event.

Definition for sequences of random variables

What we have said so far is summarized by the following definition.

Definition Let be a sequence of random variables defined on a sample space . We say that is almost surely convergent to a random variable defined on if and only if the sequence of real numbers converges to almost surely, that is, if and only if there exists a zero-probability event such that

The variable is called the almost sure limit of the sequence and convergence is indicated by

Example

The following is an example of a sequence that converges almost surely.

Suppose that the sample space is

As discussed in the lecture on Zero-probability events, it is possible to build a probability measure on , such that assigns to each sub-interval of a probability equal to its length:

Remember that in this probability model all the sample points are assigned zero probability.

In other words, each sample point, when considered as an event, is a zero-probability event:

Now, consider a sequence of random variables defined as follows: [eq19]

When , the sequence of real numbers converges to because

However, when , the sequence of real numbers is not convergent to because

Define a constant random variable as follows:

We have that

But because which means that the eventis a zero-probability event.

Therefore, the sequence converges to almost surely.

Note, however, that does not converge pointwise to because does not converge to for all .

How to generalize the definition to the multivariate case

Let be a sequence of random vectors defined on a sample space , where each random vector has dimension .

Also in the case of random vectors, the concept of almost sure convergence is obtained from the concept of pointwise convergence by relaxing the assumption that the sequence converges for all .

Remember that a sequence of real vectors converges to a real vector if and only if where denotes the Euclidean norm.

In the case of almost sure convergence, it is required that the sequence converges for almost all (i.e., almost surely).

Definition for sequences of random vectors

Here is a formal definition for the multivariate case.

Definition Let be a sequence of random vectors defined on a sample space . We say that is almost surely convergent to a random vector defined on if and only if the sequence of real vectors converges to the real vector almost surely, that is, if and only if there exists a zero-probability event such that

Also in the multivariate case, is called the almost sure limit of the sequence and convergence is indicated by

Relation between univariate and multivariate convergence

A sequence of random vectors is almost surely convergent if and only if all the sequences formed by their entries are almost surely convergent.

Proposition Let be a sequence of random vectors defined on a sample space . Denote by the sequence of random variables obtained by taking the -th entry of each random vector . The sequence converges almost surely to the random vector if and only if converges almost surely to the random variable $X_{ullet ,i}$ (the -th entry of ) for each .

Solved exercises

Below you can find some exercises with explained solutions.

Exercise 1

Let the sample space be

Sub-intervals of are assigned a probability equal to their length:

Define a sequence of random variables as follows:

Define a random variable as follows:

Does the sequence converge almost surely to ?

Solution

For a fixed sample point , the sequence of real numbers has limit

For , the sequence of real numbers has limit

Therefore, the sequence of random variables does not converge pointwise to becausefor . However, the set of sample points such that does not converge to is a zero-probability event: Therefore, the sequence converges almost surely to .

Exercise 2

Let and be two sequences of random variables defined on a sample space .

Let and be two random variables defined on such that [eq70]

Prove that

Solution

Denote by $F_{X}$ the set of sample points for which converges to :The fact that converges almost surely to implies thatwhere .

Denote by $F_{Y}$ the set of sample points for which converges to :The fact that converges almost surely to implies thatwhere .

Now, denote by $F_{XY}$ the set of sample points for which converges to :

Observe that if then converges to , because the sum of two sequences of real numbers is convergent if the two sequences are convergent. Therefore,Taking the complement of both sides, we obtain [eq91] But [eq92] and as a consequence . Thus, the set $F_{XY}^{c}$ of sample points such that does not converge to is included in the zero-probability event $E_{X}cup E_{Y}$ , which means that

Exercise 3

Let the sample space be

Sub-intervals of are assigned a probability equal to their length:

Define a sequence of random variables as follows: [eq100]

Find an almost sure limit of the sequence.

Solution

If or , then the sequence of real numbers is not convergent:

For , the sequence of real numbers has limitbecause for any we can find $n_{0}$ such that for any $ngeq n_{0}$ (as a consequence for any $ngeq n_{0}$ ).

Thus, the sequence of random variables converges almost surely to the random variable defined asbecause the set of sample points such that does not converge to is a zero-probability event: [eq112]

How to cite

Please cite as:

Taboga, Marco (2021). "Almost sure convergence", Lectures on probability theory and mathematical statistics. Kindle Direct Publishing. Online appendix. https://www.statlect.com/asymptotic-theory/almost-sure-convergence.