Search for probability and statistics terms on Statlect
StatLect
Index > Fundamentals of probability

Conditional probability with respect to a sigma-algebra

by , PhD

In this lecture we provide a rigorous definition of conditional probability, based on sigma-algebras.

The main advantage of this definition with respect to a more elementary definition is that it allows us to condition on zero-probability events.

Table of Contents

Motivation

In the introduction to conditional probability, we have stated a number of properties that conditional probabilities should satisfy to be rational in some sense.

We have proved that, whenever [eq1], these properties are satisfied if and only if[eq2]

However, we have not been able to derive a formula for probabilities conditional on zero-probability events. In other words, we have not been able to find a way to compute [eq3] when [eq4].

Thus, we have concluded that the above elementary formula cannot be taken as a general definition of conditional probability because it does not cover zero-probability events.

We now discuss a completely general definition of conditional probability, which covers also the case in which [eq4]. The resulting concept is called conditional probability with respect to a sigma-algebra.

Roadmap

The plan of the lecture is as follows:

  1. We define the concept of a partition of events.

  2. We show that, given a partition of events, conditional probability can be regarded as a random variable (probability conditional on a partition).

  3. We show that, when no zero-probability events are involved, probabilities conditional on a partition satisfy a certain property (the fundamental property of conditional probability).

  4. We require that the fundamental property of conditional probability be satisfied also when zero-probability events are involved. We show that this requirement is sufficient to unambiguously pin down probabilities conditional on a partition; therefore, it can be used to provide a completely general definition of conditional probability.

  5. Finally, we discuss how to replace partitions with sigma-algebras.

Partitions of events

Let Omega be a sample space and let [eq6] denote the probability assigned to the events $Esubseteq Omega $.

Define a partition of events of Omega as follows.

Definition Let G be a collection of non-empty subsets of Omega. G is called a partition of events of Omega if

  1. all subsets $Gin QTR{cal}{G}$ are events;

  2. if [eq7] then either $G=F$ or $Gcap F=emptyset $;

  3. [eq8].

In other words, a partition G is a subdivision of Omega into non-overlapping and non-empty events that cover all of Omega.

Example Consider the sample space[eq9]which describes the possible outcomes of the toss of a die. Define the two events[eq10]Then, [eq11] is a partition of events of Omega. In fact,[eq12]and [eq13].

Probability conditional on a partition

Take a finite partition [eq14] of events of Omega, such that [eq15] for every i.

Suppose that at a certain time in the future we will be told that the realized outcome belongs to a set $G_{i}$.

After receiving this information, we will update our assessment of the probability of an event E, by computing its conditional probability [eq16]

Before receiving the information, this conditional probability is unknown and can be regarded as a random variable, denoted by [eq17] and defined as follows:[eq18]

The random variable [eq19] is the probability of E conditional on the partition G.

Example Let us continue with the example above, where the sample space is[eq20]We consider the partition [eq11] where[eq10]We assign equal probability to all the outcomes:[eq23]We would like to analyze the conditional probability of the event [eq24]We have[eq25]The conditional probability [eq26] is a discrete random variable defined as follows:[eq27]Since [eq28], the probability mass function of [eq17] is[eq30]

The fundamental property of conditional probability

A property of [eq17] is that its expected value equals the unconditional probability [eq32]:[eq33]

Proof

This is proved as follows:[eq34]

Example In the previous example, the probability mass function of [eq35] was[eq30] It is easy to verify that [eq37]

The property above can be generalized as follows.

Proposition (Fundamental property) Let [eq38] be a finite partition of events of Omega such that [eq39] for every i. Let H be any event obtained as a union of events [eq40]. Let $1_{H}$ be the indicator function of H. Let [eq17] be defined as above. Then,[eq42]

Proof

Without loss of generality, we can assume that H is obtained as the union of the first k ($kleq n$) sets of the partition G (we can always rearrange the sets $G_{i}$ by changing their indices):[eq43]First note that[eq44]The property is proved as follows:[eq45]

The fundamental property as a defining property

Suppose that we are not able to explicitly define [eq46] because G contains a zero-probability event $G$ and, therefore, we cannot use the formula [eq2]to define [eq48] for $omega in G$.

When we are not able to explicitly define [eq49], what we can do is to define [eq46] implicitly, by requiring that it satisfies the fundamental property of conditional probability[eq42]for all events H obtained as unions of events $Gin QTR{cal}{G}$.

How can we be sure that there exists a random variable [eq46] satisfying this property?

Existence is guaranteed by the following important theorem, that we state without providing a proof.

Proposition Let G be an arbitrary partition of events of Omega. Let $Ein Omega $ be an event. Then, there exists at least one random variable Y that satisfies the property[eq53]for all the events H obtained as unions of events $Gin QTR{cal}{G}$. Furthermore, if $Y_{1}$ and $Y_{2}$ are two random variables such that[eq54]for all H, then $Y_{1}$ and $Y_{2}$ are almost surely equal.

Thus, a random variable Y satisfying the fundamental property of conditional probability exists and is unique (up to almost sure equality).

As a consequence, we can indirectly define the probability of an event E conditional on the partition G as [eq55] Y.

This indirect way of defining conditional probability is summarized as follows..

Definition (Probability conditional on a partition) Let G be a partition of events of Omega. Let $Ein Omega $ be an event. The probability of E conditional on the partition G is a random variable [eq56] that satisfies[eq42]for all events H obtainable as unions of events $Gin QTR{cal}{G}$.

As we have seen above, such a random variable is guaranteed to exist and is unique up to almost sure equality.

Different random variables satisfying the criterion in the definition are called versions of the conditional probability.

Conditioning with respect to sigma-algebras

In rigorous probability theory, conditional probability is defined with respect to sigma-algebras, rather than with respect to partitions.

Let [eq58] be a probability space.

Let $QTR{cal}{I}$ be a sub-sigma-algebra of $ ciFourier $ (i.e., $QTR{cal}{I}$ is a sigma-algebra and [eq59]).

Let $Ein ciFourier $ be an event.

We say that a random variable [eq60] is a conditional probability of E with respect to the sigma-algebra $I,$ if and only if[eq61]

It can be shown that this definition is equivalent to our definition of probability conditional on a partition.

In particular, if G is a partition of events of Omega and $QTR{cal}{I}$ is the smallest sigma-algebra containing all the possible unions of events $Gin QTR{cal}{G}$, then [eq62] is the same as [eq63].

How to interpret conditioning with respect to sigma-algebras

Thus, when we see the abstract definition of conditional probability with respect to a sigma-algebra $QTR{cal}{I}$, we can think about it as follows:

  1. there is a partition G of the sample space;

  2. the sigma-algebra $QTR{cal}{I}$ is the smallest sigma-algebra that contains all the elements of G;

  3. at some future time we will be told that the realized outcome belongs to a set [eq64];

  4. at that time, we will be able to compute a conditional probability [eq65];

  5. until that time, this conditional probability is unknown and it can be regarded as a random variable, denoted by [eq63].

Finer and coarser sigma-algebras

Let us get back to our toss-of-a-die example, in which the sample space is[eq67]

Consider the partition[eq68]

The smallest sigma-algebra containing G is[eq69]

Now, consider the partition[eq70]

The smallest sigma-algebra containing [eq71] is[eq72]

Since [eq73] contains all the sets included in $QTR{cal}{I}$ plus some more, we say that [eq73] is finer than $QTR{cal}{I}$. Conversely, we say that $QTR{cal}{I}$ is coarser than [eq75].

We also say that the conditional probability [eq76] is based on a larger information set as compared to [eq77]. In intuitive terms, this means that the information we receive when we are able to actually calculate the conditional probability is more precise.

Regular conditional probabilities

Until now, we have kept fixed the event E in the conditional probability [eq78]. Moreover, we have regarded [eq77] as a random variable.

If we allow E to vary, [eq80] becomes a random probability measure.

Or does it? Mathematicians have found some examples (see here), in which, despite a careful choice of the versions of [eq81], it is not possible to simultaneously satisfy the two requirements that

  1. [eq77] be $QTR{cal}{I}$-measurable (i.e., a proper random variable) for each choice of E;

  2. [eq83] be a proper probability measure for each choice of omega (except possibly for some omega forming a set of measure zero).

When these two requirements can be satisfied, we say that the probability space [eq84] admits a regular probability conditional on the sigma-algebra $QTR{cal}{I}$.

Applications

The abstract definition of conditional probability with respect to a sigma-algebra is extremely useful.

One of its most important applications is the derivation of conditional probability density functions for continuous random vectors. To learn about this application, read the next lecture, on Conditional probability distributions.

How to cite

Please cite as:

Taboga, Marco (2021). "Conditional probability with respect to a sigma-algebra", Lectures on probability theory and mathematical statistics. Kindle Direct Publishing. Online appendix. https://www.statlect.com/fundamentals-of-probability/conditional-probability-as-a-random-variable.

The books

Most of the learning materials found on this website are now available in a traditional textbook format.

Featured pages
Explore
Main sections
About
Glossary entries
Share
  • To enhance your privacy,
  • we removed the social buttons,
  • but don't forget to share.