Bayes' rule

by Marco Taboga, PhD

Bayes' rule, named after the English mathematician Thomas Bayes, is a rule for computing conditional probabilities.

The rule

A formal statement of Bayes' rule follows.

The following example shows how Bayes' rule can be applied in a practical situation.

Example

We built a robot that can detect defective items produced in our factory:

• if an item is defective, it is spotted with 98% probability by the robot;

• when an item is not defective, the robot will not signal any defect with 99% probability.

We draw an item at random from a production lot in which 0,1% of items are defective.

If the robot tells us that the drawn item is defective, what is the probability that the robot is right?

In probabilistic terms, what we know about this problem can be formalized as follows:

Furthermore, the unconditional probability that the robot signals a defective item can be derived using the law of total probability:

Therefore, Bayes' rule gives

Even if the robot is conditionally very accurate, the unconditional probability that the robot is right when he says that an item is defective is less than 10 per cent!

Terminology

The quantities involved in Bayes' ruleoften take the following names:

1. is called prior probability or, simply, prior;

2. is called conditional probability or likelihood;

3. is called marginal probability;

4. is called posterior probability or, simply, posterior.

Solved exercises

Below you can find some problems with explained solutions.

Exercise 1

There are two urns containing colored balls. The first urn contains 50 red balls and 50 blue balls. The second urn contains 30 red balls and 70 blue balls. One of the two urns is randomly chosen (both urns have probability 50% of being chosen) and then a ball is drawn at random from one of the two urns. If a red ball is drawn, what is the probability that it comes from the first urn?

Exercise 2

An economics consulting firm has created a model to predict recessions. The model predicts a recession with probability 80% when a recession is indeed coming and with probability 10% when no recession is coming. The unconditional probability of falling into a recession is 20%. If the model predicts a recession, what is the probability that a recession will indeed come?

Exercise 3

Alice has two coins in her pocket, a fair coin (head on one side and tail on the other side) and a two-headed coin. She picks one at random from her pocket, tosses it and obtains head. What is the probability that she flipped the fair coin?

Read more

If you want to know more about Bayes' rule and how it is used, you can read the following pages:

• prior probability, where you can find another example and an introduction to the concept of prior distribution;

• posterior probability, which also contains an example;

• Bayesian inference, which introduces an important branch of statistics entirely based on Bayes' rule.

How to cite

Please cite as:

Taboga, Marco (2021). "Bayes' rule", Lectures on probability theory and mathematical statistics. Kindle Direct Publishing. Online appendix. https://www.statlect.com/fundamentals-of-probability/Bayes-rule.