Conditional Probability and Bayes’ Rule

Videos

Marginalization and Total Probability (13:33)
Bayes’ Rule (19:38)

Special guest video!

Bayes’ Theorem, the Geometry of Changing Beliefs (15:11) by 3Blue1Brown (Grant Sanderson)

Reading

Connecting Discrete Mathematics and Computer Science by David Liben-Nowell, Chapter 10, pages 36-51

Warmup

Problem 1

Suppose that I have three coins, \(C_1\), \(C_2\), and \(C_3\). You know that \(C_1\) is a biased coin with probability of heads equal to \(p_1\), \(C_2\) is a biased coin with probability of heads equal to \(p_2\), and \(C_3\) is a biased coin with probability of heads equal to \(p_3\).

Away from your view, I first flip coin \(C_1\). If it comes up heads, I flip coin \(C_2\); if it comes up tails, I flip coin \(C_3\). I then report to you whether or not the second coin (i.e. \(C_2\) or \(C_3\)) came up heads. You know the values of \(p_1\), \(p_2\), and \(p_3\), but you do not know which coin I flipped second!

Please answer the following questions. Your answers should all be formulae in terms of \(p_1\), \(p_2\), and \(p_3\).

Part A

What is the probability that the second coin comes up heads?

Part B

I report to you that the second coin came up tails. What is the probability that the second coin was \(C_3\)?

Part C

If \(p_1 = 7/8\), then you should have a strong prior belief that the coin I flipped was probably \(C_2\). However, if I then give you additional data that the second coin came up tails, you should update that belief. Suppose that \(p_2 = 3/4\) and \(p_3 = 1/4\). Use your formula from Part B to compute the probability that the second coin was \(C_3\) given that it came up tails.

Problem 2

Recall that events are sets, so that we can talk about the probability that event \(A\) does not happen as \(\mathbb{P}(\bar{A})\).

Write a short proof showing that events \(A\) and \(B\) are independent if and only if the events \(\bar{A}\) and \(B\) are independent.