Bayes' Theorem

Bayes' theorem is simply a rule for updating a probability when you get new evidence, named after Thomas Bayes, the English minister who first worked it out. It looks like a jumble of conditionals, but it's a fraction with a clear story: on top, $P(B \mid A)$ (how likely the evidence is if $A$ is true) times $P(A)$ (how likely $A$ was before you saw anything); on the bottom, $P(B)$ (how likely the evidence is overall). The total-probability line just computes that bottom by adding up the two ways $B$ can happen, with $A$ and without it. It's nothing more than the multiplication rule rearranged, so hang in there.

Where it comes from. You already know $P(A \cap B) = P(A)\,P(B \mid A)$. But you could just as well write it the other way: $P(A \cap B) = P(B)\,P(A \mid B)$. Set the two equal and divide by $P(B)$, and that's Bayes' theorem.

Medical tests. Suppose 1% of people have a disease, the test catches 95% of cases (sensitivity), and it wrongly flags 5% of healthy people (false positive rate). It's easiest to think in people rather than formulas. Out of 10,000 people:

• 100 are sick, and 95 of them test positive.
• 9,900 are healthy, and 495 of them test positive.

So $P(\text{sick} \mid +) = \frac{95}{95 + 495} \approx 16\%$. Think about why it's so low. The test is good, but the healthy group is 99 times bigger than the sick group, so even its small 5% error rate produces five times as many false alarms as true positives.

A common mistake. It's tempting to think a 95% accurate test means a positive result is 95% likely to be right. That ignores the prior, how common the disease is, and it's called the base-rate fallacy. A test's accuracy alone doesn't tell you what a positive result means; you need to know how common the condition is.

Spam filters, medical diagnosis, search-and-rescue planning and machine learning classifiers all update probabilities with Bayes' theorem.