Simpson's Paradox in a Kidney Stone Study

In a 1986 study, treatment A beat treatment B for small kidney stones and for large ones, yet B looked better overall. Doctors had given A mostly to the harder, large-stone cases. Each overall rate is a weighted average of the two, and the weights differ.

How it works

Contingency tables, conditional rates and confounding

In one sentence

By the law of total probability, $P(S \mid A) = P(\text{small} \mid A)\,P(S \mid A, \text{small}) + P(\text{large} \mid A)\,P(S \mid A, \text{large})$ . Simpson's paradox occurs when a comparison that holds within every subgroup reverses in the combined table, because the subgroup weights differ between the groups being compared.

General formula

{\htmlData{sym=0}{\bar p_A}} = {\htmlData{sym=1}{w_A}}\,{\htmlData{sym=2}{s_A}} + (1 - {\htmlData{sym=1}{w_A}})\,{\htmlData{sym=3}{\ell_A}}

An overall rate is a blend. Blend a strong treatment with mostly hard cases and it can look worse than a weaker treatment blended with easy ones.

What the symbols mean

$\bar p_A$
A's overall success rate
Successes over all patients who got treatment A.
$w_A$
A's share of small stones
The fraction of A's patients who had small (easier) stones.
$s_A$
Success rate, small stones
How often A worked on small stones: 93% in the study.
$\ell_A$
Success rate, large stones
How often A worked on large stones: 73% in the study.

Split each treatment's patients by stone size: a contingency table of treatment, size and outcome. Within each size, A has the higher success rate.
A's overall rate is a weighted average of its two rates, $\bar p_A = w_A s_A + (1 - w_A)\ell_A$ , weighted by its mix of patients. The same holds for B.
A got mostly large stones ( $w_A \approx 0.25$ ) and B mostly small ones ( $w_B \approx 0.77$ ), so B's average leans on easy cases.
Stone size is a confounding variable: it affects both which treatment a patient got and how well they did. Randomizing treatments would give both groups the same mix and remove the paradox.

Learn the math

Experimental Design and EthicsStats · Lesson 4, Sampling and Data Contingency Tables and Tree DiagramsStats · Lesson 11, Probability