Two Independent Means

A two-sample t-test is simply a check of whether two separate groups, like students taught by two different methods, have different averages, or whether the gap you see could just be luck. The formula looks tall, but it's the same "signal over noise" as the one-sample t-test: on top, $\bar x_1 - \bar x_2$ (the gap between the two sample means), and on the bottom, $\sqrt{\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2}}$ (each group's squared standard error, added together, then square-rooted). It's two familiar standard errors combined into one, so bear with it.

Hypotheses: $H_0: \mu_1 = \mu_2$ (no difference) against $\mu_1 \ne \mu_2$, $\mu_1 > \mu_2$ or $\mu_1 < \mu_2$.

Why variances add. You might expect the uncertainties to partly cancel when you subtract. They don't. Each sample mean can land a bit too high or too low on its own, and either kind of miss makes the gap come out wrong. So the difference of two independent quantities varies more than either one: $\text{Var}(\bar x_1 - \bar x_2) = \frac{\sigma_1^2}{n_1} + \frac{\sigma_2^2}{n_2}$. It's the same rule we used for totals in the CLT unit: variances add, standard deviations don't.

Degrees of freedom. Software computes them with Welch's formula; by hand, a safe conservative choice is the smaller of $n_1 - 1$ and $n_2 - 1$.

Independent samples means the groups are unrelated individuals: two classes, a treatment group and a control group.

A common mistake. It's tempting to judge a difference by whether the two groups' distributions overlap. Heavy overlap feels like "no real difference." But the test compares the gap with the standard error of the means, which shrinks as the samples grow. With 40 people per group, two groups whose values overlap almost completely can still differ significantly; there's a picture of this further down.

Clinical trials compare treatment and control groups; A/B tests compare two website versions; economists compare wages between regions.