Comparing Two Processes

Comparing two processes is simply asking whether one really performs differently from the other, or whether the gap could be luck. You saw both tools in the intro course's two-sample lessons. Welch's t is the gap between the two sample means divided by $\sqrt{s_1^2/n_1 + s_2^2/n_2}$ (the two squared standard errors added). The paired t is the average difference divided by its own standard error, used when the same units go through both conditions. The real skill is choosing the design, and that choice can matter more than any formula.

Two ways to run the comparison. A plant wants to know whether a new cutting insert changes surface roughness. Option 1: cut 12 bars with the old insert and 12 other bars with the new one, which gives independent samples. Option 2: cut each of 12 bars half with each insert, which gives pairs. The analysis has to match the design.

Independent samples: Welch's t. $\bar x_1 - \bar x_2$ has variance $\sigma_1^2/n_1 + \sigma_2^2/n_2$: variances of independent quantities add even when you subtract, the rule $\operatorname{Var}(X - Y) = \operatorname{Var}X + \operatorname{Var}Y$ from tolerance stack-up. Estimating each variance separately, with no equal-variance assumption, gives Welch's statistic.

Where Welch's ν comes from. The estimated variance $V = s_1^2/n_1 + s_2^2/n_2$ isn't exactly a scaled χ², so Welch matches it to one with the same mean and variance. Each $s_i^2$ has variance $2\sigma_i^4/(n_i - 1)$ (because $\operatorname{Var}(\chi^2_\nu) = 2\nu$, from intervals for variability), so $\operatorname{Var}(V) = \sum 2(\sigma_i^2/n_i)^2/(n_i - 1)$. A scaled $\chi^2_\nu$ with mean $E(V)$ has variance $2E(V)^2/\nu$. Set the two equal, solve for ν and plug in the sample variances: that's the formula. It always lands between the smaller $n_i - 1$ and $n_1 + n_2 - 2$.

Paired designs. When each unit gets both treatments, compute $d_i = x_{1i} - x_{2i}$ and run a one-sample t on the d's. Why it's stronger: $\operatorname{Var}(X_1 - X_2) = \sigma_1^2 + \sigma_2^2 - 2\rho\sigma_1\sigma_2$, and unit-to-unit variation makes the correlation ρ large, so the differences are far less noisy than either measurement (matched pairs). Pairing is the simplest case of blocking (randomization, replication and blocking). Randomize which condition each unit sees first, so a warm-up or wear trend can't masquerade as the effect.

Report the interval. "Significantly different" is half an answer; the interval for $\mu_1 - \mu_2$ says whether the difference is worth acting on. Analyzing paired data as independent throws away the pairing and buries the effect; analyzing independent data as paired is simply wrong, because there are no pairs to subtract.

The pooled t uses $s_p^2 = [(n_1 - 1)s_1^2 + (n_2 - 1)s_2^2]/(n_1 + n_2 - 2)$ with $n_1 + n_2 - 2$ degrees of freedom, and it assumes equal variances. An F test (testing variability) can check that, but Welch is the safer default and loses almost nothing when the variances really are equal.

Choosing between suppliers, qualifying a second production line, validating a new fixture or tool and comparing two gauges all come down to these comparisons. Engineers pair whenever they can (the same parts through both gauges, the same machines before and after a change), because pairing turns an expensive study into a cheap one, and they report the interval for the difference so a decision can weigh its size against its cost.