Randomization, Replication and Blocking

A designed experiment is simply a test you plan in advance so that, when something changes in the results, you know what caused it. All we're really saying in this lesson is: don't let anything except the factor you're testing line up with your treatments. The three principles each guard against one way that can happen: randomization (shuffle the order, so a drift over the day can't masquerade as an effect), replication (run each setting more than once, so you can measure the noise) and blocking (compare treatments within each batch or day, so batch differences drop out). No new formulas, just three sensible habits.

The scene. A team compares three plasma power settings for etching silicon wafers. The convenient plan, setting 1 in the morning, setting 2 at noon and setting 3 in the afternoon, has a flaw: the chamber walls slowly coat during the day and the etch rate drifts. Any difference they find could be the setting or the clock. Three principles turn a comparison into an experiment.

Randomization: cut the link to everything you didn't control. Assign the runs to a random order. A drift, a warming oven or a tiring operator then lands on all treatments alike, on average, instead of piling onto whichever came last. Randomization also makes the error term honest: the F test's assumption of independent errors is justified by the coin flips themselves. (The intro course's experimental design lesson made the same point for studies on people.)

Replication: measure the noise. A replicate is a complete, independent repeat of a run: a new wafer, the recipe set up again. Measuring the same wafer three times is not replication; it only measures the gauge, the repeatability of measurement error. Replicates estimate the error variance every effect must beat, and averaging $n$ of them shrinks a treatment mean's standard error to $\sigma/\sqrt n$.

Blocking: remove a nuisance you know about. If raw material comes in batches that differ, run every treatment within each batch and compare treatments inside batches. The batch-to-batch differences cancel out of every comparison and leave the error term. The rule of thumb: block what you can, randomize what you can't.

The one-factor F test, and why it works. For $a$ treatments with $n$ replicates each, split the variation as in one-way ANOVA: $SS_{\text{tr}} = n\sum (\bar y_{i\cdot} - \bar y_{\cdot\cdot})^2$ between treatments and $SS_E = \sum\sum (y_{ij} - \bar y_{i\cdot})^2$ within them. Why compare their mean squares? Each treatment mean has variance $\sigma^2/n$. If the treatments are all alike, the treatment means scatter only that much, so $n$ times their sample variance, which is $MS_{\text{tr}}$, estimates $\sigma^2$, exactly what $MS_E$ estimates. Real effects spread the means further and add $n\sum\tau_i^2/(a - 1)$. So $F \approx 1$ when nothing is going on and grows with the effects; reject $H_0$ when $F$ beats the critical value $F_{\alpha, a - 1, N - a}$, the same F distribution as in the variance tests.

Blocking in the ANOVA. In a randomized complete block design each block contains every treatment once, in random order. The block sum of squares $SS_{\text{bl}} = a\sum (\bar y_{\cdot j} - \bar y_{\cdot\cdot})^2$ comes out of what would otherwise be error, which leaves $(a - 1)(b - 1)$ error degrees of freedom for $b$ blocks. Blocking costs $b - 1$ degrees of freedom, so it pays only when the blocks really differ; when they do, as the second picture shows, it can turn a hopeless test into a clear one.

A common mistake. It's tempting to measure one part three times and call that three replicates. It's one replicate measured three times: repeat measurements only show gauge noise, not the run-to-run variation the effects have to beat. Watch for the related slips too: running treatments in a convenient order, analyzing a blocked experiment as if it weren't blocked (or the reverse), and changing several things between runs while you're at it.

Semiconductor fabs randomize the order of wafers through a tool and block on wafer lots; pharmaceutical and food plants block on raw-material batches; agricultural trials, where these ideas began, block on fields. Test labs insist on true replicates (new specimen, new setup) because pseudo-replication is one of the most common reasons an experimental result fails to reproduce. One-factor ANOVA with an F test is the standard first analysis for comparing suppliers, machines or recipes.