One-Way ANOVA

Main idea

To tell whether several group means really differ, compare how far apart the group means are with how much the values bounce around inside each group. If the gaps between groups are big next to that everyday noise, the means probably aren't all equal.

{\htmlData{sym=0}{F}} = \frac{MS_{\text{between}}}{MS_{\text{within}}}, \qquad MS_{\text{between}} = \frac{{\htmlData{sym=1}{SS_{\text{between}}}}}{{\htmlData{sym=3}{k}} - 1}, \qquad MS_{\text{within}} = \frac{{\htmlData{sym=2}{SS_{\text{within}}}}}{{\htmlData{sym=4}{N}} - {\htmlData{sym=3}{k}}}

What the symbols mean

$F$
F statistic
Near 1 when the group means are equal; large when they differ.
$SS_{\text{between}}$
Between-group sum of squares
Group size times squared distance of each group mean from the grand mean, added up.
$SS_{\text{within}}$
Within-group sum of squares
Squared distances of each value from its own group mean, added up.
$k$
Number of groups
Degrees of freedom between groups: k − 1.
$N$
Total observations
Degrees of freedom within groups: N − k.

Explanation

ANOVA is simply a test of whether three or more groups share the same average, like whether three fertilizers grow plants equally well. The name, short for analysis of variance, sounds like it's about spread rather than averages, and that's because of how it decides: it compares two kinds of variation to judge the means. The formula $F = \frac{MS_{\text{between}}}{MS_{\text{within}}}$ looks heavy, but it's one fraction: $MS_{\text{between}}$ on top (how far the group averages are from each other) and $MS_{\text{within}}$ on the bottom (how much individual values scatter inside their own group, which is just noise). If the top is a lot bigger than the bottom, the groups really differ. Each piece is a sum of squared distances divided by a count, nothing you haven't seen before, so bear with it.

Why not several t-tests? You already know how to compare two means with a t-test, so the obvious move with three groups is to run three of them: A vs B, A vs C, B vs C. The trouble is that each test has its own 5% chance of a false alarm. Run enough of them and the chance that at least one goes off by luck alone balloons. ANOVA asks a single question instead: are all the means equal?

Hypotheses: $H_0: \mu_1 = \mu_2 = \cdots = \mu_k$ . $H_a$ : at least one mean differs.

Where the F comes from. Think about what you'd expect if $H_0$ were true. The group means would still wobble a bit, just from sampling, and how much they wobble depends on how noisy the individual values are. So we estimate the noise two ways. $MS_{\text{within}}$ measures it directly, from how spread out each group is around its own mean. $MS_{\text{between}}$ measures it indirectly, from how spread out the group means are around the grand mean. When $H_0$ holds, both estimate the same thing and their ratio $F$ sits near $1$ . When some mean really is different, the group means get pushed apart, $MS_{\text{between}}$ grows, and $F$ gets large. That's why only large $F$ counts as evidence: the test is right-tailed.

The F distribution depends on two degrees of freedom, $k - 1$ and $N - k$ . The second one is $N - k$ because each group uses up one degree of freedom estimating its own mean, the same way a single sample loses one in $s$ .

Assumptions: independent random samples, roughly normal populations, and similar standard deviations in the groups.

A common mistake. When you reject $H_0$ , it's tempting to say all the means differ. ANOVA doesn't tell you that. It only says at least one mean is off from the rest. To find out which ones, you need follow-up comparisons.

See it

Three groups of three pull apart while the spread inside each group stays exactly the same. Notice that F grows like the square of the separation, so it climbs slowly at first and then fast. Once the means are about 1.3 apart, F passes the critical value 5.14, and the gaps are too big to blame on chance. By the end F is near 19.

The three groups from the example, with each group's mean as a short bar and the grand mean 6.33 as a dashed line. ANOVA asks one thing of this picture: are the bars spread out around the dashed line more than the dots are spread out around their own bars? Here group B's mean of 8 sits well above the rest, while every group's dots stay within 1 of their bar.

Worked examples

Example 1

Groups A $= 4, 5, 6$ ; B $= 7, 8, 9$ ; C $= 5, 6, 7$ . Compute $F$ .

1
The group means are $5, 8, 6$ , and the grand mean of all nine values is $\frac{57}{9} \approx 6.33$ .

Practice

Type your answer and check it. Solve them all to complete the lesson.

Problem 1+5 XP

For the three groups above, find the grand mean.

Problem 2+5 XP

Find $SS_{\text{between}}$ .

Problem 3+5 XP

Find $SS_{\text{within}}$ .

Problem 4+5 XP

Find $F$ .

Problem 5+5 XP

The critical value is $F^* = 5.14$ for $df = (2, 6)$ at $\alpha = 0.05$ . Decision?

Problem 6+5 XP

An experiment compares 3 fertilizers on 12 plots in all. It finds $SS_{\text{between}} = 36$ and $SS_{\text{within}} = 48$ . Compute $F$ .

Problem 7+5 XP

An ANOVA has $k = 4$ groups and $N = 20$ observations. What are the two degrees of freedom of its $F$ statistic? Enter both.

Challenge problem+25 XP

The hardest one here. Get it, and you've got this lesson.

Three groups of three: A = {4, 5, 6}, B = {6, 7, 8} and C = {8, 9, 10}. Compute $F$ .

Where it's used

Agricultural trials compare several fertilizers, drug studies compare doses, and manufacturers compare machines or suppliers with ANOVA.

Three Fertilizers, One F-TestStats · Biology

Logistic Regression