Testing Variability: χ² and F Tests

Main idea

Consistency is a spec too. To test whether a process scatters more than allowed, compare $\chi^2_0 = (n-1)s^2/\sigma_0^2$ with the $\chi^2_{n-1}$ distribution; to compare two processes' scatter, use the ratio $F_0 = s_1^2/s_2^2$ . Both lean hard on normal data, far harder than t tests do.

One variance

{\htmlData{sym=0}{\chi^2_0}} = \frac{(n-1)\,s^2}{{\htmlData{sym=1}{\sigma_0^2}}}, \qquad \nu = n - 1

What the symbols mean

$\chi^2_0$
Test statistic
About n − 1 if H₀ is true; large values mean too much spread.
$\sigma_0^2$
Required variance
The spec value in H₀, squared.

Two variances

{\htmlData{sym=0}{F_0}} = \frac{s_1^2}{s_2^2} \sim {\htmlData{sym=1}{F_{n_1 - 1,\,n_2 - 1}}} \ \text{ when } \sigma_1 = \sigma_2

What the symbols mean

$F_0$
Variance ratio
Near 1 when the two processes are equally consistent.
$F_{n_1 - 1,\,n_2 - 1}$
F distribution
Numerator degrees of freedom first, then denominator.

Lower F points from upper ones

F_{1-\alpha,\,\nu_1,\,\nu_2} = \frac{1}{{\htmlData{sym=0}{F_{\alpha,\,\nu_2,\,\nu_1}}}}

What the symbols mean

$F_{\alpha,\,\nu_2,\,\nu_1}$
Upper-tail value
Leaves area α to its right. Note the swapped degrees of freedom.

Confidence interval for σ₁²/σ₂²

\frac{s_1^2}{s_2^2}\,F_{1-\alpha/2,\,\nu_2,\,\nu_1} \le {\htmlData{sym=0}{\frac{\sigma_1^2}{\sigma_2^2}}} \le \frac{s_1^2}{s_2^2}\,F_{\alpha/2,\,\nu_2,\,\nu_1}

What the symbols mean

$\frac{\sigma_1^2}{\sigma_2^2}$
True variance ratio
Take square roots for a statement about σ₁/σ₂.

Explanation

A variance test is simply a check of whether a process is more erratic than it's allowed to be, or whether one machine is more erratic than another. There are two statistics, and each is a single ratio. $\chi^2_0 = (n-1)s^2/\sigma_0^2$ compares the variance you measured with the variance the spec allows (scaled by the degrees of freedom $n - 1$ ). $F_0 = s_1^2/s_2^2$ just divides one machine's variance by the other's; near 1 means similar consistency. Two ratios, two tables to look them up in.

Why test spread. A filling line can average 500 g and still overfill and underfill constantly; a bearing race can be centered and still too inconsistent for its press fit. Specs on σ come from capability targets (process capability) and from machine purchase contracts.

One variance. If $H_0: \sigma = \sigma_0$ is true, $(n-1)s^2/\sigma_0^2 \sim \chi^2_{n-1}$ : the pivot from intervals for variability with $\sigma_0$ plugged in. The usual worry is too much spread, $H_1: \sigma > \sigma_0$ , rejected when $\chi^2_0 > \chi^2_{\alpha,n-1}$ . To demonstrate that a machine is good enough, put the claim in $H_1: \sigma < \sigma_0$ and reject when $\chi^2_0 < \chi^2_{1-\alpha,n-1}$ ; that happens exactly when the upper confidence bound for σ falls below $\sigma_0$ .

Two variances: where F comes from. The F distribution with $(\nu_1, \nu_2)$ degrees of freedom is defined as $(\chi^2_{\nu_1}/\nu_1)/(\chi^2_{\nu_2}/\nu_2)$ for independent χ²'s. Since $s_i^2/\sigma_i^2 = \chi^2_{\nu_i}/\nu_i$ with $\nu_i = n_i - 1$ ,

\frac{s_1^2/\sigma_1^2}{s_2^2/\sigma_2^2} \sim F_{n_1-1,\,n_2-1},

and when $\sigma_1 = \sigma_2$ the σ's cancel, leaving $F_0 = s_1^2/s_2^2$ . Solving the same statement for $\sigma_1^2/\sigma_2^2$ gives the confidence interval above.

The lower-tail trick. Tables list only upper points. Flipping the ratio swaps the degrees of freedom: if $F \sim F_{\nu_1,\nu_2}$ then $1/F \sim F_{\nu_2,\nu_1}$ , so $F_{1-\alpha,\nu_1,\nu_2} = 1/F_{\alpha,\nu_2,\nu_1}$ . For a two-sided test, put the larger variance on top and compare with $F_{\alpha/2}$ .

The catch: normality. A t test leans on the CLT for $\bar x$ and shrugs off skew. χ² and F tests are built from squared deviations of individual values, and heavy tails wreck them: with heavy-tailed data a nominal 5% F test can reject 15% to 20% of the time, and more data doesn't fix it. Check probability plots first, and for messy data use a robust alternative such as Levene's test.

Power is low. Variances are hard to pin down (ten parts only locate σ between about $0.69s$ and $1.83s$ ), so catching a 50% increase in σ with 90% power takes 27 parts. For an even ν = 2m the χ² tail has a closed form, because $\chi^2_{2m}/2$ is the waiting time for the $m$ -th event of a Poisson process with rate 1 (exponential lifetimes). The tail is the chance of fewer than $m$ events by time $c/2$ :

P(\chi^2_{2m} > c) = \sum_{k=0}^{m-1} e^{-c/2}\frac{(c/2)^k}{k!}.

Connections: the same F ratio, between-group variance over within-group variance, drives the F test in experiment principles and one-way ANOVA.

See it

Comparing two machines' variability with F₀ = s₁²/s₂², from 11 parts each. The solid curve is the F distribution with 10 and 10 degrees of freedom, and the dashed curve gives the two-sided p-value for every possible observed ratio. The shaded tail is the more extreme side of the observed F₀, and p drops below 0.05 exactly at the critical values 3.72 and 1/3.72 = 0.27. Swapping which machine goes on top just turns F₀ into 1/F₀, with the same p.

Power of the χ² test for a variance increase, with 11 parts (ν = 10) at α = 0.05. One curve is the statistic's distribution when σ is on spec, the other its distribution when σ = λσ₀, stretched out by λ², and the power is the stretched curve's area beyond the cutoff. As λ grows that area grows, but slowly: a 50% increase in σ gets caught only about 62% of the time, because variances are hard to pin down with few parts.

Worked examples

Example 1

A filling line's spec requires $\sigma \le 0.8$ g. Twenty bottles give $s = 1.05$ g. Test $H_0: \sigma = 0.8$ against $H_1: \sigma > 0.8$ at $\alpha = 0.05$ ( $\chi^2_{0.05,19} = 30.144$ ).

1
$\chi^2_0 = \frac{19 \times 1.05^2}{0.8^2} = \frac{20.95}{0.64} = 32.73$ .

Example 2

Two lathes: lathe 1, $n_1 = 10$ , $s_1 = 3.2$ µm; lathe 2, $n_2 = 13$ , $s_2 = 1.9$ µm. Is lathe 1 more variable? Compare a two-sided test at $\alpha = 0.05$ ( $F_{0.025,9,12} = 3.436$ ) with the one-sided test the engineer planned beforehand ( $F_{0.05,9,12} = 2.796$ ).

1
$F_0 = 3.2^2/1.9^2 = 10.24/3.61 = 2.84$ , with 9 and 12 degrees of freedom.

Practice

Type your answer and check it. Solve them all to complete the lesson.

Problem 1+5 XP

A torque wrench's calibration spec requires $\sigma \le 0.25$ N·m. Sixteen repeat readings give $s = 0.34$ N·m. Find $\chi^2_0$ for $H_1: \sigma > 0.25$ .

Problem 2+5 XP

With $\chi^2_{0.05,15} = 24.996$ , what is the decision for the wrench at $\alpha = 0.05$ ?

Problem 3+5 XP

A new grinding wheel is claimed to bring σ below 2.0 µm. Twenty-five parts give $s = 1.5$ µm. Find $\chi^2_0$ for $H_0: \sigma = 2.0$ against $H_1: \sigma < 2.0$ . Is the claim demonstrated at $\alpha = 0.05$ ( $\chi^2_{0.95,24} = 13.848$ )?

Problem 4+5 XP

Spring rates from two suppliers: A, $n = 12$ , $s = 0.42$ N/mm; B, $n = 15$ , $s = 0.25$ N/mm. Find $F_0$ with the larger variance on top. At $\alpha = 0.05$ two-sided ( $F_{0.025,11,14} = 3.095$ ), do the suppliers differ in consistency?

Problem 5+5 XP

Find the lower critical value $F_{0.975,11,14}$ given $F_{0.025,14,11} = 3.359$ . Give 3 decimal places.

Problem 6+5 XP

For the springs, find the 95% confidence interval for $\sigma_A/\sigma_B$ ( $F_{0.025,11,14} = 3.095$ , $F_{0.025,14,11} = 3.359$ ). Enter the lower and upper limits, to 3 significant figures.

Problem 7+5 XP

Three stacked plates have $\sigma = 12$ , 10 and 8 µm, independent according to the supplier, so the root-sum-square stack-up predicts the assembly's σ. Sixteen measured assemblies give $s = 25$ µm. Find $\chi^2_0$ for $H_0: \sigma = \sigma_{\text{RSS}}$ against $H_1: \sigma > \sigma_{\text{RSS}}$ . What does $\chi^2_{0.05,15} = 24.996$ say?

Challenge problem+25 XP

The hardest one here. Get it, and you've got this lesson.

A χ² test of $H_0: \sigma = \sigma_0$ against $H_1: \sigma > \sigma_0$ uses $n = 13$ parts at $\alpha = 0.05$ ( $\chi^2_{0.05,12} = 21.026$ ). What is its power if the true σ is $1.4\sigma_0$ ? Use the Poisson-sum formula for the χ² tail, and give 3 decimal places.

Where it's used

Machine acceptance contracts ("σ of 30 trial parts must not exceed 2 µm"), gauge and instrument qualification, supplier comparisons and the choice between pooled and Welch t tests all rest on these tests. In variation-reduction projects a significant drop in σ is often the goal itself, and a χ² or F test, backed by a probability plot, is how the improvement gets confirmed.

How Many Resistors Meet Spec?Stats · Electrical Engineering

Comparing Two Processes Calibration Curves

Testing Variability: χ² and F Tests

Main idea

One variance

{\htmlData{sym=0}{\chi^2_0}} = \frac{(n-1)\,s^2}{{\htmlData{sym=1}{\sigma_0^2}}}, \qquad \nu = n - 1

What the symbols mean

$\chi^2_0$
Test statistic
About n − 1 if H₀ is true; large values mean too much spread.
$\sigma_0^2$
Required variance
The spec value in H₀, squared.

Two variances

{\htmlData{sym=0}{F_0}} = \frac{s_1^2}{s_2^2} \sim {\htmlData{sym=1}{F_{n_1 - 1,\,n_2 - 1}}} \ \text{ when } \sigma_1 = \sigma_2

What the symbols mean

$F_0$
Variance ratio
Near 1 when the two processes are equally consistent.
$F_{n_1 - 1,\,n_2 - 1}$
F distribution
Numerator degrees of freedom first, then denominator.

Lower F points from upper ones

F_{1-\alpha,\,\nu_1,\,\nu_2} = \frac{1}{{\htmlData{sym=0}{F_{\alpha,\,\nu_2,\,\nu_1}}}}

What the symbols mean

$F_{\alpha,\,\nu_2,\,\nu_1}$
Upper-tail value
Leaves area α to its right. Note the swapped degrees of freedom.

Confidence interval for σ₁²/σ₂²

\frac{s_1^2}{s_2^2}\,F_{1-\alpha/2,\,\nu_2,\,\nu_1} \le {\htmlData{sym=0}{\frac{\sigma_1^2}{\sigma_2^2}}} \le \frac{s_1^2}{s_2^2}\,F_{\alpha/2,\,\nu_2,\,\nu_1}

What the symbols mean

$\frac{\sigma_1^2}{\sigma_2^2}$
True variance ratio
Take square roots for a statement about σ₁/σ₂.

Explanation

\frac{s_1^2/\sigma_1^2}{s_2^2/\sigma_2^2} \sim F_{n_1-1,\,n_2-1},

and when $\sigma_1 = \sigma_2$ the σ's cancel, leaving $F_0 = s_1^2/s_2^2$ . Solving the same statement for $\sigma_1^2/\sigma_2^2$ gives the confidence interval above.

P(\chi^2_{2m} > c) = \sum_{k=0}^{m-1} e^{-c/2}\frac{(c/2)^k}{k!}.

Connections: the same F ratio, between-group variance over within-group variance, drives the F test in experiment principles and one-way ANOVA.

See it

Worked examples

Example 1

A filling line's spec requires $\sigma \le 0.8$ g. Twenty bottles give $s = 1.05$ g. Test $H_0: \sigma = 0.8$ against $H_1: \sigma > 0.8$ at $\alpha = 0.05$ ( $\chi^2_{0.05,19} = 30.144$ ).

1
$\chi^2_0 = \frac{19 \times 1.05^2}{0.8^2} = \frac{20.95}{0.64} = 32.73$ .

Example 2

1
$F_0 = 3.2^2/1.9^2 = 10.24/3.61 = 2.84$ , with 9 and 12 degrees of freedom.

Practice

Type your answer and check it. Solve them all to complete the lesson.

Problem 1+5 XP

A torque wrench's calibration spec requires $\sigma \le 0.25$ N·m. Sixteen repeat readings give $s = 0.34$ N·m. Find $\chi^2_0$ for $H_1: \sigma > 0.25$ .

Problem 2+5 XP

With $\chi^2_{0.05,15} = 24.996$ , what is the decision for the wrench at $\alpha = 0.05$ ?

Problem 3+5 XP

Problem 4+5 XP

Problem 5+5 XP

Find the lower critical value $F_{0.975,11,14}$ given $F_{0.025,14,11} = 3.359$ . Give 3 decimal places.

Problem 6+5 XP

For the springs, find the 95% confidence interval for $\sigma_A/\sigma_B$ ( $F_{0.025,11,14} = 3.095$ , $F_{0.025,14,11} = 3.359$ ). Enter the lower and upper limits, to 3 significant figures.

Problem 7+5 XP

Challenge problem+25 XP

The hardest one here. Get it, and you've got this lesson.

Where it's used

How Many Resistors Meet Spec?Stats · Electrical Engineering