Goodness of Fit

Main idea

To check whether counts fit a claimed distribution, work out the counts the claim predicts and measure how far off you landed. The chi-square statistic adds up each squared miss relative to what was expected, so it's near zero for a good fit and big for a bad one.

{\htmlData{sym=0}{\chi^2}} = \sum\frac{({\htmlData{sym=1}{O}} - {\htmlData{sym=2}{E}})^2}{\htmlData{sym=2}{{E}}}, \qquad df = {\htmlData{sym=3}{k}} - 1

What the symbols mean

$\chi^2$
Chi-square statistic
Zero for a perfect fit; large values mean the counts disagree with the claim.
$O$
Observed count
What actually happened in each category.
$E$
Expected count
What the claimed distribution predicts: n times the category's probability.
$k$
Number of categories
Degrees of freedom are one less.

Explanation

A goodness-of-fit test is simply a check of whether counts in several categories match what some claim predicts, like whether a die's 60 rolls fit a fair die. The name says it: how good is the fit between the claim and the data? It's what you reach for when there's no single mean or proportion to test, just a row of counts. The formula $\chi^2 = \sum\frac{(O - E)^2}{E}$ looks intimidating, but it's just three small moves repeated for each category: $O - E$ (how far off the count was), squaring it (so misses up and down both count), and dividing by $E$ (judging the miss against what you expected). Then the $\sum$ adds up all the categories. That's the whole thing; it looks worse than it is.

Why divide by $E$ ? A miss of 5 means a lot when you expected 10, and very little when you expected 1000. Dividing each squared miss $(O - E)^2$ by $E$ puts every category on the same footing. Squaring does the other job: misses above and below expected can't cancel out, so every disagreement adds to $\chi^2$ .

Procedure:

$H_0$ : the claimed distribution is correct. $H_a$ : it isn't.
Expected counts: $E = n \times p_i$ for each category.
Compute $\chi^2$ and compare with the critical value for $df = k - 1$ , or find the p-value. The test is always right-tailed: only big disagreements count against $H_0$ .

The degrees of freedom are $k - 1$ rather than $k$ because the counts have to add up to $n$ . Once you know $k - 1$ of them, the last one is forced.

Condition: every expected count should be at least 5.

The chi-square distribution is skewed right and depends on its degrees of freedom; its mean is $df$ . That gives you a quick sense of scale: with $5$ degrees of freedom, a $\chi^2$ around $5$ is completely ordinary.

A common mistake. When you fail to reject, it's tempting to say the claim has been confirmed. It hasn't. A small $\chi^2$ only means your data don't contradict the claim. A different sample, or a bigger one, might.

See it

k

5.00

c

11.1

This is the chi-square distribution with k degrees of freedom, and the shaded tail beyond c is the p-value for a statistic equal to c. With 5 degrees of freedom the peak sits near 3, and c = 11.07 leaves exactly 5% in the tail. Try raising k: the hump slides right, because the mean of the distribution is k.

Worked examples

Example 1

A die is rolled 60 times, giving counts $5, 8, 9, 8, 10, 20$ for faces 1 to 6. Is it fair? ( $\alpha = 0.05$ , critical value $11.07$ for $df = 5$ .)

1
A fair die lands on each face with probability $\frac16$ , so the expected count is $60 \times \frac16 = 10$ per face.

Practice

Type your answer and check it. Solve them all to complete the lesson.

Problem 1+5 XP

For the 60 die rolls, what is the expected count for each face?

Problem 2+5 XP

Compute $\chi^2$ for the counts $5, 8, 9, 8, 10, 20$ .

Problem 3+5 XP

How many degrees of freedom does the test have?

Problem 4+5 XP

With critical value 11.07, what is the decision?

Problem 5+5 XP

A shop expects equal sales Monday to Friday and sees $22, 18, 20, 25, 15$ . Compute $\chi^2$ .

Problem 6+5 XP

A candy maker says its bags are 30% red, 20% blue and 50% green. A bag of 100 has 22 red, 28 blue and 50 green. Compute $\chi^2$ .

Problem 7+5 XP

For that bag, there are $3 - 1 = 2$ degrees of freedom and the critical value at $\alpha = 0.05$ is 5.991. What's the decision?

Challenge problem+25 XP

The hardest one here. Get it, and you've got this lesson.

Benford's law says the leading digit $d$ of many real-world numbers appears with probability $\log_{10}\left(1 + \frac1d\right)$ . Find the probability of a leading 1, and the number of 500 honest invoices expected to start with 1 or 2. Enter both.

Where it's used

Geneticists test whether offspring match Mendel's ratios, casinos audit dice and cards, and auditors use goodness of fit (Benford's law) to spot fabricated numbers.

Catching Fake Numbers with Benford's LawStats · Economics & Finance

Two Proportions Tests of Independence