Is “Free” a Spam Word? A Test of Independence

Sort emails two ways: spam or not, and containing the word “free” or not. If the word were unrelated to spam, both columns would split the same way. The chi-square statistic measures how far the table strays from that, and spam filters keep the words that stray most.

How it works

The chi-square test of independence

In one sentence

For an $r \times c$ table of counts, the test of independence compares each observed count $O$ with $E = (\text{row total})(\text{column total})/n$ , the count expected if the two variables were independent. Under $H_0$ , $\chi^2 = \sum (O-E)^2/E$ approximately follows the chi-square distribution with $(r-1)(c-1)$ degrees of freedom.

General formula

{\htmlData{sym=0}{\chi^2}} = \sum_{\text{cells}}\frac{({\htmlData{sym=1}{O}} - {\htmlData{sym=2}{E}})^2}{\htmlData{sym=2}{{E}}}, \qquad {\htmlData{sym=2}{E}} = \frac{(\text{row total})(\text{column total})}{\htmlData{sym=3}{{n}}}

Independence means knowing one variable tells you nothing about the other, so every column would split the same way. The test measures how unequal the splits are.

What the symbols mean

$\chi^2$
Chi-square statistic
The total scaled squared gap between the table and what independence predicts.
$O$
Observed count
How many emails fall in a cell, like spam containing the word.
$E$
Expected count
How many would fall in that cell if the word had nothing to do with spam.
$n$
Number of emails
The table's grand total.

A two-way table counts emails by class (spam or not) and by whether they contain the word. The row and column totals give the overall shares.
If word and class were independent, the share of emails with the word would be the same in both columns, so each cell's expected count is $E = (\text{row total})(\text{column total})/n$ .
$\chi^2$ adds the scaled squared gaps between the observed counts $O$ and the expected counts $E$ . A 2 × 2 table has $(2-1)(2-1) = 1$ degree of freedom.
A large $\chi^2$ (a small p-value) says the word carries information about spam. Ranking words by $\chi^2$ is a classic way to pick features for a text classifier.

Learn the math

Contingency Tables and Tree DiagramsStats · Lesson 11, Probability Tests of IndependenceStats · Lesson 36, Chi-Square Tests