Catching Fake Numbers with Benford's Law

In real financial records, about 30% of amounts start with the digit 1 and fewer than 5% start with 9. That's Benford's law. People inventing numbers spread first digits too evenly, and a chi-square test measures how far the observed digits stray from Benford's pattern.

How it works

The chi-square goodness-of-fit test

In one sentence

To test whether counts $O_1, \dots, O_m$ in $m$ categories fit hypothesized probabilities $p_1, \dots, p_m$ , compute $E_d = np_d$ and $\chi^2 = \sum (O_d - E_d)^2/E_d$ . Under $H_0$ , with every $E_d \ge 5$ , $\chi^2$ approximately follows the chi-square distribution with $m - 1$ degrees of freedom; large values reject the fit.

General formula

{\htmlData{sym=0}{\chi^2}} = \sum_{{\htmlData{sym=4}{d}}=1}^{9}\frac{({\htmlData{sym=1}{O_d}} - {\htmlData{sym=2}{E_d}})^2}{{\htmlData{sym=2}{E_d}}}, \qquad {\htmlData{sym=2}{E_d}} = {\htmlData{sym=3}{n}}\log_{10}\!\left(1 + \frac{1}{\htmlData{sym=4}{{d}}}\right)

Square each gap between what you saw and what you expected, scale it by what you expected, and add. A big total means the model doesn't fit.

What the symbols mean

$\chi^2$
Chi-square statistic
The total scaled squared gap between observed and expected digit counts.
$O_d$
Observed count
How many amounts start with digit d.
$E_d$
Expected count
How many would start with d under Benford's law.
$n$
Number of amounts
How many invoices were audited.
$d$
First digit
1 through 9.

Benford's law predicts first digit $d$ with probability $\log_{10}(1 + 1/d)$ : 30.1% for 1, falling to 4.6% for 9. Many naturally occurring amounts follow it, because they grow multiplicatively across several orders of magnitude.
Multiplying by the number of amounts $n$ gives the expected count $E_d$ for each digit, to compare with the observed count $O_d$ .
Each term $(O_d - E_d)^2/E_d$ measures one digit's surprise, and $\chi^2$ adds them up. With 9 categories it has $9 - 1 = 8$ degrees of freedom.
If the data follow Benford's law, $\chi^2$ above 15.5 happens only 5% of the time, so a larger value flags the records for a closer look.

Learn the math

Goodness of FitStats · Lesson 35, Chi-Square Tests