Hypotheses and Errors

Main idea

A hypothesis test starts from a boring default claim, the null hypothesis, and asks whether the data are too surprising to square with it. Like a jury, it can go wrong two ways: convicting an innocent claim (Type I) or letting a false one walk (Type II).

{\htmlData{sym=0}{H_0}}: \mu = {\htmlData{sym=2}{\mu_0}} \qquad {\htmlData{sym=1}{H_a}}: \mu \ne {\htmlData{sym=2}{\mu_0}} \ \ (\text{or } \mu < {\htmlData{sym=2}{\mu_0}},\ \mu > {\htmlData{sym=2}{\mu_0}})

What the symbols mean

$H_0$
Null hypothesis
The default claim of no effect or no difference; it always contains the equals sign.
$H_a$
Alternative hypothesis
What you're trying to find evidence for.
$\mu_0$
Hypothesized value
The value the null hypothesis claims.

{\htmlData{sym=0}{\alpha}} = P(\text{Type I error}), \qquad {\htmlData{sym=1}{\beta}} = P(\text{Type II error})

What the symbols mean

$\alpha$
Significance level
The risk of rejecting a true null you're willing to accept, often 0.05.
$\beta$
Type II error rate
The chance of missing a real effect; 1 − β is the test's power.

Explanation

A hypothesis test is simply a way to decide whether data give real evidence against a claim, or whether what you saw could easily be luck. The claim being tested is called the null hypothesis, "null" because it's usually the no-change, no-effect, nothing-to-see-here position. The notation is just two competing statements: $H_0$ (the default, with an equals sign) and $H_a$ (what you suspect instead). And $\alpha$ and $\beta$ are just names for the two ways you can be wrong: $\alpha$ (crying wolf when nothing's there) and $\beta$ (missing something that is). That's the whole framework, and every test in this unit plugs into it.

Think of a courtroom. The defendant is presumed innocent, and that presumption is $H_0$ . The prosecution has to bring evidence strong enough to overturn it "beyond reasonable doubt." If the evidence isn't strong enough, the verdict is "not guilty," which isn't the same as proving innocence. A hypothesis test works exactly the same way, with data as the evidence.

Writing hypotheses. Hypotheses are about population parameters ( $\mu$ , $p$ ), never about sample statistics. That makes sense: you already know your sample mean exactly, so there's nothing to test about it. The question is what it says about the population. The alternative decides the test's direction:

"has changed" or "is different": two-tailed, $\ne$ .
"has decreased": left-tailed, $<$ .
"has increased": right-tailed, $>$ .

The two errors:

	$H_0$ true	$H_0$ false
Reject $H_0$	Type I error ( $\alpha$ )	Correct
Fail to reject $H_0$	Correct	Type II error ( $\beta$ )

A trade-off. Lowering $\alpha$ makes Type I errors rarer but Type II errors more common. Think about the courtroom again: demand more evidence before convicting and you'll convict fewer innocent people, but more guilty ones will go free. Only more data reduces both at once. There's an animation further down that shows this tug-of-war.

A common mistake. When a test fails to reject $H_0$ , it's tempting to say "so $H_0$ is true." It isn't proven. Not finding evidence against $H_0$ is like a "not guilty" verdict: it means the case wasn't strong enough, not that the defendant is innocent. That's why we say "fail to reject," never "accept."

See it

Both kinds of error in one picture: the left curve is where the test statistic lands if H₀ is true, the right one if the effect is real, and we reject beyond the cutoff. The shaded tail of the left curve is α, the false-alarm chance; the shaded part of the right curve is β, the chance of a miss. As the cutoff slides right, α shrinks and β grows. At the usual cutoff of about 1.645, α is 0.05 and β is about 0.20.

Worked examples

Example 1

A company claims its batteries last 500 hours on average. A consumer group suspects they last less. State the hypotheses.

1
The company's claim, with its "=", is the default we'll test, so it's the null. The consumer group's suspicion that the mean is lower is what we're looking for evidence of, so it's the alternative, and it points left.

$H_0: \mu = 500$ , $H_a: \mu < 500$ (left-tailed)

Practice

Type your answer and check it. Solve them all to complete the lesson.

Problem 1+5 XP

A company claims batteries last 500 hours; you suspect less. Which is the alternative hypothesis?

Problem 2+5 XP

A new drug is approved although it doesn't work. Which error is this?

Problem 3+5 XP

A smoke detector stays silent during a real fire ( $H_0$ : no fire). Which error?

Problem 4+5 XP

With $\alpha = 0.05$ , what is the probability of rejecting $H_0$ when it is true?

Problem 5+5 XP

A test fails to reject $H_0$ . The right conclusion is:

Problem 6+5 XP

A spam filter tests the null hypothesis "this email is legitimate." It sends a real email from your boss to the spam folder. Which error is that?

Problem 7+5 XP

Twenty independent tests are run at $\alpha = 0.05$ , all on true null hypotheses. What's the probability that at least one of them rejects by mistake?

Challenge problem+25 XP

The hardest one here. Get it, and you've got this lesson.

Test $H_0: \mu = 100$ against $H_a: \mu > 100$ with $\sigma = 15$ , $n = 36$ and $\alpha = 0.05$ , rejecting when $z > 1.645$ . If the true mean is 105, what is the power, the probability of correctly rejecting $H_0$ ?

Where it's used

Drug trials, spam filters, airport security and quality inspection all balance false alarms (Type I) against misses (Type II).

Five Sigma: How Physicists Claim a DiscoveryStats · Physics

Confidence Intervals for a Proportion p-Values and the z-Test for a Mean

Hypotheses and Errors

Main idea

{\htmlData{sym=0}{H_0}}: \mu = {\htmlData{sym=2}{\mu_0}} \qquad {\htmlData{sym=1}{H_a}}: \mu \ne {\htmlData{sym=2}{\mu_0}} \ \ (\text{or } \mu < {\htmlData{sym=2}{\mu_0}},\ \mu > {\htmlData{sym=2}{\mu_0}})

What the symbols mean

$H_0$
Null hypothesis
The default claim of no effect or no difference; it always contains the equals sign.
$H_a$
Alternative hypothesis
What you're trying to find evidence for.
$\mu_0$
Hypothesized value
The value the null hypothesis claims.

{\htmlData{sym=0}{\alpha}} = P(\text{Type I error}), \qquad {\htmlData{sym=1}{\beta}} = P(\text{Type II error})

What the symbols mean

$\alpha$
Significance level
The risk of rejecting a true null you're willing to accept, often 0.05.
$\beta$
Type II error rate
The chance of missing a real effect; 1 − β is the test's power.

Explanation

"has changed" or "is different": two-tailed, $\ne$ .
"has decreased": left-tailed, $<$ .
"has increased": right-tailed, $>$ .

The two errors:

	$H_0$ true	$H_0$ false
Reject $H_0$	Type I error ( $\alpha$ )	Correct
Fail to reject $H_0$	Correct	Type II error ( $\beta$ )

See it

Worked examples

Example 1

A company claims its batteries last 500 hours on average. A consumer group suspects they last less. State the hypotheses.

1
The company's claim, with its "=", is the default we'll test, so it's the null. The consumer group's suspicion that the mean is lower is what we're looking for evidence of, so it's the alternative, and it points left.

$H_0: \mu = 500$ , $H_a: \mu < 500$ (left-tailed)

Practice

Type your answer and check it. Solve them all to complete the lesson.

Problem 1+5 XP

A company claims batteries last 500 hours; you suspect less. Which is the alternative hypothesis?

Problem 2+5 XP

A new drug is approved although it doesn't work. Which error is this?

Problem 3+5 XP

A smoke detector stays silent during a real fire ( $H_0$ : no fire). Which error?

Problem 4+5 XP

With $\alpha = 0.05$ , what is the probability of rejecting $H_0$ when it is true?

Problem 5+5 XP

A test fails to reject $H_0$ . The right conclusion is:

Problem 6+5 XP

A spam filter tests the null hypothesis "this email is legitimate." It sends a real email from your boss to the spam folder. Which error is that?

Problem 7+5 XP

Twenty independent tests are run at $\alpha = 0.05$ , all on true null hypotheses. What's the probability that at least one of them rejects by mistake?

Challenge problem+25 XP

The hardest one here. Get it, and you've got this lesson.

Where it's used

Drug trials, spam filters, airport security and quality inspection all balance false alarms (Type I) against misses (Type II).

Five Sigma: How Physicists Claim a DiscoveryStats · Physics