p-Values and the z-Test for a Mean

A p-value is simply a surprise score: the chance of getting data at least as extreme as yours if the null hypothesis were true. The "p" just stands for probability. Small means surprising, and surprising means the null looks doubtful. The test statistic $z = \frac{\bar x - \mu_0}{\sigma/\sqrt n}$ is the same z-score you've been using: $\bar x - \mu_0$ (how far your sample mean landed from the claimed value) divided by $\sigma/\sqrt n$ (the standard error, how far sample means typically wander). So $z$ just says "how many standard errors off are we?"

Where the z comes from. From the central limit theorem, if $H_0$ is true then $\bar x$ is roughly normal around $\mu_0$ with standard error $\frac{\sigma}{\sqrt n}$. Standardizing turns that into the standard normal, so the p-value is just a tail area you can look up, like any normal probability.

The procedure:

1. State $H_0$ and $H_a$, and choose $\alpha$.
2. Compute the test statistic.
3. Find the p-value: the tail area beyond the statistic (both tails for a two-tailed test).
4. Reject $H_0$ if $p < \alpha$; otherwise fail to reject.
5. State the conclusion in context.

Two-tailed p-values double the tail: $p = 2P(Z > |z|)$. A two-tailed test is looking for a difference in either direction, so a result just as far off on the other side would have counted as evidence too.

Statistical vs practical significance. With huge samples, tiny unimportant differences become "significant." Always ask how big the effect is, not just whether it exists.

A common mistake. It's tempting to read $p = 0.03$ as "a 3% chance that $H_0$ is true." It isn't that. The p-value is computed assuming $H_0$ is true, so it can't also be the probability of $H_0$. It's the chance of data this extreme if $H_0$ holds.

Scientific papers report p-values for nearly every claim; tech companies use them to decide whether a website change really moved a metric.