Calibration Curves

A calibration curve is simply the translation table between what an instrument shows and what's really there, like millivolts to kilopascals. All we're doing is the regression line from the intro course, used in reverse: fit reading against true value, then, when a new reading comes in, solve for the true value with $\hat x_0 = (y_0 - b_0)/b_1$ (subtract the intercept, divide by the slope). The uncertainty formula looks long, but it's three sources added under a square root: the new reading's own noise, the line's uncertainty at its center, and extra uncertainty the farther you are from the middle of the standards. One line, read backward, with honest error bars.

The scene. A pressure transmitter on a pipeline reports millivolts, not kilopascals. To translate, a technician connects it to a deadweight tester that applies known pressures, 0, 20, …, 100 kPa, and records the output at each. The known values are the standards $x$, the outputs are the readings $y$, and the least-squares line through them is the calibration curve.

Why the line runs from standard to reading. The standards are known almost exactly; the noise is in the reading. Least squares assumes the errors are vertical, in $y$, so the honest model is reading on standard, $\hat y = b_0 + b_1x$, the same line as in the regression line. To use the sensor you then invert it. Regressing $x$ on $y$ instead gives a different line, because it pretends the error sits in the standards.

Where $b_1$ and $b_0$ come from. Choose them to minimize $SSE(b_0, b_1) = \sum (y_i - b_0 - b_1x_i)^2$. Set both partial derivatives to zero:

The first says the line passes through $(\bar x, \bar y)$. Put $b_0 = \bar y - b_1\bar x$ into the second: $\sum x_i(y_i - \bar y) = b_1\sum x_i(x_i - \bar x)$, and since subtracting $\bar x$ from each $x_i$ changes neither sum, that's $S_{xy} = b_1S_{xx}$. These are the normal equations of least squares.

Why divide by $n - 2$. The same two equations force the residuals to satisfy $\sum e_i = 0$ and $\sum x_ie_i = 0$, so only $n - 2$ of them are free. That makes $SSE/(n - 2)$ the unbiased estimate of the noise variance $\sigma^2$, just as the sample variance divides by $n - 1$ after estimating one mean.

Using the sensor. A reading $y_0$ maps back to $\hat x_0 = (y_0 - b_0)/b_1$. Translate the residual SD the same way: $s/b_1$ is the calibration's precision in input units. Scatter of 0.2 mV on a sensor with $b_1 = 0.25$ mV/kPa means about 0.8 kPa.

The slope's own uncertainty. Because $\sum (x_i - \bar x)\bar y = 0$, the slope is a weighted sum of independent readings, $b_1 = \sum (x_i - \bar x)y_i/S_{xx}$, so $\operatorname{Var}(b_1) = \sum (x_i - \bar x)^2\sigma^2/S_{xx}^2 = \sigma^2/S_{xx}$ and $SE(b_1) = s/\sqrt{S_{xx}}$. Standards spread across the whole range make $S_{xx}$ big and the slope sharp.

How sure is $\hat x_0$? Three things wobble: the new reading (variance $\sigma^2/m$ for an average of $m$ readings), the height of the line at $\bar y$ (variance $\sigma^2/n$) and its slope (variance $\sigma^2/S_{xx}$). Write $\hat x_0 = \bar x + (y_0 - \bar y)/b_1$ and apply propagation of error: the partial derivatives with respect to $y_0$, $\bar y$ and $b_1$ are $1/b_1$, $-1/b_1$ and $-(y_0 - \bar y)/b_1^2$, the three pieces are independent, and squaring and adding gives the formula above. Its last term grows as $y_0$ moves away from $\bar y$: a calibration is sharpest in the middle of its standards and should never be extrapolated past them.

A common mistake. Since you'll use the curve to go from reading to pressure, it's tempting to fit pressure on reading directly. Don't: the standards' true values are the precise ones, so fit reading on true value and invert. Other slips to watch: quoting $s$ in millivolts when the user needs kilopascals (divide by $b_1$), trusting the curve beyond the last standard, and expecting averaging to fix everything. More readings shrink only the $1/m$ term, never the calibration's own uncertainty.

Every transmitter in a plant (pressure, flow, level, temperature) is calibrated this way against standards traceable to a national laboratory, and the calibration certificate reports the fitted line and its uncertainty. Analytical chemists do the same with absorbance against concentration, where Beer's law makes the curve a line, and their rule of reading an unknown several times comes straight from the $1/m$ term. When the uncertainty near the ends of the range is too large, metrologists add standards there rather than readings.