Residuals Reveal a Stretched Spring

Hang weights on a spring and fit a straight line: r² can look excellent even after the spring has started to stretch permanently. The residual plot, each point's miss from the line, shows the curve that r² hides.

How it works

Residuals, r² and influential points

In one sentence

For a least-squares line $\hat y = mx + b$ , the residuals $e_k = y_k - \hat y_k$ sum to zero, and $r^2 = 1 - \sum e_k^2 / \sum (y_k - \bar y)^2$ is the proportion of the variation in $y$ explained by the line (the square of the correlation). A pattern in the residual plot indicates a nonlinear relationship; an influential point is one whose removal changes the line markedly.

General formula

{\htmlData{sym=0}{e_k}} = {\htmlData{sym=1}{y_k}} - {\htmlData{sym=2}{\hat y_k}}, \qquad {\htmlData{sym=3}{r^2}} = 1 - \frac{\sum {\htmlData{sym=0}{e_k}}^2}{\sum ({\htmlData{sym=1}{y_k}} - {\htmlData{sym=4}{\bar y}})^2}

r² says how much of the scatter the line accounts for; the residual plot shows what's left over. Leftovers with a shape mean the model missed something.

What the symbols mean

$e_k$
Residual
How far reading k sits above (+) or below (−) the fitted line.
$y_k$
Measured extension
The spring's stretch for load k, in centimeters.
$\hat y_k$
Predicted extension
The line's prediction for load k.
$r^2$
Coefficient of determination
The fraction of the variation in the readings that the line explains.
$\bar y$
Mean extension
The average of all the readings.

Fit the least-squares line $\hat y = mx + b$ to the readings. Each residual $e_k = y_k - \hat y_k$ is how far a reading sits above or below the line.
$r^2$ is the fraction of the variation in $y$ that the line explains: 1 minus the residual variation over the total variation.
A good linear fit leaves residuals scattered randomly around zero. A curved pattern means the relationship isn't linear, even when r² is close to 1.
A single far-off reading can drag the line and change r² a lot, so check such influential points before trusting a fit.

Learn the math

The Least-Squares Regression LineStats · Lesson 38, Regression and Correlation Residuals, r² and OutliersStats · Lesson 39, Regression and Correlation Calibration CurvesEng Stats · Lesson 20, Regression for Engineering Residuals and DiagnosticsEng Stats · Lesson 21, Regression for Engineering