Residuals, r² and Outliers

Main idea

A residual is how far a point misses the line. Add up what the line misses and compare it with what you'd miss using no line at all, and you get $r^2$ , the share of the ups and downs in $y$ the line accounts for. Plotting the residuals tells you whether a straight line was the right choice in the first place.

{\htmlData{sym=0}{e}} = y - \hat y, \qquad {\htmlData{sym=1}{r^2}} = 1 - \frac{\sum {\htmlData{sym=0}{e}}^2}{\sum (y - \bar y)^2}

What the symbols mean

$e$
Residual
Observed minus predicted: positive when the point is above the line.
$r^2$
Coefficient of determination
The share of y's variation explained by the linear relationship with x.

Explanation

A residual is simply the leftover: what actually happened minus what the line predicted. The formulas look like more than they are. $e = y - \hat y$ (actual minus predicted, positive when the point is above the line). And $r^2$ is $1$ minus a fraction: on top, $\sum e^2$ (how much the line still misses), on the bottom, $\sum (y - \bar y)^2$ (how much you'd miss by just guessing the average every time). Its formal name is the coefficient of determination, but everyone just says "r squared," because for a straight-line fit it really is the correlation $r$ , squared.

Why that ratio? Without $x$ , your best guess for any $y$ is just $\bar y$ , and $\sum(y - \bar y)^2$ measures how badly that does. With the line, the misses shrink to $\sum e^2$ . So $\frac{\sum e^2}{\sum(y - \bar y)^2}$ is the fraction of the original variation the line didn't explain, and $1$ minus it is the fraction it did.

Residual plots. Plot residuals against $x$ . A patternless band around 0 means the line fits; a curve means a straight line is the wrong model; a fan shape means the spread changes with $x$ . Think of it this way: if the line captured the real pattern, all that's left should be random noise. Any shape in the leftovers is pattern the line missed.

Outliers and influential points. An outlier has a large residual. An influential point changes the line a lot when removed, usually because its $x$ is far from the rest. Investigate them rather than deleting them automatically.

Extrapolation, predicting far outside the range of the data, is unreliable: the pattern may not continue.

A common mistake. It's tempting to think a high $r^2$ means a straight line is the right model. It doesn't. Clearly curved data can still have $r^2$ near 0.8, because a line catches the general trend while missing the shape. The residual plot is what tells you; there's an example of this in the picture further down.

See it

Residuals from a straight-line fit, plotted against x. At first the data really are linear, so the residuals scatter randomly within about ±0.7 of zero and r² is about 0.98. As curvature creeps into the data, the residuals bend into a clear U: the line runs too high in the middle and too low at the ends. Yet r² only falls to about 0.79, which still sounds like a decent fit.

Worked examples

Example 1

For $\hat y = 2.2 + 0.6x$ , find the residual of the point $(3, 5)$ .

1
The line predicts $\hat y(3) = 2.2 + 1.8 = 4$ .

Practice

Type your answer and check it. Solve them all to complete the lesson.

Problem 1+5 XP

For $\hat y = 2.2 + 0.6x$ , find the residual of $(3, 5)$ .

Problem 2+5 XP

For the five-point data set with $r \approx 0.775$ , find $r^2$ .

Problem 3+5 XP

$r^2 = 0.6$ means:

Problem 4+5 XP

A residual plot shows a clear U-shaped pattern. What does that suggest?

Problem 5+5 XP

A model fit on children aged 2 to 10 predicts height at age 30. This is risky because:

Problem 6+5 XP

A model predicts a house will sell for 310 (thousand), and it sells for 295. What is the residual?

Problem 7+5 XP

If $r = -0.8$ , what fraction of the variation in $y$ does the regression line explain?

Challenge problem+25 XP

The hardest one here. Get it, and you've got this lesson.

For the five points $(1, 2)$ , $(2, 4)$ , $(3, 5)$ , $(4, 4)$ , $(5, 5)$ and their least-squares line $\hat y = 2.2 + 0.6x$ , find $\sum e^2$ and $r^2 = 1 - \frac{\sum e^2}{\sum(y - \bar y)^2}$ . Enter both.

Where it's used

Analysts check residual plots before trusting any regression; r² is reported for models in economics, engineering and machine learning.

Residuals Reveal a Stretched SpringStats · Physics

The Least-Squares Regression Line Logistic Regression