Orthogonal Projections

Main idea

The projection of $\vec y$ onto a subspace $W$ is the point of $W$ closest to $\vec y$ , because the leftover piece sticks straight out of $W$ at a right angle. With an orthogonal basis of $W$ , you get it by adding up the shadows of $\vec y$ on each basis vector.

\operatorname{proj}_W\vec y = \frac{\vec y\cdot{\htmlData{sym=0}{\vec u_1}}}{{\htmlData{sym=0}{\vec u_1}}\cdot{\htmlData{sym=0}{\vec u_1}}}{\htmlData{sym=0}{\vec u_1}} + \cdots + \frac{\vec y\cdot\vec u_p}{\vec u_p\cdot\vec u_p}\vec u_p

What the symbols mean

$\vec u_1$
Orthogonal basis of W
The formula needs an orthogonal basis; Gram–Schmidt provides one.

Explanation

An orthogonal projection is simply the shadow a vector casts on a subspace (a line, a plane, or something bigger) when the light shines straight down onto it. "Orthogonal" because the light comes in perpendicular to the subspace, and "projection" in the same sense as a projector throwing an image onto a screen. What it's good for: the shadow $\hat y$ is the point of $W$ closest to $\vec y$ , so projecting is how you find the best approximation inside a subspace. All we're really saying here is that the long formula for $\operatorname{proj}_W\vec y$ is a sum of pieces you already know: each term $\frac{\vec y\cdot\vec u_j}{\vec u_j\cdot\vec u_j}\vec u_j$ is the shadow of $\vec y$ on one basis vector (from the last lesson), and you add them up. It looks long, but it's the same small piece repeated once per basis vector.

Last lesson you projected $\vec y$ onto a single line and found that the leftover $\vec z = \vec y - \hat y$ was perpendicular to it. Now the target is a whole subspace, and the same thing happens.

Orthogonal decomposition. Every $\vec y$ splits uniquely as $\hat y + \vec z$ with $\hat y$ in $W$ and $\vec z$ in $W^\perp$ : one part inside $W$ , one part sticking straight out of it.

Best approximation. $\hat y = \operatorname{proj}_W\vec y$ is closer to $\vec y$ than any other point of $W$ . Think about why. Take any other point $\vec w$ in $W$ and write $\vec y - \vec w = (\vec y - \hat y) + (\hat y - \vec w)$ . The first piece is in $W^\perp$ and the second is in $W$ , so they're perpendicular, and Pythagoras from the first lesson gives

\|\vec y - \vec w\|^2 = \|\vec y - \hat y\|^2 + \|\hat y - \vec w\|^2.

The second term is positive whenever $\vec w \ne \hat y$ , so every other point is farther away. The distance from $\vec y$ to $W$ is therefore $\|\vec y - \hat y\|$ .

A common mistake. It's tempting to think you can add up the projections onto any basis of $W$ , because adding shadows worked so nicely in the last lesson. It doesn't: the formula only works when the basis is orthogonal. With a slanted basis, the shadows overlap and count some of $\vec y$ twice. The next lesson shows how to turn any basis into an orthogonal one.

This "closest point" idea is exactly what least squares uses to fit data, later in this unit. There's an animation further down that shows the leftover staying perpendicular to the plane as $\vec y$ moves around.

See it

Here y moves around above the tilted plane W, and ŷ is its projection onto W. Notice that the leftover y − ŷ always sticks straight out of W, perpendicular to every vector in it, while its length changes from about 1.07 to about 2.86 as y moves nearer to or farther from the plane. That right angle is exactly why ŷ is the closest point: any other point of W is farther away, by Pythagoras.

Worked examples

Example 1

Project $\vec y = (1, 2, 3)$ onto $W = \operatorname{Span}\{(2, 5, -1), (-2, 1, 1)\}$ .

1
The basis vectors are orthogonal ( $-4 + 5 - 1 = 0$ ), so we can add the projections onto each one.

Practice

Type your answer and check it. Solve them all to complete the lesson.

Problem 1+5 XP

Project $(1, 2, 3)$ onto the $xy$ -plane.

Problem 2+5 XP

How far is $(1, 2, 3)$ from the $xy$ -plane?

Problem 3+5 XP

Project $(3, 4)$ onto the line spanned by $(1, 1)$ .

Problem 4+5 XP

Project $(1, 2, 3)$ onto $\operatorname{Span}\{(2, 5, -1), (-2, 1, 1)\}$ .

Problem 5+5 XP

Which point of $W$ is closest to $\vec y$ ?

Problem 6+5 XP

A drone hovering at $(2, 3, 6)$ needs the nearest point on a sloped landing ramp through the origin: the plane spanned by the orthogonal vectors $\vec u_1 = (1, 0, 1)$ and $\vec u_2 = (-1, 2, 1)$ . Find that point.

Challenge problem+25 XP

The hardest one here. Get it, and you've got this lesson.

Find the distance from $\vec y = (1, 2, 3, 4)$ to the subspace $W = \operatorname{Span}\{(1, 1, 1, 1), (1, -1, 1, -1)\}$ of $\mathbb R^4$ .

Where it's used

Noise reduction projects a signal onto the subspace of "plausible" signals; GPS and sensor fusion find the closest consistent estimate by projection.

Pulling a Sled: The Part of the Force That CountsLin Alg · Physics

Orthogonal Sets The Gram–Schmidt Process