Orthogonal Sets

Main idea

Nonzero vectors that are all perpendicular to each other can't be redundant, so they're automatically independent. And in an orthogonal basis you don't need row reduction to find coordinates: each weight is just a dot product divided by a length squared.

\vec y = c_1\vec u_1 + \cdots + c_p\vec u_p, \qquad {\htmlData{sym=0}{c_j}} = \frac{\vec y\cdot\vec u_j}{\vec u_j\cdot\vec u_j}

What the symbols mean

$c_j$
Weight
How much of u_j is in y: its projection's coefficient.

Projection onto a line

\hat y = \frac{\vec y\cdot\vec u}{\vec u\cdot\vec u}\,\vec u, \qquad \vec z = \vec y - \hat y \perp \vec u

Explanation

An orthogonal set is simply a group of vectors where every pair is perpendicular, like the $x$ , $y$ and $z$ axes. ("Orthogonal" is the formal word for perpendicular, from the Greek for "right angle.") What it's good for: when your basis vectors are all at right angles, finding the coordinates of a vector stops being a system of equations and becomes one quick calculation per coordinate. All we're really saying here is that in $c_j = \frac{\vec y\cdot\vec u_j}{\vec u_j\cdot\vec u_j}$ , the top, $\vec y\cdot\vec u_j$ , measures how much $\vec y$ lines up with $\vec u_j$ , and the bottom, $\vec u_j\cdot\vec u_j$ , is $\vec u_j$ 's length squared, which corrects for how long $\vec u_j$ happens to be. That's the whole formula: one dot product over another.

Think about how you've found coordinates so far. To write $\vec y$ in a basis, you row reduced $[\vec u_1 \cdots \vec u_p \mid \vec y]$ , because every weight affected every equation. With an orthogonal basis the weights stop interfering with each other.

Why the formula works: dot both sides of $\vec y = \sum c_i\vec u_i$ with $\vec u_j$ . Every term with $i \ne j$ vanishes, because $\vec u_i\cdot\vec u_j = 0$ , and you're left with $\vec y\cdot\vec u_j = c_j\,\vec u_j\cdot\vec u_j$ . Divide and you're done. The same trick shows why an orthogonal set of nonzero vectors is independent: dot a relation $c_1\vec u_1 + \cdots + c_p\vec u_p = \vec 0$ with $\vec u_j$ and you get $c_j\,\vec u_j\cdot\vec u_j = 0$ , so every $c_j$ is $0$ .

Projection onto a line. Each term $c_j\vec u_j$ is the shadow of $\vec y$ on the line through $\vec u_j$ . For a single vector $\vec u$ , that shadow is $\hat y = \frac{\vec y\cdot\vec u}{\vec u\cdot\vec u}\vec u$ , and what's left over, $\vec z = \vec y - \hat y$ , is perpendicular to $\vec u$ . That leftover piece is going to matter a lot in the next few lessons.

Orthonormal sets are orthogonal with unit lengths. Then every $\vec u_j\cdot\vec u_j$ is $1$ , and the formula gets even simpler: $c_j = \vec y\cdot\vec u_j$ .

Orthogonal matrices have orthonormal columns: $U^TU = I$ , so $U^{-1} = U^T$ . That's because entry $(i, j)$ of $U^TU$ is column $i$ dotted with column $j$ , which is $1$ on the diagonal and $0$ everywhere else. They preserve lengths and angles: they're rotations and reflections.

A common mistake. It's tempting to think an orthogonal matrix only needs orthogonal columns, because that's what the name says. It isn't enough: the columns have to be orthonormal, unit length as well. With columns that are merely orthogonal, $U^TU$ comes out diagonal but not $I$ , and $U^{-1} = U^T$ fails.

There's an animation further down that shows a vector split into its two perpendicular pieces, one weight per dot product.

See it

Here y goes around a circle of radius 2.5, written in the orthogonal basis u₁ = (2, 1), u₂ = (−1, 2). Notice that each weight comes from a single dot product, cⱼ = (y·uⱼ)/(uⱼ·uⱼ), with uⱼ·uⱼ = 5 for both, so there's no system to solve. Dropping a perpendicular from y onto each line gives the two pieces, and they always form a rectangle. At y = (2.5, 0), for example, the weights are 1 and −0.5.

Worked examples

Example 1

Write $\vec y = (6, 1, -8)$ in the orthogonal basis $\vec u_1 = (3, 1, 1)$ , $\vec u_2 = (-1, 2, 1)$ , $\vec u_3 = \left(-\frac12, -2, \frac72\right)$ .

1
The basis is orthogonal, so each weight is $c_j = \frac{\vec y\cdot\vec u_j}{\vec u_j\cdot\vec u_j}$ , with no row reduction: $c_1 = \frac{11}{11} = 1$ , $c_2 = \frac{-12}{6} = -2$ , $c_3 = \frac{-33}{33/2} = -2$ .

$\vec y = \vec u_1 - 2\vec u_2 - 2\vec u_3$

Practice

Type your answer and check it. Solve them all to complete the lesson.

Problem 1+5 XP

Give the coordinates of $(6, 1, -8)$ in the orthogonal basis of the example.

Problem 2+5 XP

Project $\vec y = (7, 6)$ onto $\vec u = (4, 2)$ .

Problem 3+5 XP

Find the component of $(7, 6)$ orthogonal to $(4, 2)$ .

Problem 4+5 XP

Is an orthogonal set of nonzero vectors always linearly independent?

Problem 5+5 XP

For an orthogonal matrix $U$ , what is $U^{-1}$ ?

Problem 6+5 XP

An audio codec splits each 3-sample snippet into a "steady" pattern $\vec u_1 = (1, 1, 1)$ , a "ramp" $\vec u_2 = (1, 0, -1)$ and a "wiggle" $\vec u_3 = (1, -2, 1)$ , which are mutually orthogonal. Find the weights $(c_1, c_2, c_3)$ for the snippet $\vec y = (3, 0, 6)$ .

Challenge problem+25 XP

The hardest one here. Get it, and you've got this lesson.

For which $a$ and $b$ is $U = \frac13\begin{bmatrix}1 & 2 & 2\\ 2 & 1 & -2\\ a & b & 1\end{bmatrix}$ an orthogonal matrix?

Where it's used

Fourier analysis is coordinates in an orthogonal basis of sines and cosines, which is why each frequency's strength is just one integral (a dot product).

Fourier SeriesCalc II · Electrical Engineering

Inner Product, Length and Orthogonality Orthogonal Projections