Linear Independence

Main idea

Vectors are linearly independent when none of them can be built from the others, so each one adds a genuinely new direction. Put another way, the only way to combine them into $\vec 0$ is with all weights zero.

{\htmlData{sym=0}{c_1}}\vec v_1 + c_2\vec v_2 + \cdots + c_p\vec v_p = \vec 0 \implies {\htmlData{sym=0}{c_1}} = c_2 = \cdots = c_p = 0

What the symbols mean

$c_1$
Weights
Independence means only the all-zero choice works.

Explanation

A set of vectors is linearly independent when none of them is redundant: you can't build any one of them out of the others. The name fits: each vector is independent of the rest, in the sense that no linear combination of the others produces it. It's good for knowing whether you have any wasted vectors, like a thruster that only pushes in a direction the others already cover. The formula above looks formal, but it's only two pieces: $c_1\vec v_1 + \cdots + c_p\vec v_p = \vec 0$ (some combination of the vectors that lands back at the origin) and $c_1 = \cdots = c_p = 0$ (the claim that the only way to do that is to not move at all). It's a careful way of saying "no redundancy"; bear with it.

Why the zero test? Suppose one vector is a combination of the others, say $\vec v_3 = 2\vec v_1 - \vec v_2$ . Move everything to one side and you get $2\vec v_1 - \vec v_2 - \vec v_3 = \vec 0$ , a way to reach $\vec 0$ with weights that aren't all zero. It works the other way too: if some combination with a nonzero weight gives $\vec 0$ , you can solve for that vector in terms of the rest. So "some vector is a combination of the others" and "a combination with nonzero weights gives $\vec 0$ " say the same thing. The zero version is nicer because it treats all the vectors equally, without you having to guess which one is the redundant one.

Test: put the vectors in the columns of a matrix and row reduce. They're independent exactly when every column has a pivot (no free variables). That's the homogeneous system from the last lesson: $c_1\vec v_1 + \cdots + c_p\vec v_p = \vec 0$ is $A\vec c = \vec 0$ , and it has only the trivial solution exactly when there's no free variable.

Quick facts:

Two vectors are dependent exactly when one is a multiple of the other.
Any set containing $\vec 0$ is dependent. (Put weight $1$ on $\vec 0$ and $0$ on everything else.)
More vectors than entries (say 4 vectors in $\mathbb R^3$ ) are always dependent: there can't be a pivot in every column. (There are only 3 rows, so at most 3 pivots.)

A common mistake. It's tempting to think a set is independent as long as no two of the vectors are parallel, because that's the test for a pair. It isn't: with three or more vectors, one can be a combination of two others without being a multiple of either. The worked example below is exactly that: no two of $\binom{1}{2}$ , $\binom{3}{4}$ , $\binom{5}{6}$ are multiples of each other, and yet they're dependent.

Independent vectors are "efficient": each one adds a new direction to the span, so taking any one of them away makes the span smaller. There's an animation further down that shows what happens to two vectors as they line up.

See it

Here v₁ = (2, 1) stays fixed while v₂, of length 2, turns all the way around, and the dashed parallelogram is the patch the two of them span. Its area is biggest, about 4.47 (that's 2√5), when v₂ is perpendicular to v₁. As v₂ swings into line with v₁, at about 27° or 207°, the parallelogram flattens to zero area: the vectors become dependent, and together they only reach a line.

Worked examples

Example 1

Are $\binom{1}{2}$ , $\binom{3}{4}$ , $\binom{5}{6}$ independent?

1
These are three vectors in $\mathbb R^2$ , so there are more vectors than entries. A matrix with 2 rows has room for at most 2 pivots, so one of the 3 columns has to go without.

No. In fact $\binom56 = 2\binom34 - \binom12$ .

Practice

Type your answer and check it. Solve them all to complete the lesson.

Problem 1+5 XP

Are $\binom12$ and $\binom24$ independent?

Problem 2+5 XP

Are $\binom10$ and $\binom11$ independent?

Problem 3+5 XP

Can 4 vectors in $\mathbb R^3$ be independent?

Problem 4+5 XP

For which $h$ are $\binom{1}{3}$ and $\binom{2}{h}$ dependent?

Problem 5+5 XP

Find $c_2, c_3$ with $\binom56 = c_2\binom34 + c_3\binom12$ .

Problem 6+5 XP

A drone has three thrusters pushing along $(1, 0)$ , $(0, 1)$ and $(1, 1)$ . Show they're dependent by finding weights, with the first weight $1$ , that make their total thrust zero.

Challenge problem+25 XP

The hardest one here. Get it, and you've got this lesson.

For which value of $h$ are $(1, 2, 3)$ , $(2, 5, 7)$ and $(1, 4, h)$ linearly dependent?

Where it's used

In data science, dependent features (one column a combination of others) make regression unstable; independence is what "no redundant information" means mathematically.

Steering a Hovercraft with Two ThrustersLin Alg · Robotics

Solution Sets of Linear Systems Linear Transformations