The Matrix Equation Ax = b

The matrix equation $A\vec x = \vec b$ is simply a whole linear system packed into one short line: $A$ holds all the coefficients, $\vec x$ holds all the unknowns, and $\vec b$ holds all the right-hand sides. Multiplying a matrix by a vector is shorthand for the column combinations from the last lesson: take $x_1$ of the first column, $x_2$ of the second, and so on, and add them up. It's good because it lets you talk about a system with a thousand unknowns as easily as one with two. The formula above looks like a wall of brackets, but it's only three pieces: $A$ (the columns $\vec a_1, \dots, \vec a_n$ lined up side by side), the column of $x$'s (the list of weights, one per column) and $x_1\vec a_1 + \cdots + x_n\vec a_n$ (what you get when you use those weights). Same idea as before, written more compactly; bear with the notation.

Three ways to say the same thing. Take a look at the system from the first lesson: $x_1 - 2x_2 = -1$, $-x_1 + 3x_2 = 3$. You can write it as a vector equation, $x_1\binom{1}{-1} + x_2\binom{-2}{3} = \binom{-1}{3}$, or, packing the columns into a matrix, as

All three have the same solution, $(3, 2)$, and you solve all three the same way: row reduce the augmented matrix.

Computing $A\vec x$: either combine columns (the meaning), or use the row-vector rule (the shortcut): entry $i$ is row $i$ of $A$ dotted with $\vec x$. The two always agree. The row rule just collects, for each entry, the pieces every column contributes to that entry.

When does $A\vec x = \vec b$ have a solution for every $\vec b$? Exactly when the columns of $A$ span $\mathbb R^m$, which happens exactly when $A$ has a pivot in every row. Think about why the pivots matter. If some row of the echelon form has no pivot, it's all zeros on the left, and you can pick a $\vec b$ that leaves a nonzero number on the right of that row: $0 =$ nonzero, no solution. With a pivot in every row, that bad row can never show up.

A common mistake. It's tempting to think a pivot in every column is what guarantees a solution for every $\vec b$, because pivot columns are where the basic variables live. It isn't: a pivot in every column tells you that a solution, if there is one, is unique. It's a pivot in every row that rules out a $0 =$ nonzero row, and that's what makes every $\vec b$ reachable.

Matrix multiplication is linear. $A(\vec u + \vec v) = A\vec u + A\vec v$ and $A(c\vec u) = cA\vec u$. That falls right out of the column picture: adding or scaling the lists of weights adds or scales the combinations. We'll lean on this a lot when we get to linear transformations later in this unit.

There's an animation further down that shows $A\vec x$ being built from the two columns as $\vec x$ goes around a circle.

Every layer of a neural network computes $A\vec x$ (then a nonlinearity): billions of matrix–vector products power modern AI.