Diagonalization of Symmetric Matrices

A symmetric matrix is simply one that's a mirror image of itself across its main diagonal: the entry in row $i$, column $j$ equals the entry in row $j$, column $i$, so $A^T = A$. "Orthogonally diagonalizing" it means diagonalizing it the way you already know how to, $A = PDP^{-1}$, but with a $P$ whose columns are perpendicular unit vectors. What it's good for: symmetric matrices show up all over science and engineering, and for them diagonalizing is about as easy as it gets. All we're really saying here is that $A = PDP^T$ breaks the matrix into three moves: $P^T$ (rewrite $\vec x$ in eigenvector coordinates), $D$ (stretch each coordinate by its eigenvalue) and $P$ (go back to the usual coordinates). It's the same diagonalization as before, with $P^{-1}$ replaced by the much easier $P^T$.

Think about where that $P^T$ comes from. Back in the orthogonal sets lesson, you saw that a matrix with orthonormal columns has $P^TP = I$, so its inverse is just its transpose. The spectral theorem says a symmetric matrix always has enough perpendicular eigenvectors to build such a $P$.

What's guaranteed:

• All eigenvalues are real (no rotation hidden inside).
• Eigenvectors for different eigenvalues are automatically orthogonal.
• There's always a full orthonormal basis of eigenvectors, even with repeated eigenvalues.

Here's why the second one is true. Say $A\vec u_1 = \lambda_1\vec u_1$ and $A\vec u_2 = \lambda_2\vec u_2$ with $\lambda_1 \ne \lambda_2$. Then $\lambda_1\,\vec u_1\cdot\vec u_2 = (A\vec u_1)\cdot\vec u_2 = \vec u_1\cdot(A^T\vec u_2) = \vec u_1\cdot(A\vec u_2) = \lambda_2\,\vec u_1\cdot\vec u_2$, using $A^T = A$ in the middle. So $(\lambda_1 - \lambda_2)\,\vec u_1\cdot\vec u_2 = 0$, and since the eigenvalues differ, $\vec u_1\cdot\vec u_2 = 0$.

Spectral decomposition: $A = \lambda_1\vec u_1\vec u_1^T + \cdots + \lambda_n\vec u_n\vec u_n^T$. Each $\vec u_i\vec u_i^T$ is the projection onto the line through $\vec u_i$, so this says $A$ is a sum of projections onto the eigenvector directions, each weighted by its eigenvalue: split $\vec x$ into perpendicular pieces, stretch each piece, add them back.

A common mistake. It's tempting to think every diagonalizable matrix has perpendicular eigenvectors, because it works out so neatly here. It doesn't: $\begin{bmatrix}1 & 1\\ 0 & 2\end{bmatrix}$ has eigenvectors $(1, 0)$ and $(1, 1)$, which aren't perpendicular. Orthogonal eigenvectors are special to symmetric matrices.

Covariance matrices, stiffness matrices and Hessians are all symmetric, which is why this theorem is everywhere. There's an animation further down that shows a symmetric matrix acting as pure stretching along two perpendicular axes.

Vibration analysis of structures, the principal stresses in materials and the normal modes of molecules all come from orthogonally diagonalizing symmetric matrices.