How a Camera Sees

A camera flattens a 3D world into a picture. One matrix multiplication followed by a division by depth tells you where any point lands on the image.

How it works

Projection matrices and homogeneous coordinates

In one sentence

A pinhole camera maps a world point to a pixel by multiplying by the $3\times 4$ matrix $K[R\mid\mathbf t]$ and dividing by depth: $(x, y) = (u/w,\ v/w)$ .

General formula

\begin{bmatrix}u\\v\\{\htmlData{sym=4}{w}}\end{bmatrix} = {\htmlData{sym=0}{K}}\,[{\htmlData{sym=1}{R}}\mid{\htmlData{sym=2}{\mathbf{t}}}]\begin{bmatrix}{\htmlData{sym=3}{X}}\\Y\\Z\\1\end{bmatrix}, \qquad {\htmlData{sym=5}{(x, y)}} = \left(\tfrac{u}{\htmlData{sym=4}{{w}}}, \tfrac{v}{\htmlData{sym=4}{{w}}}\right)

$[R\mid\mathbf t]$ moves the world into the camera's frame, $K$ applies the lens (focal length and image center), and dividing by $w$ shrinks distant things. That division is perspective.

What the symbols mean

$K$
Camera intrinsics
The camera's own settings: focal length (zoom) and where the image center sits.
$R$
Camera rotation
Which way the camera is turned.
$\mathbf{t}$
Camera translation
Where the camera is placed, as a shift of the world.
$X$
World point
A point's position in the 3D scene (X, Y, Z). The extra 1 lets one matrix both rotate and shift.
$w$
Depth
How far in front of the camera the point is. Dividing by w makes far things smaller.
$(x, y)$
Pixel position
Where the point lands in the photo.

$[R\mid\mathbf{t}]$ moves the world into the camera's point of view.
$K$ holds the focal length $f$ and the image center. A bigger $f$ zooms in.
Dividing by the depth $w$ is what makes far objects look small.

Learn the math

Equations of LinesCalc III · Lesson 6, Three-Dimensional Space The Inverse of a MatrixLin Alg · Lesson 10, Matrix Algebra Matrices in Computer GraphicsLin Alg · Lesson 15, Matrix Algebra