Lagrange Multipliers

Lagrange multipliers are simply a method for finding the best value of something when you're not free to choose anything you like, only things that obey a rule, like the most output you can get on a fixed budget. The method is named after Joseph-Louis Lagrange, who came up with it in the 1700s, and the "multiplier" is the extra number $\lambda$ it introduces. The equation $\nabla f = \lambda\nabla g$ looks abstract, but it's three pieces: $\nabla f$ (the direction that improves what you want most), $\nabla g$ (the direction that breaks your rule fastest) and $\lambda$ (a scaling number). It says those two arrows line up. That's the whole condition; the rest is solving equations.

The last lesson found extremes on a region by checking the interior and then the edge. Often the edge is the whole problem: you're required to stay on a curve, like spending exactly your budget. Substituting the constraint can get messy, so here's a cleaner idea.

The picture. Walk along the constraint curve $g = k$, watching the value of $f$. Each time you cross a level curve of $f$, you're passing from lower values to higher ones (or the reverse), so you could keep going and do better. So at the best point, the constraint curve can't cross the level curve of $f$. It has to just touch it, tangentially. Now think about what tangent curves have in common: they face the same direction, so their normals are parallel. And we know from the last unit that the normal to a level curve is the gradient. So $\nabla f$ and $\nabla g$ point along the same line, which is what $\nabla f = \lambda\nabla g$ says.

The method: solve the system

then evaluate $f$ at every solution. When the constraint is a closed curve, the largest value is the max and the smallest is the min. A handy trick for solving: divide the first two equations to get rid of $\lambda$.

It extends directly to three variables, and to two constraints, where $\nabla f = \lambda\nabla g + \mu\nabla h$.

Consumers maximizing utility on a budget, engineers minimizing weight at fixed strength, and physicists finding equilibrium under constraints all use Lagrange multipliers; in economics $\lambda$ is the marginal value of an extra dollar.