Finite Difference Methods

A finite difference method is simply a way to make a computer solve a PDE: chop the rod into evenly spaced points, and at each tick of the clock, let every point's temperature drift a little toward its neighbors. It's called "finite" because the gaps $\Delta x$ and $\Delta t$ stay a fixed, nonzero size instead of shrinking to zero the way they do inside a derivative. It's good for exactly the problems separation of variables can't handle: odd shapes, materials that vary from place to place, real engineering. The update rule $u_i^{n+1} = u_i^n + r\left(u_{i+1}^n - 2u_i^n + u_{i-1}^n\right)$ looks busy, but it's just three parts: $u_i^n$ (the temperature at point $i$ right now), $u_{i+1}^n - 2u_i^n + u_{i-1}^n$ (how far the point is from the average of its two neighbors) and $r$ (how big a step you take toward that average). The only catch is that $r$ can't be too big, or you overshoot. That's the whole method; bear with the subscripts.

The scheme. Where does that update come from? A derivative is a limit of differences, so just don't take the limit. Replace $u_t$ by $\frac{u_i^{n+1} - u_i^n}{\Delta t}$ and $u_{xx}$ by the second difference, the formula above, which is the difference of two first differences. Solve for $u_i^{n+1}$ and you get the explicit scheme. Rearranged, it's $(1 - 2r)u_i^n + r\,u_{i+1}^n + r\,u_{i-1}^n$: each new value is a weighted average of the point and its neighbors, which is exactly what heat does.

Stability. Think about the worst case: a wiggle that alternates sign from point to point. Each point's neighbors are both on the other side, so the update pulls it hard toward zero. In fact it's multiplied by $1 - 4r$ each step. If $r > \frac12$ that factor is below $-1$, so the pull overshoots: the wiggle grows and flips every step until the numbers explode. With $r \le \frac12$, every weight $1 - 2r$ and $r$ is nonnegative, so every new value is a true average, and errors stay bounded.

The cost. $\Delta t \le \frac{\Delta x^2}{2k}$: halving the grid spacing forces four times as many time steps. Implicit schemes like Crank–Nicolson are stable for any $\Delta t$, at the price of solving a linear system each step.

A common mistake. It's tempting to think a finer grid is always safer, since smaller $\Delta x$ means a more accurate second difference. It isn't, not on its own. Shrinking $\Delta x$ while keeping the same $\Delta t$ makes $r = \frac{k\Delta t}{\Delta x^2}$ bigger, and can push a perfectly good simulation past $\frac12$ into blow-up. Refine the grid and you have to shrink the time step with it.

Other equations. For waves the condition is $\frac{c\,\Delta t}{\Delta x} \le 1$ (the CFL condition: information can't skip a grid cell per step). For Laplace's equation, repeatedly replacing each value by the average of its four neighbors converges to the solution.

Weather prediction, crash simulations, chip design and visual effects all run grid-based PDE solvers; stability limits like these decide how long the simulations take.