Fractions in Binary: Fixed and Floating Point

Fixed point and floating point are the two ways a computer stores numbers with a fractional part, like 5.75. The names say where the binary point goes: in fixed point it sits in one agreed place for good, while in floating point it floats, moving to just after the number's leading 1, as in scientific notation. Fixed point is fast and predictable on small chips, and floating point handles huge and tiny numbers alike. The float formula $x = (-1)^s \times (1.m)_2 \times 2^{E - 127}$ looks scary, but it's three pieces: $(-1)^s$ (the sign, plus or minus), $(1.m)_2$ (the significant digits) and $2^{E - 127}$ (how far to slide the point). It's just scientific notation in binary; bear with it.

Binary fractions. Place value doesn't stop at the point. In decimal, the digits after it are worth tenths, hundredths and so on, and in binary the bits after the point are worth $\frac12, \frac14, \frac18, \dots$:

To convert a decimal fraction, double it again and again and peel off the whole-number part each time. (Doubling slides every bit one place left, so the bit that crosses the point is the next one.) $0.375 \to 0.75$ (bit 0), $\to 1.5$ (bit 1), then $0.5 \to 1.0$ (bit 1), so $0.375 = 0.011_2$. A fraction ends in binary only when its denominator is a power of 2, just as $\frac13$ never ends in decimal. $\frac1{10}$ has a factor of 5 below, so it repeats forever:

Any finite number of bits has to round it, which is why 0.1 + 0.2 == 0.3 is false in nearly every programming language: the stored sum is 0.30000000000000004.

Fixed point keeps the binary point in one agreed place. In Q format, an integer $n$ with $f$ fraction bits stands for $n/2^f$. Q15 packs $-1$ to $1 - 2^{-15}$ into a 16-bit integer, in steps of $2^{-15} \approx 0.00003$. Adding is plain integer addition. Multiplying two Q15 numbers gives $2^{30}$ times the product, so you shift right by 15 to get back to Q15. Microcontrollers without a floating-point unit run control loops and audio filters this way, because integer math is fast and exactly predictable. The catch is that the step is the same everywhere, so small values get few significant bits and big ones overflow.

int16_t a = 16384;                     // 0.5 in Q15
int16_t b = 8192;                      // 0.25
int16_t p = ((int32_t)a * b) >> 15;    // 4096, which is 0.125

Floating point fixes that catch the same way scientists write $6.02 \times 10^{23}$: let the point move. It's scientific notation in base 2, $x = \pm 1.m \times 2^e$. Normalizing so exactly one 1 sits before the point means that 1 never needs storing. IEEE 754 single precision (float) uses 32 bits: 1 sign bit, 8 exponent bits holding $e + 127$, and 23 fraction bits $m$.

Exponent fields of all 0s and all 1s are reserved for zero, tiny "subnormal" numbers, infinity and NaN.

The gaps double. Between $2^e$ and $2^{e+1}$ there are $2^{23}$ evenly spaced floats, so they're $2^{e-23}$ apart. Just above 1 the gap is $2^{-23} \approx 1.2 \times 10^{-7}$ (machine epsilon). Between $2^{24}$ and $2^{25}$ it's 2, so the integer 16,777,217 has no float32 at all. Rounding to the nearest float makes a relative error of at most $2^{-24}$, the same everywhere: floats trade fixed absolute precision for fixed relative precision.

A common mistake. It's tempting to test floats with ==, since on paper $0.1 + 0.2$ is $0.3$. But each value was rounded on the way in, so the two sides can differ in the last bit. Compare $|a - b|$ against a tolerance instead. For the same reason, don't add tiny amounts to a huge running total: once the gap between floats is bigger than twice the amount, each addition rounds away to nothing.

DSP chips and small microcontrollers run filters and control loops in fixed point; phones, laptops and GPUs use IEEE 754 floats. Knowing where the gaps between floats are is how engineers avoid clocks that drift or freeze after a few weeks of uptime.