Computer Systems

IEEE 754 Floating Point

0.1 plus 0.2 is not 0.3, and a float explains why. Toggle the 32 bits of sign, exponent, and mantissa, type a decimal, and watch the nearest value your hardware can store snap into place.

ieee-754floating-pointbinarycomputer-systems

LiveInteractive · drag, toggle, run it

Computer Systems

IEEE 754 Floating Point

A float is one sign bit, a few exponent bits, and a fraction, packed into 32 or 64 bits. The value is (-1)^sign times 1.fraction times 2 raised to (exponent minus bias). Flip any bit below and the decoded number changes, or type a decimal and watch the nearest value the hardware can actually hold snap into place.

The 32 bits, live

sign1 bit

exponent8 bits

mantissa23 bits

Normal0x3e200000

sign

0 (positive)

exponent field

124 - 127 = -3

mantissa fraction

1.01

Exact value of this bit pattern

0.15625

Gap to the next float (one ULP) at this magnitude: 0.00000001490116119384765625

Type a decimal, get the real float

Your input is rounded to the nearest representable 32-bit float, then decoded back. What you typed and what gets stored often differ, and the exact stored value appears above.

Walk one float at a time:(this is +/-1 on the integer bit pattern: nextafter)

Why 0.1 + 0.2 is not 0.3

None of 0.1, 0.2, or 0.3 land exactly on a 32-bit float, so each is rounded the moment it is stored. Adding the rounded 0.1 and 0.2 lands on a different float than the rounded 0.3. Here are the real stored values, decoded exactly.

At 32-bit the two rounding paths happen to land on the same float, so here 0.1 + 0.2 === 0.3 is false. Switch to 64-bit above to see the famous 0.30000000000000004 that JavaScript and most languages actually report.

Floats thin out as they grow

Between consecutive powers of two there are always the same number of floats, so the gap between neighbours (one ULP) doubles every time the exponent grows by one. Near zero floats are dense; out past large magnitudes whole integers start falling between them.

2^-126

ulp 0.000000000000000000000000000000000000000000001401298464324817…

2^-101

ulp 0.000000000000000000000000000000000000047019774032891500318749…

2^-76

ulp 0.000000000000000000000000000001577721810442023610823457130565…

2^-51

ulp 0.000000000000000000000052939559203393771191770156292477622628…

2^-26

ulp 0.0000000000000017763568394002504646778106689453125

2^-1

ulp 0.000000059604644775390625

2^24

ulp 2

2^49

ulp 67108864

2^74

ulp 2251799813685248

2^99

ulp 75557863725914323419136

2^124

ulp 2535301200456458802993406410752

The bar marked in terracotta is the magnitude band of the value you are editing above.

What the fields mean

Bias on the exponent. The 8-bit exponent field stores an unsigned number, so to reach negative powers of two it carries a fixed offset. Subtract the bias (127) from the stored field to get the real exponent. A stored field of 127 means 2 to the power 0.

The implicit leading 1. Normal numbers are written 1.fraction in binary, and that leading 1 is always there, so it is not stored. You get an extra bit of precision for free. Subnormals are the exception: when the exponent field is all zeros the leading bit is 0 instead, which lets values shrink gradually down to zero.

The four special cases. Exponent field all zeros with a zero fraction is signed zero; all zeros with a nonzero fraction is a subnormal. Exponent field all ones with a zero fraction is infinity; all ones with a nonzero fraction is NaN. Flip the bits above into those shapes and the label updates.

Why decimal fractions break. Binary fractions are sums of 1/2, 1/4, 1/8, and so on. A value like 0.1 is 1/10, and 10 has the factor 5, which no power of two divides, so 0.1 has no finite binary expansion. The hardware stores the nearest value it can, and that tiny rounding gap is what you see below.

Every bit pattern here is produced by Float32Array / Float64Array round-trips, so it is exactly what your CPU stores.