Understanding the IEEE 754 floating-point standard
The IEEE 754 standard (originally ratified in 1985 and updated in 2008 and 2019) governs how modern microprocessors, GPUs, and programming languages store real numbers in binary hardware registers. Because computers have finite memory, continuous real numbers must be mapped into fixed-width bit patterns.
Every finite IEEE 754 floating-point number is represented in binary scientific notation using three distinct components:
- Sign bit (S): 1 bit indicating positive (0) or negative (1).
- Biased Exponent (e): Stored using excess-bias encoding (bias = 127 for 32-bit single precision; bias = 1023 for 64-bit double precision).
- Mantissa or Fraction (f): Encodes the fractional bits after the binary point.
In normalized format, the value is evaluated as:
Value = (-1)^S × (1 + f) × 2^(e - bias)
Why floating-point hardware uses biased exponents
In integer arithmetic, computers commonly use two's complement to represent signed numbers. However, IEEE 754 uses biased (excess) representation for exponents. Adding a fixed bias (127 for single precision, 1023 for double precision) shifts negative exponents into non-negative integers. For example, an unbiased exponent of -4 becomes a stored biased exponent of -4 + 127 = 123 in single precision.
This design enables high-performance hardware execution:
- Comparing two positive floating-point numbers can be performed using standard unsigned integer comparator circuits without first extracting or decoding the sign of the exponent.
- Lexicographical ordering is preserved across finite normalized numbers, dramatically accelerating sorting algorithms and hardware ALU pipeline stages.
The implicit leading 1: gaining an extra bit of precision
In base-10 scientific notation, numbers are normalized so that exactly one non-zero digit precedes the decimal point (for example, 3.14 × 10^0). In base 2, the only non-zero binary digit is 1. Therefore, every normalized binary number begins with 1 (for example, 1.1011 × 2^3).
Because the leading bit is guaranteed to be 1 for all normalized numbers, IEEE 754 architects chose not to store this bit in memory. It is treated as an implicit "hidden bit." This clever architectural choice grants 24 bits of significand precision while storing only 23 mantissa bits in single precision, and 53 bits of precision from 52 stored bits in double precision.
Worked Example 1: Converting -13.625 to 32-bit single precision
Consider the classic computer architecture homework problem: convert -13.625 into IEEE 754 single precision.
- Sign bit: The number is negative, so sign bit S = 1.
- Integer portion to binary: Convert 13 to binary by repeated division by 2:
- 13 ÷ 2 = 6 remainder 1
- 6 ÷ 2 = 3 remainder 0
- 3 ÷ 2 = 1 remainder 1
- 1 ÷ 2 = 0 remainder 1
Reading remainders from bottom to top yields 13_10 = 1101_2.
- Fractional portion to binary: Convert 0.625 to binary by repeated multiplication by 2:
- 0.625 × 2 = 1.250 -> bit 1, remainder 0.250
- 0.250 × 2 = 0.500 -> bit 0, remainder 0.500
- 0.500 × 2 = 1.000 -> bit 1, remainder 0.000 (terminates)
Reading top-down yields 0.625_10 = 0.101_2.
- Combine and normalize:
13.625_10 = 1101.101_2.
Shift the binary point 3 places to the left: 1.101101_2 × 2^3.
The unbiased exponent E is 3. - Biased exponent:
e = E + bias = 3 + 127 = 130.
Convert 130 to 8-bit unsigned binary: 130 = 128 + 2 = 10000010_2. - Mantissa field:
Drop the implicit leading 1 to get the fraction bits: 101101_2.
Pad with trailing zeros to fill the 23-bit mantissa:
10110100000000000000000_2. - Assemble the 32-bit word:
- Sign (1 bit): 1
- Exponent (8 bits): 10000010
- Mantissa (23 bits): 10110100000000000000000
Concatenated: 1100 0001 0101 1010 0000 0000 0000 0000_2.
Group into 4-bit nibbles: 0xC15A0000.
Because 0.625 is an exact sum of negative powers of two (2^-1 + 2^-3 = 0.5 + 0.125), the exact stored value is -13.625 with zero rounding error.
Worked Example 2: Why 0.1 produces floating-point rounding error
A common pitfall in software engineering is the assumption that 0.1 can be stored exactly. Just as 1/3 cannot be represented as a finite terminating decimal in base 10 (0.3333...), the fraction 0.1 cannot be represented as a finite terminating binary fraction in base 2:
- 0.1 × 2 = 0.2 -> 0
- 0.2 × 2 = 0.4 -> 0
- 0.4 × 2 = 0.8 -> 0
- 0.8 × 2 = 1.6 -> 1
- 0.6 × 2 = 1.2 -> 1
- 0.2 × 2 = 0.4 -> 0 (repeats pattern 0011 indefinitely)
0.1_10 = 0.0001100110011..._2 = 1.1001100110011001100110011..._2 × 2^-4.
The unbiased exponent is -4, so the biased exponent is -4 + 127 = 123 = 01111011_2.
When truncating to 23 mantissa bits, the dropped bits begin with 1100...
Under the default IEEE 754 "Round to Nearest, Ties to Even" mode, the value rounds upward by adding 1 to the least significant bit:
Stored mantissa: 10011001100110011001101_2.
Assembled 32-bit float: 0x3DCCCCCD.
Converting these exact stored bits back into base 10 yields 0.100000001490116119384765625.
The absolute rounding error is approximately +1.490116 × 10^-9.
Special values: zeros, subnormals, infinities, and NaN
IEEE 754 reserves specific bit patterns for boundary cases:
- Signed zeros (+0 and -0): Biased exponent e = 0 and mantissa = 0. Preserving sign allows directional limits in calculus (such as 1 / -0 = -Infinity).
- Subnormal (denormal) numbers: Biased exponent e = 0 and mantissa ≠ 0. The implicit leading bit becomes 0, and the effective exponent is fixed at 1 - bias (-126 for single precision). This provides gradual underflow, preventing catastrophic underflow when subtracting nearly equal small numbers.
- Infinities (+Infinity and -Infinity): Biased exponent e = all 1s (255 or 2047) and mantissa = 0. Produced by overflow or division by zero.
- NaN (Not a Number): Biased exponent e = all 1s and mantissa ≠ 0. Indicates mathematically undefined operations like 0 / 0 or sqrt(-1). Quiet NaNs (qNaN) propagate silently through arithmetic pipelines, while signaling NaNs (sNaN) trigger CPU traps.