Learning Objectives
By the end of this lesson you will be able to:
- Demonstrate floating-point precision loss with concrete examples
- Identify operations that accumulate precision loss rapidly
- Select inputs that expose precision loss in financial systems
Why Precision Loss Compounds
A single floating-point operation introduces a rounding error around 10^-16 to 10^-17. For most calculations, this is so small it's invisible.
But when you perform thousands or millions of these operations — as any realistic financial system does — the errors accumulate:
0.1 + 0.1 + 0.1 + ... (10 times)
= 0.9999999999999999 (not 1.0!)
Multiply this across 10,000 micro-transactions, and the discrepancy becomes visible.
The Operations Most Prone to Precision Loss
- Addition/subtraction of numbers of very different magnitudes:
1000000.0 + 0.1— the 0.1 may be lost in rounding because the exponent must align - Repeated multiplication by fractions: Compound interest applied daily compounds the error daily
- Subtraction of nearly equal numbers (catastrophic cancellation):
1000000.001 - 1000000.000— most significant digits cancel, leaving only the error in the low digits - Division followed by multiplication:
(a / b) * bshould equala, but floating-point rounding means it often doesn't
Test Cases for Precision Loss
| Test | Input | Expected | Likely Float Result |
|---|---|---|---|
| Repeated tenth-additions | 0.1 added 10 times | 1.0 | 0.9999999999999999 |
| Compound 1% interest daily | $100 × 1.01^365 | $3,778.34 | $3,778.34 (approximately) |
| Micro-fraction accumulation | $0.001 applied 1000 times | $1.00 | $1.0000000000000002 or similar |
What to Test in Financial Applications
For any financial calculation, test:
- Round amounts: $1.00, $10.00, $100.00 (should produce exactly round results)
- Fractional amounts: $0.10, $0.01, $0.001 (may produce trailing noise)
- Accumulated operations: apply the same calculation 100 times and check the result against manual calculation
If the system stores money as a floating-point double and displays raw values, you'll almost certainly find precision issues.
The Fix: Use BigDecimal or Integer Arithmetic
A correctly designed financial system:
- Stores amounts as integers (e.g., store $15.99 as 1599 cents)
- Uses
BigDecimalwith explicit rounding modes (Java, Python'sDecimal) - Only converts to float for display purposes
If you find precision bugs, the fix recommendation in your bug report should specifically mention the data type as the root cause.
Pro Tip: When testing financial calculations, always verify against a manually calculated expected value using a spreadsheet or Python's
Decimallibrary. Don't trust your calculator for this — it also uses floating-point internally.
Key Takeaways
- Single float operations are usually fine; accumulated operations compound the error
- Addition/subtraction of different-magnitude numbers and repeated fractions are most risky
- Test round amounts, fractional amounts, and accumulated operations
- The fix is always "use a precise decimal type" — not "round the display"