X

"Rounding errors" and precision in the Mac OS X 10.3 calculator

"Rounding errors" and precision in the Mac OS X 10.3 calculator

CNET staff

Enough readers have noted an issue with the Mac OS X 10.3.x calculator to warrant some explanation.

Brett Wickens writes "I have tried this on my G5 and my PowerBook under the Panther calculator, and it is a repeatable calculation flaw during simple addition:

"Add the following balances:

  • 1,866.75
  • 5,501.15
  • 30,099.00
  • 39,242.75
  • 7,766.90
  • 1,856.43

"and you will get the balance 86,352.97999999998! It seems to introduce the calculation flaw during the entry of the 30,099.00 amount, for no apparent reason.

"This makes the calculator pretty useless for financial calculations - especially long strings of calculations where the discrepancy could get amplified."

As explained by Knowledge Base article #25687, this is the result of a rounding error that is to be expected since the calculator uses floating-point arithmetic.

You can work around this issue by choosing a lower Precision setting from the View menu.

Resources

  • #25687
  • More from Late-Breakers