"Rounding errors" and precision in the Mac OS X 10.3 calculator

"Rounding errors" and precision in the Mac OS X 10.3 calculator

Enough readers have noted an issue with the Mac OS X 10.3.x calculator to warrant some explanation.

Brett Wickens writes "I have tried this on my G5 and my PowerBook under the Panther calculator, and it is a repeatable calculation flaw during simple addition:

"Add the following balances:

  • 1,866.75
  • 5,501.15
  • 30,099.00
  • 39,242.75
  • 7,766.90
  • 1,856.43

"and you will get the balance 86,352.97999999998! It seems to introduce the calculation flaw during the entry of the 30,099.00 amount, for no apparent reason.

"This makes the calculator pretty useless for financial calculations - especially long strings of calculations where the discrepancy could get amplified."

As explained by Knowledge Base article #25687, this is the result of a rounding error that is to be expected since the calculator uses floating-point arithmetic.

You can work around this issue by choosing a lower Precision setting from the View menu.

Resources
  • #25687
  • More from Late-Breakers
  •  

    Join the discussion

    Conversation powered by Livefyre

    Don't Miss
    Hot Products
    Trending on CNET

    HOT ON CNET

    iPhone running slow?

    Here are some quick fixes for some of the most common problem in iOS 7.