Cleanly dealing with rounding/accumulation errors

Question

I have an application where I am tracking volumes of fluid in µL. I'm currently using 'double' for storage of volumes throughout the system and this works fine, in most cases. However, when I start adding and subtracting large numbers of these volumes, various accumulation errors start creeping in. The errors are very small in magnitude, but they cause problems with threshold comparisons, where suddenly the volume is smaller than expected by a miniscule amount, causing validation failures. I understand this is a fairly common problem with performing accumulating floating point operations, but I'm wondering how best to address the issues. A few thoughts I've had:

1. I can replace all my double references with integers and instead track everything in nL. This would definitely solve the problem, but it's a very invasive change. The system is not yet in production use, however, which means applying it now will be a lot easier than trying to apply it later.

2. I can use Decimal instead of double. This is less invasive than changing to integers, but still requires fairly significant changes.

3. I can require that all volume comparisons allow for a specified error tolerance. This is mostly what I'm doing right now, but it makes the comparison code uglier and it requires some code review to make sure nobody forgets to apply the pattern.

4. I can perform rounding to a specified tolerance after each computation to prevent the error accumulation. This makes the comparisons cleaner, but now it has a similar problem everywhere that there are assignments.

For those who have also struggled with this problem, what solutions wound up being the cleanest to implement? Are there other gotchas I should know about when performing accumulating calculations?

Answer 1

Ideally, you want to

• use a data type that's not subject to rounding (approximation) error, and
• round to the right number of decimal places
• at the right time.

You don't need the great range of a double-precision float. You probably don't want to use integers. They're slightly faster, but using them is more complex. Use numeric or decimal. Numeric and decimal data types aren't subject to either rounding errors or to errors of approximation. But you still can't be careless or sloppy in your programming; assigning a numeric value to a variable of type double will bring the problem right back to you.

Exactly what the right number of decimal places and the right time mean are dependent on your application. The right number of decimal places might sometimes be more than the number of places you need to store as your final value. The right time can be influenced by the number and nature of intermediate calculations.

Sometimes, what people call rounding error is really approximation error. If you tell your computer to store the decimal value r in a floating-point variable or database column, it will actually store the closest floating-point approximation to r.

Canonical reference for FP arithmetic