Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Why don't operations on double-precision values give expected results?

    System.out.println(2.14656);

2.14656

    System.out.println(2.14656%2);

0.14656000000000002

WTF?

like image 804
user293756 Avatar asked Dec 04 '22 11:12

user293756


2 Answers

The do give the expected results. Your expectations are incorrect.

When you type the double-precision literal 2.14656, what you actually get is the closest double-precision value, which is:

2.14656000000000002359001882723532617092132568359375

the println happens to round this when it prints it out (to 17 significant digits), so you see the nice value that you expect.

After the modulus operation (which is exact), the value is:

0.14656000000000002359001882723532617092132568359375

Again, this is rounded when it gets printed, but because there is one less leading digit, the round point is one digit farther to the right, and so you see that trailing 2.

like image 151
Stephen Canon Avatar answered Jan 05 '23 17:01

Stephen Canon


A base-2 system is being used to store a base-10 number of arbitrary precision.

When using integer binary, the "round" numbers are 2, 4, 8, 16, 32, 64, etc. You'll note that none of these are 10, 100, 1000, 10000, or anything that would be considered a "round" number in base-10.

Float and double values were constructed for fast operations within a CPU. Math between multiple numbers can be quickly done, even if you're dividing 1.42E-13 by 2.11E47. They were not constructed to provide "round" numbers in base-10. This is the side-effect of that decision.

When using floating point numbers, you have to accept that this will happen. It's not 100% precise, but it is accurate otherwise. So you should never use floating point variables where you need 100% precision. Where you need to do this, find a way to store it in an int value. It's also a good idea, where you do use a floating point number, to round the output to a number of decimal places to avoid the occasional display like you have there.

For example, I work in the financial industry, and we represent market prices as a number of ticks rather than an actual price. When we need to display a price, we multiply this value by the tick size, 1/100, 1/32, or 1/20, etc, and then round to the appropriate number of decimal places. The number of ticks, the numerator and denominator of the tick size, are all stored as integers in our databases. We don't actually store floating point values for anything except for calculated values like moving averages.

like image 30
Erick Robertson Avatar answered Jan 05 '23 17:01

Erick Robertson