Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Why fmod(1.0,0.1) == .1?

I experienced this phenomenon in Python first, but it turned out that it is the common answer, for example MS Excel gives this. Wolfram Alpha gives an interesting schizoid answer, where it states that the rational approximation of zero is 1/5. ( 1.0 mod 0.1 )

On the other hand, if I implement the definition by hand it gives me the 'right' answer (0).

def myFmod(a,n):
    return a - floor(a/n) * n

What is going on here. Do I miss something?

like image 933
beemtee Avatar asked Nov 29 '22 04:11

beemtee


2 Answers

Because 0.1 isn't 0.1; that value isn't representable in double precision, so it gets rounded to the nearest double-precision number, which is exactly:

0.1000000000000000055511151231257827021181583404541015625

When you call fmod, you get the remainder of division by the value listed above, which is exactly:

0.0999999999999999500399638918679556809365749359130859375

which rounds to 0.1 (or maybe 0.09999999999999995) when you print it.

In other words, fmod works perfectly, but you're not giving it the input that you think you are.


Edit: Your own implementation gives you the correct answer because it is less accurate, believe it or not. First off, note that fmod computes the remainder without any rounding error; the only source of inaccuracy is the representation error introduced by using the value 0.1. Now, let's walk through your implementation, and see how the rounding error that it incurs exactly cancels out the representation error.

Evaluate a - floor(a/n) * n one step at a time, keeping track of the exact values computed at each stage:

First we evaluate 1.0/n, where n is the closest double-precision approximation to 0.1 as shown above. The result of this division is approximately:

9.999999999999999444888487687421760603063276150363492645647081359...

Note that this value is not a representable double precision number -- so it gets rounded. To see how this rounding happens, let's look at the number in binary instead of decimal:

1001.1111111111111111111111111111111111111111111111111 10110000000...

The space indicates where the rounding to double precision occurs. Since the part after the round point is larger than the exact half-way point, this value rounds up to exactly 10.

floor(10.0) is, predictably, 10.0. So all that's left is to compute 1.0 - 10.0*0.1.

In binary, the exact value of 10.0 * 0.1 is:

1.0000000000000000000000000000000000000000000000000000 0100

again, this value is not representable as a double, and so is rounded at the position indicated by a space. This time it rounds down to exactly 1.0, and so the final computation is 1.0 - 1.0, which is of course 0.0.

Your implementation contains two rounding errors, which happen to exactly cancel out the representation error of the value 0.1 in this case. fmod, by contrast, is always exact (at least on platforms with a good numerics library), and exposes the representation error of 0.1.

like image 178
Stephen Canon Avatar answered Dec 09 '22 19:12

Stephen Canon


This result is due to machine floating point representation. In your method, you are 'casting' (kinda) the float to an int and do not have this issue. The 'best' way to avoid such issues (esp for mod) is to multiply by a sufficiently large enough int (only 10 is needed in your case) and perform the operation again.

fmod(1.0,0.1)

fmod(10.0,1.0) = 0

like image 43
g19fanatic Avatar answered Dec 09 '22 21:12

g19fanatic