Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Floor division with small numbers returning wrong answer [duplicate]

I'm trying to do some calculations in my python script but i'm getting some weird results. For example:

0.03 // 0.01
>>> 2.0

If I upscale the numbers I get the expected results:

3.0 // 1.0
>>> 3.0

I'm pretty sure that the answer for the first code snippet should be 3.0 and not 2.0. Can someone explain me why this is happening and how to fix it?

like image 688
flpn Avatar asked Jan 08 '20 14:01

flpn


2 Answers

This is due to the floating point error. Note that with the above floor division, the remainder is not 0:

0.03 % 0.01
# 0.009999999999999998

So if instead we divide by:

0.03 // 0.009
# 3.0

The answer is correct. Hence 0.03 is not entirely divisible by 0.01 without some remainder due to floating point limitations

like image 86
yatu Avatar answered Sep 21 '22 11:09

yatu


As yatu already mentioned, this is due to floating point errors. Try this instead:

from decimal import Decimal
Decimal('0.03') // Decimal('0.01')
>>> 3
like image 34
Daan Klijn Avatar answered Sep 21 '22 11:09

Daan Klijn