Python and very small decimals

Question

Interesting fact on Python 2.7.6 running on macosx:

You have a very small number like:

0.000000000000000000001

You can represent it as:

>>> 0.1 / (10 ** 20)
1.0000000000000001e-21

But you can see the floating pointing error at the end. What we really have is something like this:

0.0000000000000000000010000000000000001

So we got an error in the number, but thats ok. The problem is the following:

As expected:

>>> 0.1 / (10 ** 20) == 0
False

But, wait, whats this?

>>> 0.1 / (10 ** 20) + 1 == 1
True

>>> repr(0.1 / (10 ** 20) + 1)
'1.0'

Looks like python is using another data type to represent my number as this only happens when using the 16th decimal digit and so on. Also why python decided to automatically turn my number to 0 on the addition? Should not I be dealing with a decimal number with a very small decimal part and with a floating point error?

I know this problem can fall in the floating point errors umbrella, and the solution normally is do not trust floating point for this kinda of calculation, but I would like to understand more about whats happening under the hood.

Answer 1

Consider what happens when you attempt to add 7.00 and .001 and are only permitted to use three digits for the answer. 7.001 is not allowed. What do you do?

Of the values you can return, such as 6.99, 7.00, and 7.01, the closest to the correct answer is 7.00. So, when asked to add 7.00 and .001, you return 7.00.

The computer has the same issue when you ask it to add a number near 1e-21 and 1. It has only 53 bits to use for the fraction portion of the floating-point value, and 1e-21 is almost 70 bits below 1. So, it rounds the correct result to the closest value it can, 1, and returns that.

Answer 2

Floating point numbers work like scientific notation. 1.0000000000000001e-21 fits within the 53 bits of significand/mantissa a 64-bit float allows. Adding 1 to that, many orders of magnitude larger than the tiny fraction, causes the minor detail to be discarded, and storing exactly 1 instead