Double precision theorie [duplicate]

Question

The following expression returns false (e.g in Java and C#)

0.1 + 0.1 + 0.1 == 0.3

So we learned that we always compare doubles and floats like this

Math.abs(double1 - double2) < epsilon

But why does

0.1 + 0.1 == 0.2 returns true and 
0.1 + 0.1 + 0.1 == 0.3 returns false?

I know that it has something to do with the mantissa, but I don't understand it exactly.

Answer 1

Float/double are stored as binary fractions, not decimal fractions.

There are some numbers that cannot be represented fully with our decimal notation. For example, 1/3 in decimal notation is 0.3333333... The same thing happens in binary notation, except that the numbers that cannot be represented precisely are different. Among them is the number 1/10. In binary notation that is 0.000110011001100...

Since the binary notation cannot store it precisely, it is stored in a rounded-off way. Hence your problem.

You should not compare doubles in the way you do like: 0.1 + 0.1 + 0.1 == 0.3, because you never know how exactly they are stored in the memory and you will never know what will be the result of such comparison.

Answer 2

@msporek his explanation is right. Here is in detail at bit-level why it turns out false or true in both cases.

First, let's do 0.1 + 0.1 manually using the IEEE 754 floating point model:

    Dec    IEEE 754           52-bit mantisse
             ----------------------------------------------------
    0.1 =  1.1001100110011001100110011001100110011001100110011010 * 2^-4
    0.1 =  1.1001100110011001100110011001100110011001100110011010 * 2^-4
 +  -------------------------------------------------------------------
    0.2 = 11.0011001100110011001100110011001100110011001100110100 * 2^-4
        =  1.1001100110011001100110011001100110011001100110011010 * 2^-3

This is a perfect match, which means that converting 0.2 to IEEE 754 and the sum of 0.1 and 0.1 in IEEE 754 are bitwise equal. Now let's look at: 0.2 + 0.1

    Dec    IEEE 754            52-bit mantisse
             ----------------------------------------------------
    0.2 =  1.1001100110011001100110011001100110011001100110011010 * 2^-3
    0.1 =  1.1001100110011001100110011001100110011001100110011010 * 2^-4
 +  -------------------------------------------------------------------
    0.2 =  1.1001100110011001100110011001100110011001100110011010 * 2^-3
    0.1 =  0.1100110011001100110011001100110011001100110011001101 * 2^-3
 +  -------------------------------------------------------------------
    0.3 = 10.0110011001100110011001100110011001100110011001100111  * 2^-3
        =  1.00110011001100110011001100110011001100110011001100111 * 2^-2
        =  1.0011001100110011001100110011001100110011001100110100  * 2^-2
                                                              ^^^
                                                          These bits

Now, look at the last bits of the result of the addition: it is 100. While 0.3 should have had a 011 as last bits. (We will verify this with a test program below).

You might think now that a CPU has FPUs with 80 bits mantisse, that is right, and behavior is very situation and hardware dependent, I think. Chances are that it gets rounded to 52 bits of precision.

Extra check using a test program to produce the IEEE 754 representation in memory:
Now doing it with the computer gives this as result which is perfectly in agreement with what I did by hand:

        Dec    IEEE 754            52-bit mantisse
                 ----------------------------------------------------
        0.3 =  1.0011001100110011001100110011001100110011001100110011 * 2^-2
  0.2 + 0.1 =  1.0011001100110011001100110011001100110011001100110100 * 2^-2

Indeed: the last three bits are different.