Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Can't get 0.30000000000000004 by calculating

When I run in the console 0.1 + 0.2 the result is 0.30000000000000004. So I tried to calculate it myself. Here are the steps I've taken.

1) Represent 0.1 as IEEE754 double:

0.1 = 0 01111111011 1001100110011001100110011001100110011001100110011010

2) Represent 0.2 as IEEE754 double:

0.2 = 0 01111111100 1001100110011001100110011001100110011001100110011010

The calculations should be correct here since I've checked them using my custom function that shows how the number is stored by JavaScript.

3) Convert both numbers to scientific notation:

0.1 = 1.1001100110011001100110011001100110011001100110011010 x 2-4

0.2 = 1.1001100110011001100110011001100110011001100110011010 x 2-3

Now since exponent number is different, let's adjust the 0.2 to have -4:

0.2 = 0.11001100110011001100110011001100110011001100110011010 x 2-4

4) Add them:

  1.1001100110011001100110011001100110011001100110011010
+ 0.1100110011001100110011001100110011001100110011001101
  ------------------------------------------------------
 10.0110011001100110011001100110011001100110011001100111

So the sum is:

10.0110011001100110011001100110011001100110011001100111 x 2-4

5) Normalize it:

1.00110011001100110011001100110011001100110011001100111 x 2-3

6) Round it up to 52 bits after radix point:

1.1100110011001100110011001100110011001100110011010000 x 2-3

After I remove the exponent, then I end up with the following resulting number:

0.001110011001100110011001100110011001100110011001101 

When I convert it to decimal using this calculator it shows this:

0.225000000000000088817841970012523233890533447265625

Not exactly the expected 0.30000000000000004.

What am I missing here?


Attempt #2: (Fixed error when adjusting exponents)

3) Convert both numbers to scientific notation:

0.1 = 1.1001100110011001100110011001100110011001100110011010 x 2-4

0.2 = 1.1001100110011001100110011001100110011001100110011010 x 2-3

Now since exponent number is different, let's adjust the 0.2 to have -4:

0.2 = 11.00110011001100110011001100110011001100110011001101 x 2-4

4) Add them:

      1.1001100110011001100110011001100110011001100110011010
   + 11.0011001100110011001100110011001100110011001100110100
      ------------------------------------------------------
    100.110011001100110011001100110011001100110011001100111

So the sum is:

100.110011001100110011001100110011001100110011001100111 x 2-4

5) Normalize it:

1.00110011001100110011001100110011001100110011001100111 x 2-2

6) Round it to 52 bits using round to the nearest algorithm. The nearest here is the truncated number (rounded down):

1.0011001100110011001100110011001100110011001100110011 x 2-2

After I remove the exponent, then I end up with the following resulting number:

0.010011001100110011001100110011001100110011001100110011 

When I convert it to decimal using this calculator it shows this:

0.299999999999999988897769753748434595763683319091796875

It's almost there this time :), but is still off. Can you guys please help me figure out the problem?

like image 863
Max Koretskyi Avatar asked May 19 '16 15:05

Max Koretskyi


2 Answers

You have:

0.2 = 1.1001100110011001100110011001100110011001100110011010 x 2 ** -3

If you want to get to exponent -4, you need to multiply the mantissa by 2, not divide it:

0.2 = 10.1001100110011001100110011001100110011001100110011010 x 2 ** -4

But actually what you want is probably 0.1 with a -3 exponent:

0.1 = 0.1100110011001100110011001100110011001100110011001101(0) x 2 ** -3
like image 74
Holt Avatar answered Oct 20 '22 04:10

Holt


To answer your specific question in your "Attempt #2"...

1.00110011001100110011001100110011001100110011001100111 x 2^-2

is 54 bits, with bit 54 being '1', a halfway case. You have to round up to make the significand "even", so the answer is

1.00110011001100110011001100110011001100110011001101 x 2^-2

which is

0.0100110011001100110011001100110011001100110011001101

which is

0.3000000000000000444089209850062616169452667236328125
like image 2
Rick Regan Avatar answered Oct 20 '22 04:10

Rick Regan