Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Why is (long)9223372036854665200d giving me 9223372036854665216?

I know about weird stuff with precision errors, but I can't fathom,

Why is (long)9223372036854665200d giving me 9223372036854665216 ?

like image 299
Pacerier Avatar asked Oct 18 '11 17:10

Pacerier


3 Answers

9223372036854665200d is a constant of type double. However, 9223372036854665200 does not fit in a double without loss of precision. A double only has 52 bits of mantissa, whereas the number in question requires 63 bits to be represented exactly.

The nearest double to 9223372036854665200d is the number whose mantissa equals 1.1111111111111111111111111111111111111111111110010100 in binary and whose exponent is 63 (decimal). This number is none other than 9223372036854665216 (call it U).

If we decrease the mantissa one notch to 1.1...0011, we get 9223372036854664192 (call it L).

The original number is between L and U and is much closer to U than it is to L

Finally, if you think that this truncation of the mantissa ought to result in a number that ends in a bunch of zeros, you're right. Only it happens in binary, not in decimal: U in base-16 is 0x7ffffffffffe5000 and L is 0x7ffffffffffe4c00.

like image 117
NPE Avatar answered Nov 07 '22 22:11

NPE


Because doubles don't have that much precision. Why are you doing such a strange thing? Change the d to l.

like image 16
user207421 Avatar answered Nov 07 '22 22:11

user207421


Doubles have 52-53 bit precision, whereas a long has 64 bit precision (for integers only). The loss of precision in a double is used to represent the exponent, which allows a double to represent larger/smaller numbers than a long can.

Your number is 19 digits long, whereas a double can only store roughly 16 digits of (decimal) integer data. Thus the final number ends up being rounded.

Reference: Double - Wikipedia

like image 6
ty1824 Avatar answered Nov 07 '22 20:11

ty1824