Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Why is implicitly rounding up a number no matter what dtype i use?

Tags:

python

I have been trying to fix the precision issue in my code that has been breaking. I want the value to be presented exactly as i provided but it seems like Python is rounding up the number. Below is the sample.

x = 3069682093880544.81
print (x)
3069682093880545.0
x = Decimal(3069682093880544.81)
print (x)
3069682093880545
x = float(3069682093880544.81)
print(x)
3069682093880545.0
x = Decimal(str(3069682093880544.81))
print(x)
3069682093880545.0
3069682093880545.0
x = str(3069682093880544.81)
print(x)
3069682093880545.0

All i want is to be able to assign exact value to the variable and it provides me the same value when called. What am i doing wrong?

like image 748
Hardy Avatar asked Dec 17 '22 13:12

Hardy


1 Answers

The number 3069682093880544.81 is being converted into a 64 bit floating point number according the IEEE format. The closest number in that format is 43910A47D717A69F. However, converting that number back will be 3069682093880544.64. As you can see, the last 2 digits after the comma have changed.

The number of significant digits in a IEEE 64 bit float is 16 digits. And that's likely why the printed output choses to stop printing after 16 digits, which is 3069682093880545.

If you want more decimal places, you need to chose a method which does not have a IEEE floating point number in the way of its processing. (Note that even the source code interpreter will parse numbers into floating point format already.) As mentioned by @LeopardShark,

from decimal import *
print(Decimal("3069682093880544.81"))

goes from String to Decimal without any processing as float.

like image 73
Thomas Weller Avatar answered May 03 '23 20:05

Thomas Weller