Can some more experienced guys explain this strange error I found today? I was getting strange amounts when I was loading table data with my C# script.
As it turns out the problem is a different output from similar functions:
string amount_in_string = "1234567,15";
double amount_in_double = double.Parse(amount_in_string);
float amount_in_float = float.Parse(amount_in_string);
//amount_in_double = 1234567.15;
//amount_in_float = 1234567.13;
Why do I get such a different result when float and double are similar types (floating point). Can the precision make a difference with small amounts like these?
When “1234567.15” is converted to double
, the result is the closest value representable in double
, which is 1234567.1499999999068677425384521484375. Although you report in the question that the value is 1234567.15, the actual value is 1234567.1499999999068677425384521484375. “1234567.15” would be displayed when the value is displayed with a limited number of decimal digits.
When “1234567.15” is converted to float
, the result is the closet value representable in float
, which is 1234567.125. Although you report the value is 1234567.13, the actual value is 1234567.125. “1234567.13” may be displayed when the value is displayed with a limited number of decimal digits.
Observe that 1234567 exceeds 1,048,576, which is 220. The 32-bit floating-point format used for float
uses 24 bits for the significand (fraction portion of the number). If the high bit of the significand represents 220, the low bit represents 220−23 = 2−3 = ⅛. This is why you see “1234567.15” converted to a value rounded to the nearest eighth.
Floating point numbers are never exact, they are representations of numbers. An example commonly used is think of
1/3 + 1/3 = 2/3
...so the answer in floating point numbers, .33333 + .33333
, is not 2/3rds exactly, it is .66666
.
Long story short, the more precise fraction you take that can't be converted to an exact binary is going to always have a rounding number. The more precise the more likely it will have rounding errors.
Keep in mind if you do multiple different fractions, you can even have multiple different rounding errors that either make the number accidentally correct, or even further off.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With