Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Bad float conversion on C# using VS2012

I am getting a very weird behavior here with C#:

float num = 144771463f;
// num is 144771456.0

I have also tried

float num = Convert.ToSingle(144771463f);
// num is still 144771456.0

Why does it happen?

like image 487
MatanKri Avatar asked Mar 19 '23 17:03

MatanKri


1 Answers

You're hitting inherent limtations in the System.Single type (float is a name-alias of Single).

According to MSDN ( http://msdn.microsoft.com/en-us/library/system.single.aspx ) the Single type has 22 bits dedicated to the mantissa (also known as the significand), these are the bits that store the actual 'numbers' in the value (the other 10 bits store the positive/negative sign and the exponent).

22 bits means you have an effective range of 0-4,194,304 - whereas the number you're storing is 144,771,463, which is 2 decimal digits beyond the (approximate) 7-digit decimal precision of Single. So it will only store 144,771,46X (the value of the X digit won't be stored, but its magnitude will be stored in the exponent area).

You have a few options:

  1. Use System.Double which provides for 51 bits of mantissa
  2. Use System.Decimal which provides for 96 bits of numeric information, note that Decimal handles mathematical operations specially in a way that preserves more information at a cost of speed (I do not know the technical details, sorry).
like image 76
Dai Avatar answered Mar 28 '23 21:03

Dai