Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Is this a bug? Float operation being treated as integer

Tags:

c#

asp.net

This operation returns a 0:

string value = “0.01”;
float convertedValue = float.Parse(value);
return (int)(convertedValue * 100.0f);

But this operation returns a 1:

string value = “0.01”;
float convertedValue = float.Parse(value) * 100.0f;
return (int)(convertedValue);

Because the convertedValue is a float, and it is in parenthesis *100f shouldn't it still be treated as float operation?

like image 257
Mark1270287 Avatar asked Mar 14 '12 22:03

Mark1270287


2 Answers

The difference between the two lies in the way the compiler optimizes floating point operations. Let me explain.

string value = "0.01";
float convertedValue = float.Parse(value);
return (int)(convertedValue * 100.0f);

In this example, the value is parsed into an 80-bit floating point number for use in the inner floating point dungeons of the computer. Then this is converted to a 32-bit float for storage in the convertedValue variable. This causes the value to be rounded to, seemingly, a number slightly less than 0.01. Then it is converted back to an 80-bit float and multiplied by 100, increasing the rounding error 100-fold. Then it is converted to an 32-bit int. This causes the float to be truncated, and since it is actually slightly less than 1, the int conversion returns 0.

string value = "0.01";
float convertedValue = float.Parse(value) * 100.0f;
return (int)(convertedValue);

In this example, the value is parsed into an 80-bit floating point number again. It is then multiplied by 100, before it is converted to a 32-bit float. This means that the rounding error is so small that when it is converted to a 32-bit float for storage in convertedValue, it rounds to exactly 1. Then when it is converted to an int, you get 1.

The main idea is that the computer uses high-precision floats for calculations, and then rounds the values whenever they are stored in a variable. The more assignments you have with floats, the more the rounding errors accumulate.

like image 183
Kendall Frey Avatar answered Sep 22 '22 04:09

Kendall Frey


Please read an introduction to floatingpoint. This is a typical floating point problem. Binary floating points can't represent 0.01 exactly.

0.01 * 100 is approximately 1.

If it happens to be rounded to 0.999... you get 0, and if it gets rounded to 1.000... you get 1. Which one of those you get is undefined.

The jit compiler is not required to round the same way every time it encounters a similar expression(or even the same expression in different contexts). In particular it can use higher precision whenever it wants to, but can downgrade to 32 bit floats if it thinks that's a good idea.


One interesting point is an explicit cast to float (even if you already have an expression of type float). This forces the JITer to reduce the precision to 32 bit floats at that point. The exact rounding is still undefined though.

Since the rounding is undefined, it can vary between .net versions, debug/release builds, the presence of debuggers (and possibly the phase of the moon :P).

Storage locations for floating-point numbers (statics, array elements, and fields of classes) are of fixed size. The supported storage sizes are float32 and float64. Everywhere else (on the evaluation stack, as arguments, as return types, and as local variables) floating-point numbers are represented using an internal floating-point type.

When a floating-point value whose internal representation has greater range and/or precision than its nominal type is put in a storage location, it is automatically coerced to the type of the storage location. This can involve a loss of precision or the creation of an out-of-range value (NaN, +infinity, or -infinity). However, the value might be retained in the internal representation for future use, if it is reloaded from the storage location without having been modified. It is the responsibility of the compiler to ensure that the retained value is still valid at the time of a subsequent load, taking into account the effects of aliasing and other execution threads (see memory model (§12.6)). This freedom to carry extra precision is not permitted, however, following the execution of an explicit conversion (conv.r4 or conv.r8), at which time the internal representation must be exactly representable in the associated type.


Your specific problem can be solved by using Decimal, but similar problems with 3*(1/3f) won't be solved by this, since Decimal can't represent one third exactly either.

like image 38
CodesInChaos Avatar answered Sep 22 '22 04:09

CodesInChaos