Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

erroneous Visual C float / double conversion?

In Visual C++ i wrote the following sample in a C++ program:

float f1 = 42.48f;
double d1 = 42.48;
double d2 = f1;

I compiled the program with Visual Studio 2005. In the debugger i see the following values:

f1  42.480000   float
d1  42.479999999999997  double
d2  42.479999542236328  double

d1 by my knowledege is OK, but d2 is wrong.

The problem occurs as well with /fp=precise as with /fp=strict as with /fp=fast.

Whats the problem here? Any hint how to avoid this Problem? This leads to serious numerical problems.

like image 934
RED SOFT ADAIR Avatar asked Apr 13 '26 00:04

RED SOFT ADAIR


1 Answers

This isn't an issue with VC++ or anything like that - it's a fundamental issue with how floating point numbers are stored on the computer. For more information, see IEEE-754.

The issue is that a conversion from float to double is done such that converting back from double to float results in exactly the same float value that you started with. I'm not aware of any way around the loss of precision, except to use only doubles when you need the longer precision. It may be that trying to round the converted float to two decimal places will set it to the correct value, but I'm not sure of that.

like image 62
Daniel G Avatar answered Apr 15 '26 15:04

Daniel G



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!