Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Why float variable saves value by cutting digits after point in a weird way?

I have this simple code line:

float val = 123456.123456;

when i print this val or look in scope, it stores value 123456.13

Ok, it's fine, it can't store all those digits after point just in 4 bytes, but why does it make 13 after the point? Shouldn't it be 12?

(using vc++ 2010 express on win32)

like image 526
Kosmo零 Avatar asked Feb 17 '12 13:02

Kosmo零


4 Answers

In binary,123456.123456 is 11110001001000000.000111111001... (infinite). It rounds to 11110001001000000.001, or 123456.125. That rounds to 123456.13 when printed.

like image 55
Rick Regan Avatar answered Nov 20 '22 01:11

Rick Regan


The value stored in val is equal to 123456.125. You are getting .13 because you are rounding it:

float val = 123456.123456;
printf("%.4f %.2f\n", val, val);

output: 123456.1250 123456.13

You should use double in this case to avoid truncation. Compiler should also warn you: "warning C4305: 'initializing' : truncation from 'double' to 'float'".

like image 39
LihO Avatar answered Nov 20 '22 03:11

LihO


When represented as a float, your number has an exponent of 16 (i.e. the value is its mantisse times 2^16, or 65536). The mantisse then becomes

123456.123456 / 65536 = 1.8837909462890625

In order to fit in a 32-bit float, the mantisse is truncated to 23 bits, so now it becomes 1.883791. When multiplied back by 65536, it becomes 123456.125.

Note the 5 in the third position after the decimal point: the output routine of C++ that you used rounds it up, making your final number look like 123456.13.

EDIT Explanation of the rounding: (Rick Regan's comment)

The rounding occurs first in binary (to 24 bits), in decimal to binary conversion, and then to decimal, in printf. The stored value is 1.1110001001000000001 x 2^16 = 1.8837909698486328125 x 2^16 = 123456.125. It prints as 123456.13, but only because Visual C++ uses "round half away from zero" rounding.

Rick has an outstanding article on the subject, too.

If you would like to play with other numbers and their float representations, here is a very useful IEEE-754 calculator.

like image 7
Sergey Kalinichenko Avatar answered Nov 20 '22 01:11

Sergey Kalinichenko


Try printing the value of std::numeric_limits<float>::digits10. This is roughly speaking how much precision in base 10 a float has. You're trying to exceed it, so you're experiencing a loss of precision (meaning that digits beyond the significant ones are not really meaningful).

See e.g. What is the meaning of numeric_limits<double>::digits10

like image 2
John Zwinck Avatar answered Nov 20 '22 02:11

John Zwinck