Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Apparent F#/BCL floating point bug

The following is in FSI:

> System.Math.Round(0.2916, 2);;
val it : float = 0.29
> it * 100.;;
val it : float = 29.0
> int it;;
val it : int = 28

The result is the same everywhere I tried - a compiled F# 3.1/.NET 4.0 application, FSI in Visual Studio 2013 and 2015, .NET Fiddle, @fsibot....

Surely this is a bug somewhere, isn't it? What's going on here?

like image 720
TeaDrivenDev Avatar asked Aug 13 '26 16:08

TeaDrivenDev


1 Answers

This is just how floating point numbers work. The number that appears as 29 in the output is actually slightly smaller than 29 (because floating point numbers are not precise):

> (System.Math.Round(0.2916, 2) * 100.0) - 29.0;;
val it : float = -3.552713679e-15
like image 158
Tomas Petricek Avatar answered Aug 16 '26 11:08

Tomas Petricek