Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Converting float to int in F#

Tags:

f#

Can someone please explain why int (0.4 * 10.0) is 4 yet int ((2.4 - 2.0) * 10.0) is 3?

And then also please explain how to get 4. i.e. Do I really have to do int (System.Math.Round((2.4 - 2.0) * 10.0))?

It's just so ugly for something that should be so simple.

like image 600
Lee Richardson Avatar asked Jan 04 '09 16:01

Lee Richardson


People also ask

Can you convert a float to an int?

A float value can be converted to an int value no larger than the input by using the math. floor() function, whereas it can also be converted to an int value which is the smallest integer greater than the input using math. ceil() function. The math module is to be imported in order to use these methods.

What is F in float value?

The "F" indicates that the literal numeric value it is appended to is a float value. This is necessary information for the compiler and if not present can lead to errors or the compiler otherwise interpreting the number incorrectly.

How do you convert a float to an int in Python?

Converting Floats to Integers Python also has a built-in function to convert floats to integers: int() . In this case, 390.8 will be converted to 390 . When converting floats to integers with the int() function, Python cuts off the decimal and remaining numbers of a float to create an integer.


2 Answers

The double representation of 2.4 - 2.0 is 0.399999999999. When you convert a number to int the decimal part is truncated so 3.999999999 is truncated to 3.

like image 106
Pedro Henriques Avatar answered Sep 19 '22 11:09

Pedro Henriques


Actually, in .Net there is a decimal data type which will help you with this problem, if your numbers can be represented exactly as decimal fractions.

2.4m - 2.0m == 0.4m
2.4 - 2.0   != 0.4
like image 26
Alexey Romanov Avatar answered Sep 17 '22 11:09

Alexey Romanov