I am doing some float manipulation and end up with the following numbers:
-0.5
-0.4
-0.3000000000000000004
-0.2000000000000000004
-0.1000000000000000003
1.10E-16
0.1
0.2
0.30000000000000000004
0.4
0.5
The algorithm is the following:
var inc:Number = nextMultiple(min, stepSize);
trace(String(inc));
private function nextMultiple(x:Number, y:Number) {
return Math.ceil(x/y)*y;
}
I understand the fact the float cannot always be represented accurately in a byte. e.g 1/3. I also know my stepsize being 0.1. If I have the stepsize how could I get a proper output?
The strange thing is that its the first time I've encountered this type of problem. Maybe I dont play with float enough.
A language agnostic solution would be to store your numbers as an integer number of steps, given that you know your step size, instead of as floats.
A non-language agnostic solution would be to find out what your language's implementation of printf is.
printf ("float: %.1f\n", number);
The limited floating point precision of binary numbers is your problem, as you recognize. One way around this is not to do floating point math. Translate the problem to integers, then translate back for the output.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With