Here's a part of code that I dont understand:
byte b1 = (byte)(64 / 0.8f); // b1 is 79
int b2 = (int)(64 / 0.8f); // b2 is 79
float fl = (64 / 0.8f); // fl is 80
Why are the first two calculations off by one? How should I perform this operation, so its fast and correct?
EDIT: I would need the result in byte
EDIT: Not entirely correct, see: Why does a division result differ based on the cast type? (Followup)
Rounding issue: By converting to byte / int, you are clipping of the decimal places.
But 64 / 0.8
should not result in any decimal places? Wrong: Due to the nature of floating point numbers, 0.8f can not be represented exactly like that in memory; it is stored as something close to 0.8f (but not exactly). See Floating point inaccuracy examples or similar threads. Thus, the result of the calculation is not 80.0f, but 79.xxx where xxx is close to 1 but still not exactly one.
You can verify this by typing the following into the Immediate Window in Visual Studio:
(64 / 0.8f)
80.0
(64 / 0.8f) - 80
-0.0000011920929
100 * 0.8f - 80
0.0000011920929
You can solve this by using rounding:
byte b1 = (byte)(64 / 0.8f + 0.5f);
int b2 = (int)(64 / 0.8f + 0.5f);
float fl = (64 / 0.8f);
I'm afraid fast and correct are at odds in cases like this.
Binary floating point arithmetic almost always creates small errors, due to the underlying representation in our CPU architectures. So in your initial expression you actually get a value a little bit smaller than the mathematically correct one. If you expect an integer as the result of a particular mathematic operation and you get something very close to it, you can use the Math.Round(Double, MidpointRounding)
method to perform the correct rounding and compensate for small errors (and make sure you pick the MidpointRounding
strategy you expect).
Simply casting the result to a type such as byte
or int
doesn't do rounding - it simply cuts off the fractional part (even 1.99999f
will become 1
when you just cast it to these types).
Decimal floating point arithmetic is slower and more memory intensive, but doesn't cause these errors. To perform it, use decimal
literals instead of float
literals (e.g. 64 / 0.8m
).
The rule of thumb is:
decimal
.double
.float
.If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With