I'm getting some strange results doing a calculation for the application I'm working on and I thought someone on here might be able to help figure out what's going on.
The requirements for this particular calculation state that the calculation should look like this:
A and B are known
A * B = C
For this particular calculation
A = 0.0410
B = 123456789010
Here are the results I'm seeing:
Calculator:
0.0410 * 123456789010 = 5061728349.41
Java:
B is a double:
0.0410f * 123456789010d = 5.061728489223363E9 = 5061728489.223363
B is a long:
0.0410f * 123456789010l = 5.0617288E9
The loss of precision is of less importance to me (I only need 9 digits of precision anyway) than the difference in the 10s and 1s spot. Why does doing the calculation using the double give me the "wrong" result?
Incidentally, I tried doing the calculation using BigDecimal
and got the same result as I did using a double.
The various type conversions that happen are specified by the JLS #5.6.2. In your case (extract):
- If either operand is of type double, the other is converted to double.
- Otherwise, if either operand is of type float, the other is converted to float.
In 0.0410f * 123456789010d = 506172848.9223363
, 0.0410f
is first converted to a double which is not necessarily equal to 0.0410d
. Actually you can try it and see that is is not:
double d1 = 0.041d;
double d2 = 0.041f;
System.out.println(new BigDecimal(d1));
System.out.println(new BigDecimal(d2));
outputs:
0.041000000000000001720845688168992637656629085540771484375
0.041000001132488250732421875
In your next example:
0.0410f * 123456789010L = 506172832
the long is converted to a float, which you can verify with this example:
float f1 = 0.0410f;
float f2 = 123456789010L;
System.out.println(new BigDecimal(f1)); // 0.041000001132488250732421875
System.out.println(new BigDecimal(f2)); // 123456790528
System.out.println(new BigDecimal(0.0410f * 123456789010L)); // 5061728768
System.out.println(new BigDecimal(f1 * f2)); // 5061728768
As for the precision of float / double operations in general, check this question.
Finally, if you use a BigDecimal, you get the correct answer:
BigDecimal a = new BigDecimal("0.041");
BigDecimal b = new BigDecimal("123456789010");
System.out.println(a.multiply(b)); // outputs 5061728349.410
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With