Precision in Erlang

Question

Next code gives me 5.999999999999998 in Result, but right answer is 6.

Alpha = math:acos((4*4 + 5*5 - 3*3) / (2*4*5))
Area = 1/2 * 4 * 5 * math:sin(Alpha)

Is it possible to get 6?

Answer 1

You have run into a problem so common that it has its own web site, What Every Programmer Should Know About Floating-Point Arithmetic. The problem is due to the way floating-point arithmetic works in pretty much every CPU on the market that supports FP arithmetic; it is not specific to Erlang.

If regular floating point arithmetic does not give you the precision or accuracy you need, you can use an arbitrary precision arithmetic library instead of the built-in arithmetic. Perhaps the most well-known such library is GMP, but you'd have to wrap it in NIFs to use it from Erlang.

There is at least one pure-Erlang alternative, but I have no experience with it, so I cannot personally endorse it.

Answer 2

The calculation is done using standard floating point arithmetic on your hardware. Sometimes rounding errors show up.

Do you really need 15 digits of precision?

To get a more "exact" value there are multiple options:

> round(Area). % Will round to integer
6

or you could round to some precision

round(Area * 10000000) / 10000000.
6.0

If the purpose is to print the value, then printing with the default output for floats give you less precision.

io:format("~f~n", [Area]).
6.000000
ok

or with a specific precision

io:format("~.14f~n", [Area]).
6.00000000000000
ok

HTH