Two General CS Questions [duplicate]

Question

When comparing two "real" numbers for equality, why should I not use the == operator, and what should I use instead?

What is the difference between coersion and casting? It is my general assumption that casting is when you force a value to be of another type like so:

int n = 9;
return double(n)/5;

Answer 1

To answer the first question directly: “[Why] should I not use the == operator”? The answer is because earlier operations have produced rounding errors, and it is, in general, impossible to compute a correct result of a function applied to incorrect data. If you have computed values x and y that model exact mathematical values x and y, but x and y have been affected by rounding errors, then there is no function of x and y that tells us whether x equals y.

This means it is impossible to compute whether x equals y in these circumstances. It is not the == operator that is a problem. (== is in fact one of the few floating-point operations that is always computed with no error at all; it always returns the exactly correct result given its inputs.) The problem is that there is no function that gives a correct answer from this incorrect input. (And this is not just a problem with ==. If you have an x with rounding errors, then almost any function computed with it will contain errors: sqrt(1-x*x) will have errors, and acos(x) will have errors. Worse, they might signal exceptions because 1-x*x might be incorrectly negative or x might be incorrectly greater than one.)

Then the question becomes “What do we do instead?”

“Fudging” whether a comparison reports true or false introduces new errors to a program. So the question is, what errors are acceptable in your application? If the comparison reports that two numbers are equal when they would be unequal with exact mathematics, is that acceptable or unacceptable? If the comparison reports that two numbers are unequal when they would be equal, is that acceptable or unacceptable?

The answers to these questions differ from program to program. Generally, there are multiple things to consider:

• How much error may already be present in the values being compared?
• For what cases in which the exact values would differ is it acceptable that the comparison report true?
• For what cases in which the exact values would be equal is it acceptable that the comparison report false?

The answers to the above question depend on each application, so there is no general answer about what to use instead of ==. Some applications may be able to use a relative tolerance in comparison, some may be able to use an absolute tolerance, some may need something else. And some applications might not find any comparison acceptable given the errors in the values. In such cases, they need to redesign the calculations to reduce errors, or find other solutions.

So:

• Beware of any advice to compare with a tolerance, either relative or absolute. Whether using a tolerance is acceptable depends on your application; there is no general rule. How much tolerance is acceptable depends on your application; there is no general rule.
• Whether there is any acceptable solution depends on the errors in your computed values. The errors may be too large to allow a solution. Beware of any general recommendations about magnitude of tolerance. Floating-point errors can vary from zero to infinity and are dependent on circumstances, so there is no general tolerance that works.
• Be aware that all functions computed from data with errors result in errors, not just ==.