How to choose epsilon value for floating point? [closed]

Question

Since we know that 0.1 + 0.2 != 0.3 due to limited number representation, we need to instead check hat abs(0.1+0.2 - 0.3) < ε. The question is, what ε value should we generally choose for different types? Is it possible to estimate it depending on the number of bits and the number and types of operations that are likely to be performed?

Answer 1

A baseline value for epsilon is the difference between 1.0 and the next highest representable value. In C++, this value is available as std::numeric_limits<T>::epsilon().

Note that, at the minimum, you need to scale this value as a proportion of the actual number you're testing. Also, since the precision scales only roughly with the numeric value, you may want to increase your margin by a small factor to prevent spurious errors:

double epsilon = std::numeric_limits<double>::epsilon();

// C++ literals and math functions are double by default
bool is_near = abs(0.1+0.2 - 0.3) <= 0.3 * (2*epsilon);

As a more complete example, a function for comparing doubles:

bool is_approximately_equal(double a, double b) {
  double scale = max(abs(a), abs(b));
  return abs(a - b) <= scale * (2*epsilon);
}

In practice, the actual epsilon value you should use depends on what you're doing, and what kind of tolerance you actually need. Numeric algorithms will typically have precision tolerances (average and maximum) as well as time and space estimates. But the precision estimate typically starts with something like characteristic_value * epsilon.

Answer 2

You can estimate the machine epsilon using the algorithm below. You need to multiply this epsilon with the integer value of 1+(log(number)/log(2)). After you have determined this value for all numbers in your equation, you can use error analysis to estimate the epsilon value for a specific calculation.

epsilon=1.0

while (1.0 + (epsilon/2.0) > 1.0) {
  epsilon = epsilon /2.0     
}
//Calculate error using error analysis for a + b
epsilon_equation=Math.sqrt(2*epsilon*epsilon)

document.write('Epsilon: ' + epsilon_equation+'<br>')
document.write('Floating point error: ' + Math.abs(0.2 + 0.4 -0.6)+'<br>')
document.write('Comparison using epsilon: ')
document.write(Math.abs(0.2 + 0.4 -0.6)<epsilon_equation)

Following your comment, I have tried the same approach in C# and it seems to work:

using System;

namespace ConsoleApplication
{
   
    public class Program
    {
        public static void Main(string[] args)
        {
            double epsilon = 1.0;

            while (1.0 + (epsilon/2.0) > 1.0)
            {
                epsilon = epsilon/2.0;
            }

            double epsilon_equation = Math.Sqrt(2*epsilon*epsilon);

            Console.WriteLine(Math.Abs(1.0 + 2.0 - 3.0) < Math.Sqrt(3.0 * epsilon_equation * epsilon_equation));
        }
    }
}