Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Double precision problems on .NET

Tags:

c#

.net

math

double

I have a simple C# function:

public static double Floor(double value, double step)
{
    return Math.Floor(value / step) * step;
}

That calculates the higher number, lower than or equal to "value", that is multiple of "step". But it lacks precision, as seen in the following tests:

[TestMethod()]
public void FloorTest()
{
    int decimals = 6;
    double value = 5F;
    double step = 2F;
    double expected = 4F;
    double actual = Class.Floor(value, step);
    Assert.AreEqual(expected, actual);
    value = -11.5F;
    step = 1.1F;
    expected = -12.1F;
    actual = Class.Floor(value, step);
    Assert.AreEqual(Math.Round(expected, decimals),Math.Round(actual, decimals));
    Assert.AreEqual(expected, actual);
}

The first and second asserts are ok, but the third fails, because the result is only equal until the 6th decimal place. Why is that? Is there any way to correct this?

Update If I debug the test I see that the values are equal until the 8th decimal place instead of the 6th, maybe because Math.Round introduces some imprecision.

Note In my test code I wrote the "F" suffix (explicit float constant) where I meant "D" (double), so if I change that I can have more precision.

like image 528
Jader Dias Avatar asked Feb 19 '09 20:02

Jader Dias


People also ask

Does C# have doubles?

The double is a fundamental data type built into the compiler and used to define numeric variables holding numbers with decimal points. C, C++, C# and many other programming languages recognize the double as a type.

What does double precision mean?

Double precision means the numbers takes twice the word-length to store. On a 32-bit processor, the words are all 32 bits, so doubles are 64 bits.

How accurate is double precision?

Double precision numbers are accurate up to sixteen decimal places but after calculations have been done there may be some rounding errors to account for. In theory this should affect no more than the last significant digit but in practice it is safer to rely upon fewer decimal places.


2 Answers

I actually sort of wish they hadn't implemented the == operator for floats and doubles. It's almost always the wrong thing to do to ever ask if a double or a float is equal to any other value.

like image 74
Jon Grant Avatar answered Oct 07 '22 13:10

Jon Grant


Floating point arithmetic on computers are not Exact Science :).

If you want exact precision to a predefined number of decimals use Decimal instead of double or accept a minor interval.

like image 45
veggerby Avatar answered Oct 07 '22 14:10

veggerby