Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Decimal rounding problems

Tags:

c#

types

Given that Decimal.MaxValue = 79228162514264337593543950335m

Why does the next line give me 7922816251426433759354395034M in the Local window instead of 7922816251426433759354395033.5m as expected?

Decimal target = Decimal.MaxValue / 10m;

like image 868
izokurew Avatar asked Dec 04 '08 20:12

izokurew


1 Answers

I suspect this is a compiler error, actually.

Here's a short but complete program to show why I think that:

using System;

class Test
{
    static void Main()
    {
        decimal constant = decimal.MaxValue / 10m;
        decimal calculated = decimal.MaxValue;
        calculated /= 10m;

        Console.WriteLine (constant);
        Console.WriteLine (calculated);        
    }
}

Output:

7922816251426433759354395034
7922816251426433759354395033.5

I'll dig into the spec to see what guarantees are given.

EDIT: In the spec, section 7.18 claims:

The compile-time evaluation of constant expressions uses the same rules as run-time evaluation of non-constant expressions, except that where run-time evaluation would have thrown an exception, compile-time evaluation causes a compile-time error to occur.

That's clearly not the case here. Hmm.

EDIT: I've submitted a bug to Microsoft Connect. We'll see what happens to it.

like image 154
Jon Skeet Avatar answered Oct 01 '22 01:10

Jon Skeet