Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Is C# Decimal Rounding Inconsistent?

Tags:

c#

rounding

I've been fighting decimal precision in C# coming from a SQL Decimal (38,30) and I've finally made it all the way to a rounding oddity. I know I'm probably overlooking the obvious here, but I need a little insight.

The problem I'm having is that C# doesn't produce what I would consider to be consistent output.

decimal a = 0.387518769125m; decimal b = 0.3875187691250002636113061835m;  Console.WriteLine(Math.Round(a, 11)); Console.WriteLine(Math.Round(b, 11)); Console.WriteLine(Math.Round(a, 11) == Math.Round(b, 11)); 

Yields

0.38751876912 0.38751876913 False 

Uhh, 0.38751876913? Really? What am I missing here?

From MSDN:

If the digit in the decimals position is odd, it is changed to an even digit. Otherwise, it is left unchanged.

Why am I seeing inconsistent results? The additional precision isn't changing the 'digit in the decimals position'...

like image 317
Bennett Dill Avatar asked Apr 12 '12 16:04

Bennett Dill


2 Answers

From MSDN:

If there is a single non-zero digit in d to the right of the decimals decimal position and its value is 5, the digit in the decimals position is rounded up if it is odd, or left unchanged if it is even. If d has fewer fractional digits than decimals, dis returned unchanged.

In your first case

decimal a = 0.387518769125m; Console.WriteLine(Math.Round(a, 11)); 

there is a single digit to the right of the 11th place, and that number is 5. Therefore, since position 11 is even, it is left unchanged. Thus, you get

0.38751876912 

In your second case

decimal b = 0.3875187691250002636113061835m; Console.WriteLine(Math.Round(b, 11)); 

there is not a single digit to the right of the 11th place. Therefore, this is straight up grade-school rounding; you round up if the next digit is greater than 4, otherwise you round down. Since the digit to the right of the 11th place is more than 4 (it's a 5), we round up so you see

0.38751876913 

Why am I seeing inconsistent results?

You're not. The results are completely consistent with the documentation.

like image 161
jason Avatar answered Oct 03 '22 23:10

jason


From MSDN - Math.Round Method (Decimal, Int32):

If there is a single non-zero digit in d to the right of the decimals decimal position and its value is 5, the digit in the decimals position is rounded up if it is odd, or left unchanged if it is even. If d has fewer fractional digits than decimals, d is returned unchanged.

The behavior of this method follows IEEE Standard 754, section 4. This kind of rounding is sometimes called rounding to nearest, or banker's rounding. It minimizes rounding errors that result from consistently rounding a midpoint value in a single direction.

Note the use of single non-zero digit. This corresponds to your first examples, but not the second.

And:

To control the type of rounding used by the Round(Decimal, Int32) method, call the Decimal.Round(Decimal, Int32, MidpointRounding) overload.

like image 30
Oded Avatar answered Oct 03 '22 21:10

Oded