Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

custom method for returning decimal places shows odd behavior

I am writing a simple method that will calculate the number of decimal places in a decimal value. The method looks like this:

public int GetDecimalPlaces(decimal decimalNumber) { 
try {
    int decimalPlaces = 1;
    double powers = 10.0;
    if (decimalNumber > 0.0m) {
        while (((double)decimalNumber * powers) % 1 != 0.0) {
            powers *= 10.0;
            ++decimalPlaces;
        }
    }
    return decimalPlaces;

I have run it against some test values to make sure that everything is working fine but am getting some really weird behavior back on the last one:

int test = GetDecimalPlaces(0.1m);
int test2 = GetDecimalPlaces(0.01m);
int test3 = GetDecimalPlaces(0.001m);
int test4 = GetDecimalPlaces(0.0000000001m);
int test5 = GetDecimalPlaces(0.00000000010000000001m);
int test6 = GetDecimalPlaces(0.0000000001000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001m);

Tests 1-5 work fine but test6 returns 23. I know that the value being passed in exceeds the maximum decimal precision but why 23? The other thing I found odd is when I put a breakpoint inside the GetDecimalPlaces method following my call from test6 the value of decimalNumber inside the method comes through as the same value that would have come from test5 (20 decimal places) yet even though the value passed in has 20 decimal places 23 is returned.

Maybe its just because I'm passing in a number that has way too many decimal places and things go wonky but I want to make sure that I'm not missing something fundamentally wrong here that might throw off calculations for the other values later down the road.

like image 980
Jesse Carter Avatar asked Nov 20 '12 20:11

Jesse Carter


1 Answers

The number you're actually testing is this:

0.0000000001000000000100000000

That's the closest exact decimal value to 0.0000000001000000000100000000010000000001000000000100000000010000000001000000000100000000010000000001.

So the correct answer is actually 20. However, your code is giving you 23 because you're using binary floating point arithmetic, for no obvious reason. That's going to be introducing errors into your calculations, completely unnecessarily. If you change to use decimal consistently, it's fine:

public static int GetDecimalPlaces(decimal decimalNumber) {
    int decimalPlaces = 1;
    decimal powers = 10.0m;
    if (decimalNumber > 0.0m) {
        while ((decimalNumber * powers) % 1 != 0.0m) {
            powers *= 10.0m;
            ++decimalPlaces;
        }
    }
    return decimalPlaces;
}
like image 144
Jon Skeet Avatar answered Oct 04 '22 23:10

Jon Skeet