Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Is this C# SqlDecimal math bug?

Tags:

c#

I'm trying to implement SQL Server Vardecimal decompression. Values stored as 3 digits decimals per every 10 bits. But during implementation I found strange behavior of math. Here is simple test I made

private SqlDecimal Test() {

  SqlDecimal mantissa = 0;
  SqlDecimal sign = -1;
  byte exponent = 0x20;
  int numDigits = 0;

  // -999999999999999999999999999999999.99999

  for (int i = 0; i < 13; i++) {

    int temp = 999;

    //equal to mantissa = mantissa * 1000 + temp;
    numDigits += 3;
    int pwr = exponent - (numDigits - 1);
    mantissa += temp * (SqlDecimal)Math.Pow(10, pwr);
  }

  return sign * mantissa;
}

First 2 passes are fine, I have

999000000000000000000000000000000
999999000000000000000000000000000

but third have

999999998999999999999980020000000

Is it some bug in C# SqlDecimal math or am I doing something wrong?

like image 938
user2091150 Avatar asked Mar 04 '23 22:03

user2091150


2 Answers

This is an issue with how you're constructing the value to add here:

mantissa += temp * (SqlDecimal)Math.Pow(10, pwr);

The problem starts when pwr is 24. You can see this very clearly here:

Console.WriteLine((SqlDecimal) Math.Pow(10, 24));

The output on my box is:

999999999999999980000000

Now I don't know exactly where that's coming from - but it's simplest to remove the floating point arithmetic entirely. While it may not be efficient, this is a simple way of avoiding the problem:

static SqlDecimal PowerOfTen(int power)
{
    // Note: only works for non-negative power values at the moment!
    // (To handle negative input, divide by 10 on each iteration instead.)
    SqlDecimal result = 1;
    for (int i = 0; i < power; i++)
    {
        result = result * 10;
    }
    return result;
}

If you then change the line to:

mantissa += temp * PowerOfTen(pwr);

then you'll get the results you expect - at least while pwr is greater than zero. It should be easy to fix PowerOfTen to handle negative values as well though.

like image 93
Jon Skeet Avatar answered Mar 16 '23 06:03

Jon Skeet


Update

Amending the below method to just work with Parse and ToString should improve performance for larger numbers (which would be the general use case for these types):

public static SqlDecimal ToSqlDecimal(this BigInteger bigint)
{
    return SqlDecimal.Parse(bigint.ToString());
}

This trick also works for the double returned by the original Math.Pow call; so you could just do:

SqlDecimal.Parse(string.Format("{0:0}",Math.Pow(10,24)))

Original Answer

Obviously @JonSkeet's answer's best, as it only involves 24 iterations, vs potentially thousands in my attempt. However, here's an alternate solution, which may help out in other scenarios where you need to convert large integers (i.e. System.Numeric.BigInteger) to SqlDecimal / where performance is less of a concern.

Fiddle Example

//using System.Data.SqlTypes;
//using System.Numerics; //also needs an assembly reference to System.Numerics.dll

public static class BigIntegerExtensions
{
    public static SqlDecimal ToSqlDecimal(this BigInteger bigint)
    {
        SqlDecimal result = 0;
        var longMax = (SqlDecimal)long.MaxValue; //cache the converted value to minimise conversions
        var longMin = (SqlDecimal)long.MinValue;

        while (bigint > long.MaxValue)
        {
            result += longMax;
            bigint -= long.MaxValue;
        }
        while (bigint < long.MinValue)
        {
            result += longMin;
            bigint -= long.MinValue;
        }
        return result + (SqlDecimal)(long)bigint;
    }
}

For your above use case, you could use this like so (uses the BigInteger.Pow method):

mantissa += temp * BigInteger.Pow(10, pwr).ToSqlDecimal(); 
like image 43
JohnLBevan Avatar answered Mar 16 '23 06:03

JohnLBevan