Copy decimal into byte array without allocations

Question

The Dream

I'm looking for a way to copy a decimal value into a byte array buffer and then be able to read those bytes back into a decimal without any heap allocations. Ideally this wouldn't require an unsafe context.

My Attempts

I've used makeshift unions in C# to do some crazy things before. It's a pretty cool way to just read memory any random way you want, but you've gotta be careful. You can get into corrupted states where a variable my explicitly be, for example byte[] but the value as viewed in the debugger is int[]. I didn't even know such a thing was possible!

NOTE: Marc makes a really important point in the comments below. You can't reliably convert numbers directly into bytes using an overlapping struct concept like this due to endianness. In this case you can safely use ints because the decimal type used 4 ints internally. Here's an [example] from protobuf-net's decimal serializer.

1: struct union w/ decimal and byte[]

This first attempt tries to use the struct union concept w/ a decimal and byte[] field, both at the 0 offset, so they're taking up the exact same memory location. I could then write to one field and read from the other field.

[StructLayout(LayoutKind.Explicit)]
private readonly struct DecimalByteConverter
{
    [FieldOffset(0)]
    public readonly decimal value;
    [FieldOffset(0)]
    public readonly byte[] bytes;

    public DecimalByteConverter(decimal value)
    {
        bytes = default;
        this.value = value;
    }
}

This doesn't even run w/out throwing -- type fails to load with the following:

System.TypeLoadException : Could not load type 'DecimalByteConverter' from assembly 'teloneum, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null' because it contains an object field at offset 0 that is incorrectly aligned or overlapped by a non-object field.

Apparently the CLR doesn't like them being overlapped -- but only because one is a value type and the other is a reference type.

2: struct union w/ decimal and 16 bytes!

So I decided to remove the reference type! Now, the following works, or at least, it will work, but the ordering is all messed up. I can certainly can a big decimal number, run it through here, and then just make by field offsets such that it matches my test, but that just seems wrong -- plus, there's at least three zero bytes in every decimal I tried this for :) Maybe I just got lazy at the end... it is late after all.

[StructLayout(LayoutKind.Explicit)]
private readonly struct DecimalByteConverter
{
    [FieldOffset(0)] public readonly decimal value;

    [FieldOffset( 0)] public readonly byte byte_1;
    [FieldOffset( 1)] public readonly byte byte_2;
    [FieldOffset( 2)] public readonly byte byte_3;
    [FieldOffset( 3)] public readonly byte byte_4;
    [FieldOffset( 4)] public readonly byte byte_5;
    [FieldOffset( 5)] public readonly byte byte_6;
    [FieldOffset( 6)] public readonly byte byte_7;
    [FieldOffset( 7)] public readonly byte byte_8;
    [FieldOffset( 8)] public readonly byte byte_9;
    [FieldOffset( 9)] public readonly byte byte_10;
    [FieldOffset(10)] public readonly byte byte_11;
    [FieldOffset(11)] public readonly byte byte_12;
    [FieldOffset(12)] public readonly byte byte_13;
    [FieldOffset(13)] public readonly byte byte_14;
    [FieldOffset(14)] public readonly byte byte_15;
    [FieldOffset(15)] public readonly byte byte_16;

    public DecimalByteConverter(decimal value)
    {
        byte_1  = default;
        byte_2  = default;
        byte_3  = default;
        byte_4  = default;
        byte_5  = default;
        byte_6  = default;
        byte_7  = default;
        byte_8  = default;
        byte_9  = default;
        byte_10 = default;
        byte_11 = default;
        byte_12 = default;
        byte_13 = default;
        byte_14 = default;
        byte_15 = default;
        byte_16 = default;
        this.value = value;
    }

    public DecimalByteConverter(int startIndex, byte[] buffer)
    {
        value = default;
        byte_1 = buffer[startIndex++];
        byte_2 = buffer[startIndex++];
        byte_3 = buffer[startIndex++];
        byte_4 = buffer[startIndex++];
        byte_5 = buffer[startIndex++];
        byte_6 = buffer[startIndex++];
        byte_7 = buffer[startIndex++];
        byte_8 = buffer[startIndex++];
        byte_9 = buffer[startIndex++];
        byte_10 = buffer[startIndex++];
        byte_11 = buffer[startIndex++];
        byte_12 = buffer[startIndex++];
        byte_13 = buffer[startIndex++];
        byte_14 = buffer[startIndex++];
        byte_15 = buffer[startIndex++];
        byte_16 = buffer[startIndex];
    }


    public static void Copy(decimal value, int startIndex, byte[] buffer)
    {
        var convert = new DecimalByteConverter(value);

        buffer[startIndex++] = convert.byte_1;
        buffer[startIndex++] = convert.byte_2;
        buffer[startIndex++] = convert.byte_3;
        buffer[startIndex++] = convert.byte_4;
        buffer[startIndex++] = convert.byte_5;
        buffer[startIndex++] = convert.byte_6;
        buffer[startIndex++] = convert.byte_7;
        buffer[startIndex++] = convert.byte_8;
        buffer[startIndex++] = convert.byte_9;
        buffer[startIndex++] = convert.byte_10;
        buffer[startIndex++] = convert.byte_11;
        buffer[startIndex++] = convert.byte_12;
        buffer[startIndex++] = convert.byte_13;
        buffer[startIndex++] = convert.byte_14;
        buffer[startIndex++] = convert.byte_15;
        buffer[startIndex]   = convert.byte_16;
    }

    public static decimal Read(int startIndex, byte[] buffer)
    {
        var convert = new DecimalByteConverter(startIndex, buffer);

        return convert.value;
    }
}

Answer 1

Before .NET 5.0, this was awkward without some ugly hackery. From .NET 5.0, there are more methods accepting spans.

You can use the GetBits(decimal d, Span<int>) method using a stack-allocated span, and then convert the four integers into the existing byte array however you want, e.g. with BitConverter.TryWriteBytes.

In the other direction, there's a Decimal(ReadOnlySpan<int>) constructor, so again you can stackalloc a Span<int>, use BitConverter.ToInt32(ReadOnlySpan<byte>) repeatedly to populate that span from the byte array, and pass it to the constructor.

As an aside, instead of accepting byte arrays and start indexes, you might want to embrace spans more widely through your code base.

Here's some sample code which does all of the above - it may well be possible to do it slightly more efficiently, but hopefully this gets the idea across, and this does avoid allocation:

using System;

class Program
{
    public static void Copy(decimal value, int startIndex, byte[] buffer)
    {
        Span<int> int32s = stackalloc int[4];
        decimal.GetBits(value, int32s);

        var bufferSpan = buffer.AsSpan();
        for (int i = 0; i < 4; i++)
        {
            // These slices are bigger than we need, but this is the simplest approach.
            var slice = bufferSpan.Slice(startIndex + i * 4);
            if (!BitConverter.TryWriteBytes(slice, int32s[i]))
            {
                throw new ArgumentException("Not enough space in span");
            }
        }
    }

    public static decimal Read(int startIndex, byte[] buffer)
    {
        Span<int> int32s = stackalloc int[4];
        ReadOnlySpan<byte> bufferSpan = buffer.AsSpan();
        for (int i = 0; i < 4; i++)
        {
            var slice = bufferSpan.Slice(startIndex + i * 4);
            int32s[i] = BitConverter.ToInt32(slice);
        }
        return new decimal(int32s);
    }

    static void Main()
    {
        byte[] bytes = new byte[16];
        decimal original = 1234.567m;
        Copy(original, 0, bytes);
        decimal restored = Read(0, bytes);
        Console.WriteLine(restored);
    }
}

---

or the same thing using MemoryMarshal:

public static void Copy(decimal value, int startIndex, byte[] buffer)
    => decimal.GetBits(value, MemoryMarshal.Cast<byte, int>(buffer.AsSpan(startIndex)));

public static decimal Read(int startIndex, byte[] buffer)
    => new decimal(MemoryMarshal.Cast<byte, int>(buffer.AsSpan(startIndex)));