Writing packed structs with a MemoryMappedViewAccessor bug?

Question

I have a struct that is defined with Pack=1 and it is 29 bytes long. If it is not packed, it would be 32 bytes long.

• Marshal.SizeOf(TypeOf(StructName)) returns 29.

• StructName struct; sizeof(struct) returns 32.

When I write that struct out using MemoryMappedViewAccessor it writes out 32 bytes, NOT 29 bytes.

So, short of marshalling the struct to a byte array and writing it out that way, is there any way to get it to write out that struct correctly?

More detail: if you use Explicit layout, Write will, in fact, write out 29 bytes. WriteArray, though, writes out 32 bytes for each element.

And avip, yes, meticulous byte serializing will probably work, but (and I didn't profile it but I am guessing) it is probably orders of magnitude slower than a WriteArray would be, no?

Answer 1

Edit: OK I finally understood what you're really asking. We usually don'y use MemoryMappedViewAccessor to serialize objects, and now you know why.

The following will give you the expected result.

public static class ByteSerializer
{
    public static Byte[] Serialize<T>(IEnumerable<T> msg) where T : struct
    {
        List<byte> res = new List<byte>();
        foreach (var s in msg)
        {
            res.AddRange(Serialize(s));
        }
        return res.ToArray();
    }

    public static Byte[] Serialize<T>(T msg) where T : struct
    {
        int objsize = Marshal.SizeOf(typeof(T));
        Byte[] ret = new Byte[objsize];

        IntPtr buff = Marshal.AllocHGlobal(objsize);
        Marshal.StructureToPtr(msg, buff, true);
        Marshal.Copy(buff, ret, 0, objsize);
        Marshal.FreeHGlobal(buff);
        return ret;
    }
}

class Program
{
    [StructLayout(LayoutKind.Sequential, Pack = 1)]
    struct Yours
    {
        public Int64 int1;
        public DateTime dt1;
        public float f1;
        public float f2;
        public float f3;
        public byte b;
    }

    static void Main()
    {
        var file = @"c:\temp\test.bin";
        IEnumerable<Yours> t = new Yours[3];
        File.WriteAllBytes(file, ByteSerializer.Serialize(t));

        using (var stream = File.OpenRead(file))
        {
            Console.WriteLine("file size: " + stream.Length);
        }
    }
}

EDIT: So it appears DateTime really likes to be on an aligned memory address. Though you can define the explicit layout, I think a more simple approach would be:

[StructLayout(LayoutKind.Sequential, Pack = 1)]
public struct Test
{
    private long dt1; 
    public byte b;
    public Int64 int1;
    public float f1;
    public float f2;
    public float f3;

    public DateTime DT
    {
        get { return new DateTime(dt1); }
        set { dt1 = value.Ticks; }
    }
}

Though I don't see why you should care about managed memory representation at all.

Alternatively, [StructLayout(LayoutKind.Explicit)] should prevent memory alignment.

Example (the 'managed sizeof' is taken from this post)

[StructLayout(LayoutKind.Explicit, Size = 9)]
public struct Test
{
    [FieldOffset(0)]
    public DateTime dt1;
    [FieldOffset(8)]
    public byte b;
}

class Program
{
    static readonly Func<Type, uint> SizeOfType = (Func<Type, uint>)Delegate.CreateDelegate(typeof(Func<Type, uint>), typeof(Marshal).GetMethod("SizeOfType", BindingFlags.NonPublic | BindingFlags.Static));

    static void Main()
    {
        Test t = new Test() { dt1 = DateTime.MaxValue, b = 42 };
        Console.WriteLine("Managed size: " + SizeOfType(typeof(Test)));
        Console.WriteLine("Unmanaged size: " + Marshal.SizeOf(t));
        using (MemoryMappedFile file = MemoryMappedFile.CreateNew(null, 1))
        using (MemoryMappedViewAccessor accessor = file.CreateViewAccessor())
        {
            accessor.Write(0L, ref t);
            long pos = 0;

            for (int i = 0; i < 9; i++)
                Console.Write("|" + accessor.ReadByte(pos++));
            Console.Write("|\n");
        }
    }
}

Output:

Managed size: 9
Unmanaged size: 9
|255|63|55|244|117|40|202|43|42|   // managed memory layout is as expected

BTW, DateTime seems to break the sequential contract - but remember the contract is for the Marshaled memory map. There is no specification with regard to managed memory layout.

[StructLayout(LayoutKind.Sequential, Pack = 1, Size = 9)]
public struct Test
{
    public DateTime dt1;
    public byte b;
}

And the output of above code:

Managed size: 12
Unmanaged size: 9
|42|0|0|0|255|63|55|244|117|40|202|43|   // finally found those 3 missing bytes :-)