Using reflection to determine how a .Net type is layed out in memory

Question

I'm experimenting with optimizing parser combinators in C#. One possible optimization, when the serialized format matches the in-memory format, is to just do an (unsafe) memcpy of the data to be parsed over an instance or even many instances of the type.

I want to write code that determines if the in-memory format matches the serialized format, in order to dynamically determine if the optimization can be applied. (Obviously this is an unsafe optimization and might not work for a whole bunch of subtle reasons. I'm just experimenting, not planning to use this in production code.)

I use the attribute [StructLayout(LayoutKind.Sequential, Pack = 1)] to force no padding and to force the in-memory order to match declaration order. I check for that attribute with reflection, but really all this confirms is "no padding". I also need the order of the fields. (I would strongly prefer to not have to manually specified FieldOffset attributes for every field, since that would be very error prone.)

I assumed I could use the order of fields returned by GetFields, but the documentation explicitly calls out that the order is unspecified.

Given that I am forcing the order of fields with the StructLayout attribute, is there a way to reflect on that ordering?

edit I'm fine with the restriction that all of the fields must be blittable.

Answer 1

This is unnecessary if using LayoutKind.Sequential with blittable types

You don't need to use reflection or any other mechanism to find out the order of struct fields in memory, as long as all the fields are blittable.

The blittable fields for a struct declared with LayoutKind.Sequential will be in memory in the order in which the fields are declared. That's what LayoutKind.Sequential means!

From this documentation:

For blittable types, LayoutKind.Sequential controls both the layout in managed memory and the layout in unmanaged memory. For non-blittable types, it controls the layout when the class or structure is marshaled to unmanaged code, but does not control the layout in managed memory.

Note that this doesn't tell you how much padding each field is using. To find that out, see below.

To determine the field order when using LayoutKind.Auto, or the field offsets when using any layout

It's fairly easy to find the struct field offsets if you're happy to use unsafe code, and to not use reflection.

You just need to take the address of each field of the struct and calculate its offset from the start of the struct. Knowing the offsets of each field, you can calculate their order (and any padding bytes between them). To calculate the padding bytes used for the last field (if any) you will also need to get the total size of the struct using sizeof(StructType).

The following example works for 32-bit and 64-bit. Note that you don't need to use fixed keyword because the struct is already fixed due to it being on the stack (you'll get a compile error if you try to use fixed with it):

using System;
using System.Runtime.InteropServices;

namespace Demo
{
    [StructLayout(LayoutKind.Auto, Pack = 1)]

    public struct TestStruct
    {
        public int    I;
        public double D;
        public short  S;
        public byte   B;
        public long   L;
    }

    class Program
    {
        void run()
        {
            var t = new TestStruct();

            unsafe
            {
                IntPtr p  = new IntPtr(&t);
                IntPtr pI = new IntPtr(&t.I);
                IntPtr pD = new IntPtr(&t.D);
                IntPtr pS = new IntPtr(&t.S);
                IntPtr pB = new IntPtr(&t.B);
                IntPtr pL = new IntPtr(&t.L);

                Console.WriteLine("I offset = " + ptrDiff(p, pI));
                Console.WriteLine("D offset = " + ptrDiff(p, pD));
                Console.WriteLine("S offset = " + ptrDiff(p, pS));
                Console.WriteLine("B offset = " + ptrDiff(p, pB));
                Console.WriteLine("L offset = " + ptrDiff(p, pL));

                Console.WriteLine("Total struct size = " + sizeof(TestStruct));
            }
        }

        long ptrDiff(IntPtr p1, IntPtr p2)
        {
            return p2.ToInt64() - p1.ToInt64();
        }

        static void Main()
        {
            new Program().run();
        }
    }
}

To determine the field offsets when using LayoutKind.Sequential

If your struct uses LayoutKind.Sequential then you can use Marshal.OffsetOf() to get the offset directly, but this does not work with LayoutKind.Auto:

foreach (var field in typeof(TestStruct).GetFields())
{
    var offset = Marshal.OffsetOf(typeof (TestStruct), field.Name);
    Console.WriteLine("Offset of " + field.Name + " = " + offset);
}

This is clearly a better way to do it if you are using LayoutKind.Sequential since it doesn't require unsafe code, and it's much shorter - and you don't need to know the names of the fields in advance. As I said above, it is not needed to determine the order of the fields in memory - but this might be useful if you need to find out about how much padding is used.

Answer 2

As a reference for those who want to know the order and the kind of layout. For example if a type contains non-blittable types.

var fields = typeof(T).GetFields(BindingFlags.NonPublic | BindingFlags.Public | BindingFlags.Instance);
fields.SortByFieldOffset();

var isExplicit = typeof(T).IsExplicitLayout;
var isSequential = typeof(T).IsLayoutSequential;

It uses an extension method that I wrote:

    public static void SortByFieldOffset(this FieldInfo[] fields) {
        Array.Sort(fields, (a, b) => OffsetOf(a).CompareTo(OffsetOf(b)) );
    }

    private static int OffsetOf(FieldInfo field) {
        return Marshal.OffsetOf(field.DeclaringType, field.Name).ToInt32();
    }

MSDN contains useful info on IsLayoutSequential.