Difference between Marshal.SizeOf and sizeof, I just don't get it

Question

Until now I have just taken for granted that Marshal.SizeOf is the right way to compute the memory size of a blittable struct on the unmanaged heap (which seems to be the consensus here on SO and almost everywhere else on the web).

But after having read some cautions against Marshal.SizeOf (this article after "But there's a problem...") I tried it out and now I am completely confused:

public struct TestStruct
{
    public char x;
    public char y;
}

class Program
{
    public static unsafe void Main(string[] args)
    {
        TestStruct s;
        s.x = (char)0xABCD;
        s.y = (char)0x1234;

        // this results in size 4 (two Unicode characters)
        Console.WriteLine(sizeof(TestStruct));

        TestStruct* ps = &s;

        // shows how the struct is seen from the managed side... okay!      
        Console.WriteLine((int)s.x);
        Console.WriteLine((int)s.y);

        // shows the same as before (meaning that -> is based on 
        // the same memory layout as in the managed case?)... okay!
        Console.WriteLine((int)ps->x);
        Console.WriteLine((int)ps->y);

        // let's try the same on the unmanaged heap
        int marshalSize = Marshal.SizeOf(typeof(TestStruct));
        // this results in size 2 (two single byte characters)
        Console.WriteLine(marshalSize);

        TestStruct* ps2 = (TestStruct*)Marshal.AllocHGlobal(marshalSize);

        // hmmm, put to 16 bit numbers into only 2 allocated 
        // bytes, this must surely fail...
        ps2->x = (char)0xABCD;
        ps2->y = (char)0x1234;

        // huh??? same result as before, storing two 16bit values in 
        // only two bytes??? next will be a perpetuum mobile...
        // at least I'd expect an access violation
        Console.WriteLine((int)ps2->x);
        Console.WriteLine((int)ps2->y);

        Console.Write("Press any key to continue . . . ");
        Console.ReadKey(true);
    }
}

What's going wrong here? What memory layout does the field dereferencing operator '->' assume? Is '->' even the right operator for addressing unmanaged structs? Or is Marshal.SizeOf the wrong size operator for unmanaged structs?

I have found nothing that explains this in a language I understand. Except for "...struct layout is undiscoverable..." and "...in most cases..." wishy-washy kind of stuff.

Answer 1

The difference is: the sizeof operator takes a type name and tells you how many bytes of managed memory need to be allocated for an instance of that struct.This is not necessarily stack memory; structs are allocated off the heap when they are array elements, fields of a class, and so on. By contrast, Marshal.SizeOf takes either a type object or an instance of the type, and tells you how many bytes of unmanaged memory need to be allocated. These can be different for a variety of reasons. The name of the type gives you a clue: Marshal.SizeOf is intended to be used when marshaling a structure to unmanaged memory.

Another difference between the two is that the sizeof operator can only take the name of an unmanaged type; that is, a struct type whose fields are only integral types, Booleans, pointers and so on. (See the specification for an exact definition.) Marshal.SizeOf by contrast can take any class or struct type.

Answer 2

I think the one question you still don't have answered is what's going on in your particular situation:

&ps2->x
0x02ca4370  <------
    *&ps2->x: 0xabcd 'ꯍ'
&ps2->y
0x02ca4372  <-------
    *&ps2->y: 0x1234 'ሴ'

You are writing to and reading from (possibly) unallocated memory. Because of the memory area you're in, it's not detected.

This will reproduce the expected behavior (at least on my system, YMMV):

  TestStruct* ps2 = (TestStruct*)Marshal.AllocHGlobal(marshalSize*10000);

  // hmmm, put to 16 bit numbers into only 2 allocated 
  // bytes, this must surely fail...
  for (int i = 0; i < 10000; i++)
  {
    ps2->x = (char)0xABCD;
    ps2->y = (char)0x1234;
    ps2++;
  }

Answer 3

What memory layout does the field dereferencing operator '->' assume?

Whatever the CLI decides

Is '->' even the right operator for addressing unmanaged structs?

That is an ambiguous concept. There are structs in unmanaged memory accessed via the CLI: these follow CLI rules. And there are structs that are merely notional monikers for unmanaged code (perhaps C/C++) accessing the same memory. This follows the rules of that framework. Marshalling usually refers to P/Invoke, but that isn't necessarily applicable here.

Or is Marshal.SizeOf the wrong size operator for unmanaged structs?

I'd default to Unsafe.SizeOf<T>, which is essentially sizeof(T) - which is perfectly well-defined for the CLI/IL (including padding rules etc), but isn't possible in C#.