byte[] array to struct with variable length array

Question

I'm receiving an array of bytes from a socket and the structure of the bytes is simply a large char array of fixed width strings. In some cases, the last field is dynamic (instead of fixed length) and I'm trying to Marshal the bytes to a struct. I've read that the variable length char array needs to be IntPtr, but I haven't figured out how to Marshal it with the remaining bytes. I've also read in some articles that I might need a second structure, but still can't figure out how to Marshal it properly.

Here's one such site

What's the proper way to deal with variable length char arrays in structs?

The struct:

[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Ansi)]
public struct Header
{
    #region private member fields

    [MarshalAs(UnmanagedType.ByValArray, SizeConst = 16)]
    private char[] _f1;

    [MarshalAs(UnmanagedType.ByValArray, SizeConst = 16)]
    private char[] _f2;

    [MarshalAs(UnmanagedType.ByValArray, SizeConst = 16)]
    private char[] _f3;

    [MarshalAs(UnmanagedType.ByValArray, SizeConst = 16)]
    private char[] _f4;

    [MarshalAs(UnmanagedType.ByValArray, SizeConst = 16)]
    private char[] _f5;

    [MarshalAs(UnmanagedType.ByValArray, SizeConst = 16)]
    private char[] _f6;

    [MarshalAs(UnmanagedType.ByValArray, SizeConst = 4)]
    private char[] _f7;

    [MarshalAs(UnmanagedType.ByValArray, SizeConst = 12)]
    private char[] _f8;

    [MarshalAs(UnmanagedType.ByValArray, SizeConst = 32)]
    private char[] _f9;

    [MarshalAs(UnmanagedType.ByValArray, SizeConst = 16)]
    private char[] _f10;

    // how would this get filled with a char[] array from the byte array?
    public IntPtr VariableLengthData;

    #endregion
}

The function:

public static Header FromArray(byte[] array)
{
    IntPtr buff = IntPtr.Zero;

    try
    {
        int objsize = Marshal.SizeOf(typeof(Header));
        buff = Marshal.AllocHGlobal(objsize);
        Marshal.Copy(array, 0, buff, objsize);
        var result = (Header)Marshal.PtrToStructure(buff, typeof(HostHeader));

        // the variable length data needs to be filled in somehow
        // but also note that an extra 4 bytes was added to the size
        // of the struct with the IntPtr
        if(objsize < array.Length)
        {
            Marshal.Copy(array, array.Length - objsize, result.VariableLengthData, array.Length - objsize);
        }

        return result;
    }
    finally
    {
        if (buff != IntPtr.Zero)
        {
            Marshal.FreeHGlobal(buff);
            buff = IntPtr.Zero;
        }
    }
}

This works - but now Marshal.SizeOf(headerObj) says it's smaller than it really is when I attempt to convert it back to a byte[] array. Other than that, anything wrong with this solution?

[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Ansi)]
public struct Header
{
    [MarshalAs(UnmanagedType.ByValArray, SizeConst = 8)] 
    public char[] Field1;

    [MarshalAs(UnmanagedType.ByValArray, SizeConst = 8)] 
    public char[] Field2;

    [MarshalAs(UnmanagedType.ByValArray, SizeConst = 8)] 
    public char[] Field3;

    [MarshalAs(UnmanagedType.ByValArray, SizeConst = 1)]
    public char[] Field4;
}

public static Header DeserializeHeader(byte[] data)
{
    int objsize = Marshal.SizeOf(typeof(Header));
    IntPtr buff = Marshal.AllocHGlobal(objsize);
    Marshal.Copy(data, 0, buff, objsize);
    var header = (Header)Marshal.PtrToStructure(buff, typeof(Header));
    Marshal.FreeHGlobal(buff);

    // resize Field4 to hold all the remaining bytes
    if(objsize < data.Length)
    {
        header.Field4 = Encoding.ASCII.GetChars(data, objsize - header.Field4.Length, data.Length - objsize - header.Field4.Length);
    }
    return header;
}

Answer 1

Just don't try too hard to make your C# declarations match the packet format exactly. You'd favor this:

[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Ansi, Pack = 1)]
public class Header {
    //....
}

public class Packet {
    public Header Header;
    public byte[] VariableLengthData;
}

Now it becomes simple. You can marshal the header in one fell swoop and just copy whatever extra bytes are present:

static unsafe Packet FromArray(byte[] data) {
    var hdrsize = Marshal.SizeOf(typeof(Header));
    if (data.Length < hdrsize) throw new ArgumentException();
    Packet result = new Packet();
    // Marshal the header
    fixed (byte* pdata = &data[0]) {
        result.Header = (Header)Marshal.PtrToStructure(new IntPtr(pdata), typeof(Header)); 
    }
    // Copy the rest
    var varsize = data.Length - hdrsize;
    result.VariableLengthData = new byte[varsize];
    Array.Copy(data, hdrsize, result.VariableLengthData, 0, varsize);
    return result;
}

And modify the Packet class as you see fit, you probably want to add a bunch of properties to deal with the char[].