ProtoBuf corrupts byte array during deserialization (extra 0's added)

Question

I'm using ProtoBuf.NET to serialize/deserialize some classes. I'm finding that on deserializing, I'm getting a corrupt byte[] (extra 0's). Before you ask, yes, I need the *WithLengthPrefix() versions of the ProtoBuf API since the ProtoBuf portion is at the starts of a custom stream :)

Anyway, I see

Original object is (JSON depiction):
{"ByteArray":"M+B6q+PXNuF8P5hl","ByteArray2":"DmErxVQ2y87IypSRcfxcWA==","K":2,"V
":1.0}

Protobuf: Raw Hex (42 bytes):
29-2A-20-0A-0C-33-E0-7A-AB-E3-D7-36-E1-7C-3F-98-65-12-10-0E-61-2B-C5-54-36-CB-CE
-C8-CA-94-91-71-FC-5C-58-08-02-15-00-00-80-3F

Regenerated object is (JSON depiction):
{"ByteArray":"AAAAAAAAAAAAAAAAM+B6q+PXNuF8P5hl","ByteArray2":"DmErxVQ2y87IypSRcf
xcWA==","K":2,"V":1.0}

The extra AAA*A in the ByteArray member are basically hex 0x00's in base64.

The app logic is similar to

static void Main(string[] args)
{
    var parent = new Parent();
    parent.Init();

    Console.WriteLine("\nOriginal object is (JSON depiction):");
    Console.WriteLine(JsonConvert.SerializeObject(parent));

    using (var ms = new MemoryStream())
    {
        Serializer.SerializeWithLengthPrefix(ms, parent, PrefixStyle.Base128);
        byte[] bytes2 = ms.ToArray();
        var hex2 = BitConverter.ToString(bytes2);
        Console.WriteLine("\nProtobuf: Hex ({0} bytes):\n{1}", bytes2.Length, hex2);

        ms.Seek(0, SeekOrigin.Begin);
        var backFirst = Serializer.DeserializeWithLengthPrefix<Parent>(ms,PrefixStyle.Base128);

        Console.WriteLine("\nRegenerated object is (JSON depiction):");
        Console.WriteLine(JsonConvert.SerializeObject(backFirst));
    }
}

The DTO classes are

[DataContract]
[ProtoContract]
internal class Parent : Child
{
    [DataMember(Name = "ByteArray", Order = 10)]
    [ProtoMember(1)]
    public byte[] ByteArray { get; set; }

    [DataMember(Name = "ByteArray2", Order = 30, EmitDefaultValue = false)]
    [ProtoMember(2)]
    public byte[] ByteArray2 { get; set; }

    public Parent()
    {
        ByteArray = new byte[12];
    }

    internal void Init(bool bindRow = false)
    {
        base.Init();
        var rng = new RNGCryptoServiceProvider();
        rng.GetBytes(ByteArray);

        ByteArray2 = new byte[16];
        rng.GetBytes(ByteArray2);
    }
}

[DataContract]
[ProtoContract]
[ProtoInclude(5, typeof(Parent))]
public class Child
{
    [DataMember(Name = "K", Order = 100)]
    [ProtoMember(1)]
    public Int32 K { get; set; }

    [DataMember(Name = "V", Order = 110)]
    [ProtoMember(2)]
    public float V { get; set; }

    internal void Init()
    {
        K = 2;
        V = 1.0f;
    }
}

I do see that when I move ByteArray = new byte[12] out form the Parent constructor into it's Init() method, ProtoBuf works fine. However we have app logic that prevents that in the real version (vs the SO trim down code you see above).

Are we doing something wrong or is this a bug in ProtoBuf?

Answer 1

Here we go:

public Parent()
{
    ByteArray = new byte[12];
}

Note: protobuf is designed (by google) to be both appendable and mergeable. Where append / merge is synonymous (for lists / arrays etc) with "append".

Two options (both possible via attributes):

• disable the constructor: [ProtoContract(SkipConstructor = true)]
• disable the append: [ProtoMember(1, OverwriteList = true)]

There are other options too, but those are the ones I'd lean towards.

You remark that the array initialisation is different in the real code, but: I can't comment on code I can't see.