How to serialize large nested arrays with protobuf?

Question

I am using protobuf-net for binary serialization. I am gettig OutOfMemory while serializing Class A. The same object is well serialized with BinaryFormatter.

Below is class example:

[ProtoContract]
class A:
   [ProtoMember(1, DataFormat = DataFormat.Group)]
   B[] Array1 {get; set;}
   ....

class B:
   [ProtoMember(1)]
   string Field1 {get; set;}

   [ProtoMember(2)]
   string Field1 {get; set;}

   [ProtoMember(3, DataFormat = DataFormat.Group)]
   C[] Array2 {get; set;} // 20000 elements
   ....

class C:
   [ProtoMember(1)]
   string Field1 {get; set;}

   [ProtoMember(2)]
   string Field1 {get; set;}

Answer 1

Wow. Simply wow. Thanks for asking this question. There was a glitch where-by it was incorrectly not applying group-encoding in a few scenarios, yours included. For protobuf-net, this isn't a biggie since group and string are treated interchangeably, but this is still a bit of an embarrassing glitch, not least because "group" is (as you have used correctly) key in making things forwards-only, for serializing large graphs.

I have fixed this locally and in the source - however, I want to do a bit more stability testing before doing a formal release. If you are happy to build from source, it should work fine now - or I can email you a dll if you want.