protobuf-net: Possible recursion detected

Question

I get an exception trying to serialize object graph (not very deep). Meaningful part of it is like this:

[ERROR] FATAL UNHANDLED EXCEPTION: ProtoBuf.ProtoException: Possible recursion d etected (offset: 5 level(s)): red at ProtoBuf.ProtoWriter.CheckRecursionStackAndPush (object) <0x00127> at ProtoBuf.ProtoWriter.StartSubItem (object,ProtoBuf.ProtoWriter,bool) <0x0002f>

The graph represents file/directory structure and my model (simplified) looks like this:

[ProtoContract] 
[ProtoInclude(100, typeof(PackageDirectory))]
[ProtoInclude(200, typeof(PackageFile))]
public abstract class PackageMember
{
   [ProtoMember(1)] 
   public virtual string Name { get; protected set; }

   [ProtoMember(2, AsReference=true)] 
   public PackageDirectory ParentDirectory { get; protected set; }  
}

[ProtoContract]
public class PackageDirectory : PackageMember
{
   [ProtoMember(3)]
   private Dictionary<string, PackageMember> _children;

   public PackageDirectory()
   {
      _children = new Dictionary<string, PackageMember>();
   }

   public PackageDirectory (string name, PackageDirectory parentDirectory)
      : this()
   {
      this.ParentDirectory = parentDirectory;
      this.Name = name;         
   }

   public void Add (PackageMember member)
   {
      _children.Add(member.Name, member);
   }
}

[ProtoContract]
public class PackageFile : PackageMember
{
   private Stream _file;
   private BinaryReader _reader;

   private PackageFile() 
   {}

   public PackageFile (string name, int offset, int length, PackageDirectory directory,  Stream file)
   {
      this.Name = name;
      this.Length = length;
      this.Offset = offset;
      this.ParentDirectory = directory;

      _file = file;
      _reader = new BinaryReader(_file);
   }

   [OnDeserialized]
   protected virtual void OnDeserialized(SerializationContext context)
   {
      var deserializationContext = context.Context as DeserializationContext;

      if (deserializationContext != null)
      { 
         _file = deserializationContext.FileStream;
         _reader = new BinaryReader(_file);
      }
   }

   [ProtoMember(3)]
   public int Offset { get; private set; }

   [ProtoMember(4)]
   public int Length { get; private set; }
}

The depth of this tree is near 10-15 levels, which less than ProtoBuf.ProtoWriter.RecursionCheckDepth value (25). (So maybe this is a bug?) The version of protobuf-net used is one compiled from trunk v2 (rev 491).

Actually, i solved it with modification of protobuf-net code. I changed value of ProtoBuf.ProtoWriter.RecursionCheckDepth to 100 and everything seems to be ok.

The question is if there any "true" way to serialize such kind of graph without modification of protobuf code? Is such a behavior correct or it's a bug?

My platform is Mono-2.10-8 on Windows 7 Professional 64-bit

P.S. Also i found that if i deserizlie with thw following code, i should have PackageDirectory parameterless constructor to be public.

var value = new PackageDirectory();
RuntimeTypeModel.Default.Deserialize(ms, value, typeof(PackageDirectory), new SerializationContext {
   Context = new DeserializationContext {
   FileStream = _file,
}});

It's another topic but it's well illustrated with presented code. I think that in this case declaring private constructor should be allowed because now the behavior differs from one for Serializer.Deserialize(...).

Answer 1

This exception is thrown only when the same reference is seen in the data (twice in the same path), and tracking is only enabled when the depth is at least RecursionCheckDepth. Which immediately makes me suspicious of the 10-15 depth limit cited, although it is not necessarily the case that protobuf handles levels quite the same as you are counting. It does not make sense to me that raising this number to 100 should make it work - in fact, the very existence of this RecursionCheckDepth is purely an optimisation to limit the effort involved in "typical" graphs, only enabling the more rigorous checking if it starts to look deep.

I am, however, mindful that this might also suggest some subtle bug in inheritance-based handling, perhaps also related to AsReference. I use protobuf-net extensively and constantly, and I haven't seen such an issue. If you have a reproducible repro I would very much like to see it.