Protobuf-net Object graph reference integrity

Question

There's a neat option in protobuf-net ver 2 called [ProtoMember(2,AsReference=true)]. For the most part, this is a follow up question to:

• protobuf-net serializing object graph

I started wondering if reference integrity is always maintained irrespective of when serialization / deserialization occurs. Is protobuf-net already doing this when you use the AsReference option?

I threw together a basic code example for illustration purposes, and then thought, "perhaps I need to borrow some ideas from the ORM world?" Should I be implementing an identity map? Should I somehow tell protobuf (via a delegates?) to resolve references to/from foreign key values instead.

The answer I'd like to hear is that somehow protobuf-net can maintain reference integrity across assembly boundaries, even with types that just look like each other.

However, here's an alternative sequence just-in-case:

• a => 1. resolve a reference to a primary key int, 2. serialize

• b => 3. deserialize, 4. resolve primary key int to a reference

Notes/Constraints:

• classes are a mirror image of each other, but are recompiled in each assembly.

• the object graph needs to be the same irrespective of when the objects gets serialized. i.e. A.ref=B (serialize / deserialize). C.ref=B (serialize / deserialize).

Example for discussion:

using System;
using System.Collections.Generic;
using ProtoBuf;

namespace protobuf_question
{
    class Program
    {
        static void Main(string[] args)
        {
            var a = new A() { key = 1 };
            var b = new B() { A = a, key = 2 };
        }
    }

    [ProtoContract]
    public class A
    {
        [ProtoMember(1)]
        public int key { get; set; }
    }
    [ProtoContract]
    public class B
    {
        [ProtoMember(1)]
        public int key { get; set; }
        [ProtoMember(2,AsReference=true)]
        public A A { get; set; }  // a reference
    }
    [ProtoContract]
    public class IdentityMap<T,TKey>
    {        
        public static readonly IdentityMap<T,TKey> instance = new IdentityMap<T,TKey>();  // singleton
        private Dictionary<string, T> identitySpace { get; set; }
        public IEnumerable<string> GetIdentitySet (/* some criteria */)
        {            
            // used for serializing with reference safety between assemblies.
            throw new NotImplementedException();
        }
        public TKey GetKey(T reference)
        {
            // map object reference to identity map; return identity.
            throw new KeyNotFoundException();
        }
    }

}

Answer 1

The answer I'd like to hear is that somehow protobuf-net can maintain reference integrity across assembly boundaries, even with types that just look like each other.

Yes, it can. When using AsReference=true it internally generates an opaque key unrelated to anything else, and uses that on the wire; this key is based on the deterministic nature of the stream. Everything remains 100% contract based. As long as the contracts are compatible it doesn't matter whether it is the same type, same process, same machine, same OS, etc.

The only exception to this is when using the DynamicType=true option, where it burns type metadata into the wire, but even that is not required to be the actual type data - there is an event you can subscribe to if you want to provide something more fine-grained and controlled than the default (type.AssemblyQualifiedName). If you were using DynamicType, this would allow you to transparently swap types between implementations. Of course, if you aren't using DynamicType then this problem doesn't exist in the first place.

---

Re your identity map... I'm unclear on the motivation there. But to be explicit: the existing code does not attempt to identify any "primary key" candidates and serialize just those; it serializes the object the first time it sees it (and invents a key), otherwise it just writes the key it made up last time.