Protobuf-net deserializes empty collection to null when the collection is a property of another type

Question

I've run in to a problem with protobuf-net and I hope it is user error and not a bug with protobuf-net.

I can serialize an empty collection (for example, IDictionary<string, string>()) and then deserialize the collection. The deserialization results in a non-null object (exactly as I serialized).

However, things get wonky if the collection belongs to another type. Serializing the custom type with a non-null empty collection results in a deserialization where the collection is null.

Below is an example of what I am working on and unit tests that illustrate the problem.

The test Test_Protobuf_EmptyDictionary_SerializeDeserialize passes. The test Test_Protobuf_EmptyCollectionAndPopulatedCollection_SerializeDeserialize fails.

[ProtoContract]
[ProtoInclude(100, typeof(ConcreteA))]
public abstract class AbstractClass
{
    [ProtoMember(1)]
    public string Name { get; set; }

    [ProtoMember(2)]
    public IDictionary<string, string> FieldsA { get; set; }

    [ProtoMember(3)]
    public IDictionary<string, string> FieldsB { get; set; }

    [ProtoMember(4)]
    public ICollection<string> FieldsC { get; set; }

    [ProtoMember(5)]
    public ICollection<string> FieldsD { get; set; } 
}

[ProtoContract]
[ProtoInclude(110, typeof(ConcreteB))]
public class ConcreteA : AbstractClass
{
    public ConcreteA() {}
}

[ProtoContract]
[ProtoInclude(120, typeof(ConcreteC))]
public class ConcreteB : ConcreteA
{
    [ProtoMember(1)]
    public int Age { get; set; }

    public ConcreteB() {}
}

[ProtoContract]
public class ConcreteC : ConcreteB
{
    [ProtoMember(1)]
    public string HairColor { get; set; }
}

[TestFixture]
public class ProtobufTests
{
    [Test]
    public void Test_Protobuf_EmptyDictionary_SerializeDeserialize()
    {
        IDictionary<string,string> dictionary = new Dictionary<string, string>();
        ICollection<string> collection = new List<string>();

        Assert.IsNotNull(dictionary);
        Assert.IsNotNull(collection);
        Assert.AreEqual(0, dictionary.Keys.Count);
        Assert.AreEqual(0, collection.Count);

        using (MemoryStream ms = new MemoryStream())
        {
            Serializer.Serialize(ms, dictionary);
            ms.Position = 0;
            var deserialized = Serializer.Deserialize<IDictionary<string, string>>(ms);

            Assert.IsNotNull(deserialized);
            Assert.AreEqual(0, deserialized.Keys.Count);
        }

        using (MemoryStream ms = new MemoryStream())
        {
            Serializer.Serialize(ms, collection);
            ms.Position = 0;
            var deserialized = Serializer.Deserialize<ICollection<string>>(ms);

            Assert.IsNotNull(deserialized);
            Assert.AreEqual(0, deserialized.Count);
        }
    }

    [Test]
    public void Test_Protobuf_EmptyCollectionAndPopulatedCollection_SerializeDeserialize()
    {
        ConcreteC c = new ConcreteC {
                                        FieldsA = new Dictionary<string, string>(),
                                        FieldsB = new Dictionary<string, string> {{"john", "elway"}},
                                        FieldsC = new List<string>(),
                                        FieldsD = new List<string>{"james", "jones"}
                                    };
        Assert.IsNotNull(c);
        Assert.IsNotNull(c.FieldsA);
        Assert.IsNotNull(c.FieldsB);
        Assert.IsNotNull(c.FieldsC);
        Assert.IsNotNull(c.FieldsD);
        Assert.AreEqual(0, c.FieldsA.Keys.Count);
        Assert.AreEqual(1, c.FieldsB.Keys.Count);
        Assert.AreEqual(0, c.FieldsC.Count);
        Assert.AreEqual(2, c.FieldsD.Count);

        using (MemoryStream ms = new MemoryStream())
        {
            Serializer.Serialize(ms, c);
            ms.Position = 0;
            var deserialized = Serializer.Deserialize<ConcreteC>(ms);

            Assert.IsNotNull(deserialized);
            Assert.IsNotNull(deserialized.FieldsA); // first failing test
            Assert.IsNotNull(deserialized.FieldsB);
            Assert.IsNotNull(deserialized.FieldsC);
            Assert.IsNotNull(deserialized.FieldsD);
            Assert.AreEqual(0, deserialized.FieldsA.Keys.Count);
            Assert.AreEqual(1, deserialized.FieldsB.Keys.Count);
            Assert.AreEqual(0, deserialized.FieldsC.Count);
            Assert.AreEqual(1, deserialized.FieldsD.Count);
        }
    }
}

Answer 1

Ultimately, this comes down to the fact that the protobuf specification has no concept of null, and no way of expressing null. A collection is really just a repeated block (from the .proto specification); and a repeated block of 0 items is nothing whatsoever; no bytes; nothing. Since the collection member itself does not appear in protobuf, there is nowhere to say whether that thing is null vs not-null.

In xml terms, if is like using [XmlElement] to embed child items inside a parent (rather than [XmlArray] / [XmlArrayItem] - i.e.

<Foo>
    <Name>abc</Name>
    <Item>x</Item>
    <Item>y</Item>
    <Item>z</Item>
</Foo>

Here, a Foo with 0 Items would be:

<Foo>
    <Name>abc</Name>
</Foo>

from which, it is impossible to say whether the collection itself was null vs empty. Obviously protobuf isn't actually xml, but the above is meant purely as an illustrative example.

So: protobuf has no way to express this scenario, and therefore neither does protobuf-net.

In the other scenario you represent: the serialized representation of an empty list (as the root element) is: zero bytes. When you deserialize, if it finds the stream is empty, it always returns a non-null value as the root object, that being the most likely version of what you wanted.

Answer 2

make sure you have a public default constructor so protobuf-net can give you an empty object.