deserialize "long" string with protobuf for c# doesn't work properly for me

Question

I'm obviously doing something basic wrong but I can't figure it out and don't find documentation.

I'm experimenting with proto-buf for .NET by Marc Gravell, and trying to serialize and deserialize objects. Once an object contains a string which is "too long" (didn't try to pinpoint the size threshold but it's few hundred bytes) the string doesn't deserialize properly for me.

This is my code:

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.IO;
using System.Linq;
using System.Reflection;
using System.Text;
using ProtoBuf;

namespace ConsoleApplication1
{
public class Program
{
    [ProtoContract]
    public class test
    {
        [ProtoMember(1)]            
        public int i;
        [ProtoMember(2)]
        public string s1;
        [ProtoMember(3)]
        public string s2;
        [ProtoMember(4)]
        public char[] arrchars;
        [ProtoMember(5)]
        public Dictionary<int, string> Dict = new Dictionary<int, string>();

    }


    static void Main(string[] args)
    {
        test var1 = new test();
        var1.i = 10;
        var1.s1 = "Hello";
        var1.arrchars = new char[] {'A', 'B', 'C'};
        var1.Dict.Add(10, "ten");
        var1.Dict.Add(5, "five");
        var1.s2 = new String('X', 520);

        string s = PBSerializer.Serialize(typeof (test), var1);

        test var2 = null;
        PBSerializer.Deserialize(s, out var2);
    }


    public static class PBSerializer
    {
        public static string Serialize(Type objType, object obj)
        {
            MemoryStream stream = new MemoryStream();
            ProtoBuf.Serializer.Serialize(stream, obj);
            // ProtoBuf.Serializer.SerializeWithLengthPrefix(stream, obj, PrefixStyle.Fixed32, 1);
            stream.Flush();
            stream.Position = 0;
            StreamReader sr = new StreamReader(stream);
            string res = sr.ReadToEnd();
            stream.Dispose();
            sr.Dispose();
            return res;
        }

        public static void Deserialize(string serializedObj, out test obj)
        {
            MemoryStream stream = new MemoryStream(Encoding.ASCII.GetBytes(serializedObj));
            obj = ProtoBuf.Serializer.Deserialize<test>(stream);
            // obj = ProtoBuf.Serializer.DeserializeWithLengthPrefix<test>(stream, PrefixStyle.Fixed32, 1);
            stream.Dispose();
        }

    }

}
}

var2.s2 is not identical to var1.s2 - it has an extra character at the beginning of the string and truncates the majority of the end of the string. If however, I change the length of var1.s2 to a small number (say 52 instead of 520 chars) my problem goes away but I need to be able to serialize long strings. I assume it has to do with something I do wrong with the setting the PrefixStyle (?) or perhaps I'm not using the right encoding (?). However, trial and error didn't help me sort it out.

I'm using .NET 3.5 and tried it with versions 444 & 450 with the same result.

Thanks.

Answer 1

You're serializing binary data - but then trying to read it as if it were text. It's not - so don't do that.

If you have to turn arbitrary binary data into text, use Convert.ToBase64String and Convert.FromBase64String.

public static class PBSerializer
{
    public static string Serialize(Type objType, object obj)
    {
        using (MemoryStream stream = new MemoryStream())
        {
            ProtoBuf.Serializer.Serialize(stream, obj);
            return Convert.ToBase64String(stream.ToArray());
        }
    }

    // Ideally change this to use a return value instead of an out parameter...
    public static void Deserialize(string serializedObj, out test obj)
    {
        byte[] data = Convert.FromBase64String(serializedObj);
        using (MemoryStream stream = new MemoryStream(data))
        {
            obj = ProtoBuf.Serializer.Deserialize<test>(stream);
        }
    }