Logo Questions Linux Laravel Mysql Ubuntu Git Menu
HTML CSS JAVASCRIPT SQL PYTHON PHP BOOTSTRAP JAVA JQUERY R React Kotlin
 

How to (de)serialize a XmlException with Newtonsoft JSON?

Tags:

json

c#

json.net

This sample code:

var json = JsonConvert.SerializeObject(new XmlException("bla"));
var exception = JsonConvert.DeserializeObject<XmlException>(json);

throws an InvalidCastException in Newtonsoft.Json.dll: Unable to cast object of type 'Newtonsoft.Json.Linq.JValue' to type 'System.String' with the following stack trace:

at System.Xml.XmlException..ctor(SerializationInfo info, StreamingContext context)
at Void .ctor(System.Runtime.Serialization.SerializationInfo, System.Runtime.Serialization.StreamingContext)(Object[] )
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateISerializable(JsonReader reader, JsonISerializableContract contract, JsonProperty member, String id)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent)
at Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType)
at Newtonsoft.Json.JsonConvert.DeserializeObject(String value, Type type, JsonSerializerSettings settings)
at Newtonsoft.Json.JsonConvert.DeserializeObject[T](String value, JsonSerializerSettings settings)
at Newtonsoft.Json.JsonConvert.DeserializeObject[T](String value)
at TestJson.Program.Main(String[] args) in C:\Projects\TestJson\TestJson\Program.cs:line 21
at System.AppDomain._nExecuteAssembly(RuntimeAssembly assembly, String[] args)
at System.AppDomain.ExecuteAssembly(String assemblyFile, Evidence assemblySecurity, String[] args)
at Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()
at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading.ThreadHelper.ThreadStart()

Am I missing anything?

An issue has been created at https://github.com/JamesNK/Newtonsoft.Json/issues/801

like image 450
Fabian Vilers Avatar asked Jan 26 '16 13:01

Fabian Vilers

1 Answers

The Problem

The basic problem here is an incompatibility between JSON, which is weakly typed, and ISerializabe + SerializationInfo which were originally designed to work with BinaryFormatter whose streams are strongly typed. I.e. implementations of ISerializable sometimes expect that the serialization stream contains the complete type information for serialized fields. And it turns out that XmlException has one such implementation.

The specifics are as follows. When Json.NET goes to call the serialization constructor for an ISerializable type, it constructs a SerializationInfo and passes a JsonFormatterConverter that should handle the job of converting from JSON data to the required type when SerializationInfo.GetValue(String, Type) is called. Now, this method throws an exception when the named value is not found. And, unfortunately, there is no SerializationInfo.TryGetValue() method, requiring classes that need to deserialize optional fields to loop through the properties manually with GetEnumerator(). But in addition, there is no method to retrieve the converter set in the constructor, meaning that optional fields cannot be converted when required so must needs have been deserialized with precisely the expected type.

You can see this in the reference source for the constructor of XmlException:

    protected XmlException(SerializationInfo info, StreamingContext context) : base(info, context) {
        res                 = (string)  info.GetValue("res"  , typeof(string));
        args                = (string[])info.GetValue("args", typeof(string[]));
        lineNumber          = (int)     info.GetValue("lineNumber", typeof(int));
        linePosition        = (int)     info.GetValue("linePosition", typeof(int));

        // deserialize optional members
        sourceUri = string.Empty;
        string version = null;
        foreach ( SerializationEntry e in info ) {
            switch ( e.Name ) {
                case "sourceUri":
                    sourceUri = (string)e.Value;
                    break;
                case "version":
                    version = (string)e.Value;
                    break;
            }
        }

It turns out that e.Value is still a JValue not a string at this point, so deserialization chokes.

Json.NET could fix this specific problem by, in JsonSerializerInternalReader.CreateISerializable(), replacing string-valued JValue tokens with actual strings when constructing its SerializationInfo, then later re-converting to JValue in JsonFormatterConverter if conversion is necessary. However, this would not fix this category of problems. For instance, when an int is round-tripped by Json.NET it becomes a long, which will throw if cast without conversion. And of course a DateTime field will throw without conversion. It would also be a breaking change in that ISerializable classes that had previously been hand-crafted to work with Json.NET could break.

You might report an issue about this, but I'm skeptical it will get fixed any time soon.

A more robust approach to solving the problem would be to create a custom JsonConverter that embeds complete type information for ISerializable types.

Solution 1: Embed Binary

The first, simplest solution would be to embed a BinaryFormatter stream inside your JSON. The serialization code for the Exception classes was originally designed to be compatible with BinaryFormatter, so this should be fairly reliable:

public class BinaryConverter<T> : JsonConverter where T : ISerializable
{
    class BinaryData
    {
        public byte[] binaryData { get; set; }
    }

    public override bool CanConvert(Type objectType)
    {
        return typeof(T).IsAssignableFrom(objectType);
    }

    public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
    {
        if (reader.TokenType == JsonToken.Null)
            return null;
        var data = serializer.Deserialize<BinaryData>(reader);
        if (data == null || data.binaryData == null)
            return null;
        return BinaryFormatterHelper.FromByteArray<T>(data.binaryData);

    }

    public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
    {
        var data = new BinaryData { binaryData = BinaryFormatterHelper.ToByteArray(value) };
        serializer.Serialize(writer, data);
    }
}

public static class BinaryFormatterHelper
{
    public static byte [] ToByteArray<T>(T obj)
    {
        using (var stream = new MemoryStream())
        {
            new BinaryFormatter().Serialize(stream, obj);
            return stream.ToArray();
        }
    }

    public static T FromByteArray<T>(byte[] data)
    {
        return FromByteArray<T>(data, null);
    }

    public static T FromByteArray<T>(byte[] data, BinaryFormatter formatter)
    {
        using (var stream = new MemoryStream(data))
        {
            formatter = (formatter ?? new BinaryFormatter());
            var obj = formatter.Deserialize(stream);
            if (obj is T)
                return (T)obj;
            return default(T);
        }
    }
}

And then serialize with the following settings:

var settings = new JsonSerializerSettings { Converters =  new[] { new BinaryConverter<Exception>() } };

The drawbacks are:

  1. There is a severe security hazard deserializing untrusted data. Since the type information is completely embedded inside the proprietary, unreadable serialization stream, you cannot know what you are going to construct until you have done so.

  2. The JSON is completely unreadable.

  3. I believe BinaryFormatter is missing on some .Net versions.

  4. I believe BinaryFormatter can only be used in full trust.

But if all you are trying to do is to serialize an exception between processes under your control, this might be good enough.

Solution 2: Embed type information using TypeNameHandling.

Json.NET also has the optional ability to embed .NET type information for non-primitive types in a serialization stream, by setting JsonSerializer.TypeNameHandling to an appropriate value. Using this ability along with wrappers for primitive types, it's possible to create a JsonConverter that encapsulates SerializationInfo and SerializationEntry and contains all known type information:

public class ISerializableConverter<T> : JsonConverter where T : ISerializable
{
    public override bool CanConvert(Type objectType)
    {
        return typeof(T).IsAssignableFrom(objectType);
    }

    public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
    {
        if (reader.TokenType == JsonToken.Null)
            return null;
        var oldTypeNameHandling = serializer.TypeNameHandling;
        var oldAssemblyFormat = serializer.TypeNameAssemblyFormat;
        try
        {
            if (serializer.TypeNameHandling == TypeNameHandling.None)
                serializer.TypeNameHandling = TypeNameHandling.Auto;
            else if (serializer.TypeNameHandling == TypeNameHandling.Arrays)
                serializer.TypeNameHandling = TypeNameHandling.All;
            var data = serializer.Deserialize<SerializableData>(reader);
            var type = data.ObjectType;
            var info = new SerializationInfo(type, new FormatterConverter());
            foreach (var item in data.Values)
                info.AddValue(item.Key, item.Value.ObjectValue, item.Value.ObjectType);
            var value = Activator.CreateInstance(type, BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance, null, new object[] { info, serializer.Context }, serializer.Culture);
            if (value is IObjectReference)
                value = ((IObjectReference)value).GetRealObject(serializer.Context);
            return value;
        }
        finally
        {
            serializer.TypeNameHandling = oldTypeNameHandling;
            serializer.TypeNameAssemblyFormat = oldAssemblyFormat;
        }
    }

    public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
    {
        var oldTypeNameHandling = serializer.TypeNameHandling;
        var oldAssemblyFormat = serializer.TypeNameAssemblyFormat;
        try
        {
            var serializable = (ISerializable)value;
            var context = serializer.Context;
            var info = new SerializationInfo(value.GetType(), new FormatterConverter());
            serializable.GetObjectData(info, context);
            var data = SerializableData.CreateData(info, value.GetType());

            if (serializer.TypeNameHandling == TypeNameHandling.None)
                serializer.TypeNameHandling = TypeNameHandling.Auto;
            else if (serializer.TypeNameHandling == TypeNameHandling.Arrays)
                serializer.TypeNameHandling = TypeNameHandling.All;
            // The following seems to be required by https://github.com/JamesNK/Newtonsoft.Json/issues/787
            serializer.TypeNameAssemblyFormat = System.Runtime.Serialization.Formatters.FormatterAssemblyStyle.Full;
            serializer.Serialize(writer, data, typeof(SerializableData));
        }
        finally
        {
            serializer.TypeNameHandling = oldTypeNameHandling;
            serializer.TypeNameAssemblyFormat = oldAssemblyFormat;
        }
    }
}

abstract class SerializableValue
{
    [JsonIgnore]
    public abstract object ObjectValue { get; }

    [JsonIgnore]
    public abstract Type ObjectType { get; }

    public static SerializableValue CreateValue(SerializationEntry entry)
    {
        return CreateValue(entry.ObjectType, entry.Value);
    }

    public static SerializableValue CreateValue(Type type, object value)
    {
        if (value == null)
        {
            if (type == null)
                throw new ArgumentException("type and value are both null");
            return (SerializableValue)Activator.CreateInstance(typeof(SerializableValue<>).MakeGenericType(type));
        }
        else
        {
            type = value.GetType(); // Use most derived type
            return (SerializableValue)Activator.CreateInstance(typeof(SerializableValue<>).MakeGenericType(type), value);
        }
    }
}

sealed class SerializableValue<T> : SerializableValue
{
    public SerializableValue() : base() { }

    public SerializableValue(T value)
        : base()
    {
        this.Value = value;
    }

    public override object ObjectValue { get { return Value; } }

    public override Type ObjectType { get { return typeof(T); } }

    [JsonProperty("value", NullValueHandling = NullValueHandling.Ignore)]
    public T Value { get; private set; }
}

abstract class SerializableData
{
    public SerializableData()
    {
        this.Values = new Dictionary<string, SerializableValue>();
    }

    public SerializableData(IEnumerable<SerializationEntry> values)
    {
        this.Values = values.ToDictionary(v => v.Name, v => SerializableValue.CreateValue(v));
    }

    [JsonProperty("values", ItemTypeNameHandling = TypeNameHandling.Auto)]
    public Dictionary<string, SerializableValue> Values { get; private set; }

    [JsonIgnore]
    public abstract Type ObjectType { get; }

    public static SerializableData CreateData(SerializationInfo info, Type initialType)
    {
        if (info == null)
            throw new ArgumentNullException("info");
        var type = info.GetSavedType(initialType);
        if (type == null)
            throw new InvalidOperationException("type == null");
        return (SerializableData)Activator.CreateInstance(typeof(SerializableData<>).MakeGenericType(type), info.AsEnumerable());
    }
}

sealed class SerializableData<T> : SerializableData
{
    public SerializableData() : base() { }

    public SerializableData(IEnumerable<SerializationEntry> values) : base(values) { }

    public override Type ObjectType { get { return typeof(T); } }
}

public static class SerializationInfoExtensions
{
    public static IEnumerable<SerializationEntry> AsEnumerable(this SerializationInfo info)
    {
        if (info == null)
            throw new NullReferenceException();
        var enumerator = info.GetEnumerator();
        while (enumerator.MoveNext())
        {
            yield return enumerator.Current;
        }
    }

    public static Type GetSavedType(this SerializationInfo info, Type initialType)
    {
        if (initialType != null)
        {
            if (info.FullTypeName == initialType.FullName
                && info.AssemblyName == initialType.Module.Assembly.FullName)
                return initialType;
        }
        var assembly = Assembly.Load(info.AssemblyName);
        if (assembly != null)
        {
            var type = assembly.GetType(info.FullTypeName);
            if (type != null)
                return type;
        }
        return initialType;
    }
}

And then use the following settings:

This produces semi-readable JSON like the following:

{
  "$type": "Question35015357.SerializableData`1[[System.Xml.XmlException, System.Xml, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]], Tile, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null",
  "values": {
    "ClassName": {
      "$type": "Question35015357.SerializableValue`1[[System.String, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]], Tile, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null",
      "value": "System.Xml.XmlException"
    },
    "Message": {
      "$type": "Question35015357.SerializableValue`1[[System.String, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]], Tile, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null",
      "value": "bla"
    },
    "Data": {
      "$type": "Question35015357.SerializableValue`1[[System.Collections.IDictionary, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]], Tile, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"
    },
    "InnerException": {
      "$type": "Question35015357.SerializableValue`1[[System.Exception, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]], Tile, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"
    },
    "HelpURL": {
      "$type": "Question35015357.SerializableValue`1[[System.String, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]], Tile, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"
    },
    "StackTraceString": {
      "$type": "Question35015357.SerializableValue`1[[System.String, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]], Tile, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"
    },
    "RemoteStackTraceString": {
      "$type": "Question35015357.SerializableValue`1[[System.String, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]], Tile, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"
    },
    "RemoteStackIndex": {
      "$type": "Question35015357.SerializableValue`1[[System.Int32, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]], Tile, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null",
      "value": 0
    },
    "ExceptionMethod": {
      "$type": "Question35015357.SerializableValue`1[[System.String, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]], Tile, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"
    },
    "HResult": {
      "$type": "Question35015357.SerializableValue`1[[System.Int32, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]], Tile, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null",
      "value": -2146232000
    },
    "Source": {
      "$type": "Question35015357.SerializableValue`1[[System.String, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]], Tile, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"
    },
    "res": {
      "$type": "Question35015357.SerializableValue`1[[System.String, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]], Tile, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null",
      "value": "Xml_UserException"
    },
    "args": {
      "$type": "Question35015357.SerializableValue`1[[System.String[], mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]], Tile, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null",
      "value": [
        "bla"
      ]
    },
    "lineNumber": {
      "$type": "Question35015357.SerializableValue`1[[System.Int32, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]], Tile, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null",
      "value": 0
    },
    "linePosition": {
      "$type": "Question35015357.SerializableValue`1[[System.Int32, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]], Tile, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null",
      "value": 0
    },
    "sourceUri": {
      "$type": "Question35015357.SerializableValue`1[[System.Object, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]], Tile, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"
    },
    "version": {
      "$type": "Question35015357.SerializableValue`1[[System.String, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]], Tile, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null",
      "value": "2.0"
    }
  }
}

As you can see, the security hazard is somewhat mitigated by the readability of the JSON. You could also create a custom SerializationBinder to further reduce the security hazard loading only expected types as explained in TypeNameHandling caution in Newtonsoft Json.

I'm not sure what should be done in partial trust situations. JsonSerializerInternalReader.CreateISerializable() throws in partial trust:

    private object CreateISerializable(JsonReader reader, JsonISerializableContract contract, JsonProperty member, string id)
    {
        Type objectType = contract.UnderlyingType;

        if (!JsonTypeReflector.FullyTrusted)
        {
            string message = @"Type '{0}' implements ISerializable but cannot be deserialized using the ISerializable interface because the current application is not fully trusted and ISerializable can expose secure data." + Environment.NewLine +
                             @"To fix this error either change the environment to be fully trusted, change the application to not deserialize the type, add JsonObjectAttribute to the type or change the JsonSerializer setting ContractResolver to use a new DefaultContractResolver with IgnoreSerializableInterface set to true." + Environment.NewLine;
            message = message.FormatWith(CultureInfo.InvariantCulture, objectType);

            throw JsonSerializationException.Create(reader, message);
        }

So perhaps the converter should as well.

like image 124
dbc Avatar answered Oct 23 '22 02:10

dbc
Sign in to Comment

Related questions

azure service bus not receiving all messages (only ~65%)
Failed to Add a New Scaffolded Item from a different Project into my Web Project (MVC 5)
Can't assign methods that return value types to Func<dynamic> [duplicate]
c# custom attribute for a method parameter - how it works?
In AutoMapper can you apply the same value resolver to multiple members
Programmatically set filename and path in Microsoft Print to PDF printer
Can Entity Framework generate an UPDATE ... WHERE statement in SQL?
Visual Studio "Pin is required to establish a connection"
Import Project to Immediate
Why do I need to change the Binding Source Position before I can SaveChanges
How can I store List<T> in local settings?
Linq SelectMany include parent
Strongly typed url action
Color tiles in Windows 10 desktop apps
Visual Studio Protocol Handler - Open File
How to call a c# web service in java?
Why Doesn't Visual Studio's Designer Match What I See At Runtime?
WPF - Multiline TextBox text gets blurry while typing a long text
How can I find the last character of a textbox text?
IIS Mixed Anonymous and Windows Authentication

Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!