Explanation for ObjectCreationHandling using Newtonsoft JSON?

Question

I was tracking down a bug and I noticed that Newtonsoft JSON will append items to a List<> that's been initialized in the default constructor. I did a little more digging and discussed with some people on the C# chat and we noticed that this behavior doesn't apply to all other collection types.

https://dotnetfiddle.net/ikNyiT

using System;
using Newtonsoft.Json;
using System.Collections.Generic;
using System.Collections.ObjectModel;

public class TestClass
{
    public Collection<string> Collection = new Collection<string>(new [] { "ABC", "DEF" });
    public List<string> List = new List<string>(new [] { "ABC", "DEF" });
    public ReadOnlyCollection<string> ReadOnlyCollection = new ReadOnlyCollection<string>(new [] { "ABC", "DEF" });
}

public class Program
{
    public static void Main()
    {
        var serialized = @"{
            Collection: [ 'Goodbye', 'AOL' ],
            List: [ 'Goodbye', 'AOL' ],
            ReadOnlyCollection: [ 'Goodbye', 'AOL' ]
        }";


        var testObj = JsonConvert.DeserializeObject<TestClass>(serialized);

        Console.WriteLine("testObj.Collection: " + string.Join(",", testObj.Collection));
        Console.WriteLine("testObj.List: " + string.Join(",", testObj.List));
        Console.WriteLine("testObj.ReadOnlyCollection: " + string.Join(",", testObj.ReadOnlyCollection));
    }
}

Output:

testObj.Collection: ABC,DEF
testObj.List: ABC,DEF,Goodbye,AOL
testObj.ReadOnlyCollection: Goodbye,AOL

As you can see the Collection<> property is unaffected by deserialization, the List<> is appended to and ReadOnlyCollection<> is replaced. Is this intended behavior? What was the reasoning?

Answer 1

It basically boils down to type instantiation and the ObjectCreationHandling setting. There are three settings for ObjectCreationHandling

Auto 0 Reuse existing objects, create new objects when needed.
Reuse 1 Only reuse existing objects.
Replace 2 Always create new objects.

The default is auto (Line 44).

Auto is only overwritten after a series of checks which determine if the current type has a TypeInitializer which is null. At that point it checks if there is a parameterless constructor.

///
/// Create a factory function that can be used to create instances of a JsonConverter described by the
/// argument type.
/// The returned function can then be used to either invoke the converter's default ctor, or any
/// parameterized constructors by way of an object array.
///

Essentially it acts like this (what it looks like is about 1500 lines of code in 6 classes).

ObjectCreationHandling och = ObjectCreationHandling.Auto;
if( typeInitializer == null )
{
 if( parameterlessConstructor )
 {
  och = ObjectCreationHandling.Reuse;
 }
 else
 {
  och = ObjectCreationHandling.Replace;
 }
}

This setting is a part of the JsonSerializerSettings which are composed inside of the visitor pattern constructor for DeserializeObject. As shown above, each setting has a different function.

Getting back to List, Collection, and ReadOnlyCollection, we will look at the set of conditional statements for each.

List

testObj.List.GetType().TypeInitializer == null is false. As a result, List receives the default ObjectCreationHandling.Auto and the instantiated List for the testObj instance is used during deserialization, as well as a new List being instantiated with the serialized string.

testObj.List: ABC,DEF,Goodbye,AOL

Collection

testObj.Collection.GetType().TypeInitializer == null is true indicating there was no reflected type initializer available, so we go to the next condition which is to check if there is a parameterless constructor. testObj.Collection.GetType().GetConstructor(Type.EmptyTypes) == null is false. As a result Collection receives the value of ObjectCreationHandling.Reuse (only reuse existing objects). The instantiated instance for Collection is used from testObj, but the serialized string is not able to be instantiated.

testObj.Collection: ABC,DEF

ReadOnlyCollection

testObj.ReadOnlyCollection.GetType().TypeInitializer == null is true indicating there was no reflected type initializer available, so we go to the next condition which is to check if there is a parameterless constructor. testObj.ReadOnlyCollection.GetType().GetConstructor(Type.EmptyTypes) == null is also true. As a result ReadOnlyCollection recieves the value of ObjectCreationHandling.Replace (always create new objects). Only the instantiated value from the serialized string is used.

testObj.ReadOnlyCollection: Goodbye,AOL

Answer 2

Although this has been resolved, I wanted to post this answer to a duplicate question originally, but that question was closed, so I am publishing my answer here, as it contains some internal look into this.

Because Json.NET is open source, we can luckily track down the reason down to its root :-) .

If you check the Json.NET source, you can find class JsonSerializerInternalReader which handles the deserialization (complete source here). This class has a method SetPropertyValue, which sets the deserialized value on the newly created object (code abbreviated):

private bool SetPropertyValue(JsonProperty property, ..., object target)
{
    ...
    if (CalculatePropertyDetails(
          property, 
          ...,
          out useExistingValue,
          ... ))
    {
        return false;
    }

    ...

    if (propertyConverter != null && propertyConverter.CanRead)
    {
        ...
    }
    else
    {
        value = CreateValueInternal(
           ...,
           (useExistingValue) ? currentValue : null);
    }

    if ((!useExistingValue || value != currentValue)
        && ShouldSetPropertyValue(property, value))
    {
        property.ValueProvider.SetValue(target, value);
        ...    
        return true;
    }
    return useExistingValue;
}

As you can see, there is a boolean flag useExistingValue which determines whether or not the existing value is reused or replaced.

Inside the CalculatePropertyDetails method is the following snippet:

        if ((objectCreationHandling != ObjectCreationHandling.Replace)
            && (tokenType == JsonToken.StartArray || tokenType == JsonToken.StartObject)
            && property.Readable)
        {
            currentValue = property.ValueProvider.GetValue(target);
            gottenCurrentValue = true;

            if (currentValue != null)
            {
                ...

                useExistingValue = (
                   !propertyContract.IsReadOnlyOrFixedSize &&
                   !propertyContract.UnderlyingType.IsValueType());
            }
        }

In case of the List<T> underlying collection, the IsReadOnlyOrFixedSize returns false and IsValueType() returns false - hence the underlying existing value is reused.

For Array, IsValueType() is also false, but the IsReadOnlyOrFixedSize is true for obvious reasons, therefore the useExistingValue flag is set to false and the CreateValueInternal call in the SetPropertyValue method receives a null reference which is an indicator not to reuse the existing value, but to create a new one, which is then set on the new instance.

As was mentioned, this behavior can be altered using ObjectCreationHandling.Replace, as this is checked before setting the useExistingValue in the CalculatePropertyDetails method.