Why are all the collections in my POCO are null when deserializing some valid json with the .NET Newtonsoft.Json component

Question

Problem

When trying to deserialize some Json using Newtonsoft's Json.NET nuget library my collection-properties are null when I do NOT provide a JsonSerializerSettings - why?

Details

I have some valid json data and deserializing it into my custom class / POCO. Now, all the simple properties (like strings, int's, etc) all get set correctly. My simple child classes also get set properly (eg. a User property or a BuildingDetails property, etc).

All my collections are null, though. The setters are never called (I've put a break point in there and they don't get set, but the others do get called/set).

eg. property.

List<Media> Images { get { ... } set { ... } }

eg.

var foo = new Foo();
var json = JsonConvert.Serialize(foo);
var anotherFoo = JsonConvert.Deserialize<Foo>(json);

// the collection properties on anotherFoo are null

Now - this is the crazy thing: when I use a JsonSerializerSettings it now works:

// NOTE: ModifiedDataContractResolver <-- skips any property that is a type
/        ModifiedData (which is a simple custom class I have).
var settings = new JsonSerializerSettings
{
    ContractResolver = new ModifiedDataContractResolver(),
    ObjectCreationHandling = ObjectCreationHandling.Replace,
    Formatting = Formatting.Indented
};

var foo = new Foo();
var json = JsonConvert.Serialize(foo, settings);
var anotherFoo = JsonConvert.Deserialize<Foo>(json, settings);

my collections are now set!

Take note of this: ObjectCreationHandling = ObjectCreationHandling.Replace. This is the magical setting that makes things work.

What is this doing and why does not having this setting mean my collections are not getting set/created/etc?

Another clue. If I do not have the setter, collection is null / not set. If I have the setter like this, it fails too:

public List<Media> Images
{
    get { return new List<Media> { new Media{..}, new Media{..} }; }
   set { AddImages(value); }
}

If I do NOT return the list, in the getter the collection is SET! IT WORKS!

private List<Media> _images;
public List<Media> Images
{
    get { return _images; }
    set { AddImages(value); }
}

Notice how it's now just using a baking field instead?

It's like there's some weird connection/correlation between the collection's GETTER property and result and how Json.net sets this data using the auto setting?

Answer 1

Your problem is that the property you are trying to deserialize gets and sets a proxy collection rather than a "real" collection in your object graph. Doing so falls afoul of an implementation decision about the algorithm Json.NET uses to construct and populate properties returning reference type objects, collections and dictionaries. That algorithm is:

1. It calls the getter in the parent class to get the current value of the property being deserialized.

2. If null, and unless a custom constructor is being used, it allocates an instance of the property's returned type (using the JsonContract.DefaultCreator method for the type).

3. It calls the setter in the parent to set the allocated instance back into the parent.

4. It proceeds to populate the instance of the type.

5. It does not set the instance back a second time, after it has been populated.

Calling the getter at the beginning makes it possible to populate an instance of a type (e.g. with JsonConvert.PopulateObject()), recursively populating any pre-allocated instances of other types to which the type refers. (That's the purpose of the existingValue parameter in JsonConverter.ReadJson().)

However, the ability to populate a pre-allocated object graph in this manner comes with a cost: each object encountered in the graph must be the real object and not some proxy object created for serialization purposes. If the object returned by the getter is just some proxy, the proxy will get populated, and not the "real" object - unless the proxy has some sort of mechanism to pass changes to its data back to its originator.

(While this decision may seem undesirable, it's not unusual; XmlSerializer works the same way. For a list of serializers that do this, see XML Deserialization of collection property with code defaults.)

ObjectCreationHandling = ObjectCreationHandling.Replace, as you have observed, changes this algorithm, so that collections are allocated, populated, and set back. That is one way to enable deserialization of proxy collections.

As another workaround, you could choose instead to serialize and deserialize a proxy array:

[JsonIgnore]
public List<Media> Images
{
    get { return new List<Media> { new Media{..}, new Media{..} }; }
    set { AddImages(value); }
}

[JsonProperty("Images")] // Could be private
Media [] ImagesArray
{
    get { return Images.ToArray(); }
    set { AddImages(value); }
}

For arrays, Json.NET (and XmlSerializer) must call the setter after the array is fully read, since the size cannot be known until fully read, and so the array cannot be allocated and set back until fully read.

(You could also do tricks with a proxy ObservableCollection, but I wouldn't recommend it.)