Newtonsoft JSON.NET Deserialization error

Question

I am have a bunch of long json output in individual files. I need to read these files and deserialize them into the entities that originally generated the json (I have access to the original entities). Each file has the json output that was generated by serializing an object of type IEnumerable<Response<MyEntity>>.

I am getting an exception when attempting to deserialize, but I don't understand why. Here is my attempt to deserialize:

List<string> jsonOutputStrings;
// Read json from files and populate jsonOutputStrings list
List<Response<MyEntity>> apiResponses = new List<Response<MyEntity>>();

foreach (string json in jsonOutputStrings)
{
    apiResponses.AddRange(JsonConvert.DeserializeObject<List<Response<MyEntity>>>(json, new JsonSerializerSettings { TypeNameHandling = TypeNameHandling.All }).ToList());
}

I also tried deserializing to an IEnumerable instead of a list:

apiResponses.AddRange(JsonConvert.DeserializeObject<IEnumerable<Response<MyEntity>>>(json, new JsonSerializerSettings { TypeNameHandling = TypeNameHandling.All }).ToList());

I get the following exception:

A first chance exception of type 'Newtonsoft.Json.JsonSerializationException' occurred in Newtonsoft.Json.dll

Additional information: Cannot create and populate list type System.Linq.Enumerable+WhereSelectListIterator`2[Entities.Requirement,Entities.RequirementEntity]. Path '$values[0].ReturnedEntity.Summaries.$values[0].Requirements.$values', line 1, position 715.

The entire json is too long to post (and also contains some confidential data) but I did find a place in the json that has this:

"Requirements":{"$id":"7","$type":"System.Linq.Enumerable+WhereSelectListIterator`2[[Entities.Requirement, Entities],[Entities.RequirementEntity, API.Entities]], System.Core","$values":[...]}

In the entity that I'm trying to deserialize into (same one that was originally serialized), Requirements is of type IEnumerable<Entities.RequirementEntity>.

It doesn't make sense to me how serialization from MyEntity works but deserializing to the same type doesn't. How do I solve this problem?

Answer 1

You must browse through Response<> and MyEntity and see how collections are initialized. This tells us that one of the collections in some class is created using Where method from linq. You can reproduce this error by executing this code:

class MyEntity
{
    public MyEntity()
    {
        Data = new List<string>().Where(x => true);
    }

    public IEnumerable<string> Data { get; set; }

}

class Program
{
    static void Main(string[] args)
    {
        string data = @"[{""Data"":[""a"",""b""]}]";
        var j = JsonConvert.DeserializeObject<IEnumerable<MyEntity>>(data);
    }
}

Another possibility is to have metadata in json. Then you have 2 solutions:

• Set TypeNameHandling to TypeNameHandling.None (as somebody mentioned in comment);
• Replace not working types in string with working ones

Using TypeNameHandling.None may lead to wrong deserialization for exmaple when you have IEnumerable<BaseType> and that list contains subtype of BaseType.

In that case you should go for second option. Basically you should replace any type that is not deserializing and replace it for example with List.

Sample code:

class MyEntity
{
    public IEnumerable<string> Data { get; set; }
}

class Program
{
    static void Main(string[] args)
    {
        IList<MyEntity> entities = new MyEntity[] {
            new MyEntity { Data = new [] { "1", "2" }.Where(x => x != string.Empty) },
            new MyEntity { Data = new [] { "A", "B" }.AsQueryable().Where(x => x != string.Empty) },
            new MyEntity { Data = new List<string> { "A", "B" } },
        };

        string data = JsonConvert.SerializeObject(entities, Formatting.Indented, new JsonSerializerSettings { TypeNameHandling = TypeNameHandling.All });
        data = Regex.Replace(data, "\"\\$type\":\\s+\"System.Linq.Enumerable\\+WhereArrayIterator(.+?), System.Core\",", "\"$type\": \"System.Collections.Generic.List$1, mscorlib\",", RegexOptions.Singleline);

        var j = JsonConvert.DeserializeObject<IEnumerable<MyEntity>>(data, new JsonSerializerSettings { TypeNameHandling = TypeNameHandling.All });
    }
}

Answer 2

Is there a Linq result in the serialized entity? Maybe defined as an IEnumerable and filled with a Linq Where result? It seems that's where your problem lies. You should convert it to a List or Array before serializing.