Cannot preserve reference to array or readonly list, or list created from a non-default constructor

Question

I came across the below question which is mostly identical to the issue I am having:

JSON.NET cannot handle simple array deserialization?

However, my situation is slightly different. If I modify the Test class from that question to have an array property of the same type, I get the same deserialization error.

class Test
{
    public Test[] Tests;
}

var settings = new JsonSerializerSettings
{
    PreserveReferencesHandling = PreserveReferencesHandling.All
};

var o = new Test { Tests = new[] { new Test(), new Test() } };
//var o = new Test(); //this works if I leave the Tests array property null
var arr = new[] { o, o };
var ser = JsonConvert.SerializeObject(arr, settings);

arr = ((JArray)JsonConvert.DeserializeObject(ser, settings)).ToObject<Test[]>();

I bet I am missing an important attribute on the Tests property.

Answer 1

Json.NET simply hasn't implemented preserving of references for read-only collections and arrays. This is explicitly stated in the exception message:

Newtonsoft.Json.JsonSerializationException: Cannot preserve reference to array or readonly list, or list created from a non-default constructor: Question41293407.Test[].

The reason that Newtonsoft has not implemented this is that their reference tracking functionality is intended to be capable of preserving recursive self references. Thus the object being deserialized must be allocated before reading its contents, so that nested back-references can be successfully resolved during content deserialization. However, a read-only collection can only be allocated after its contents have been read, since by definition it is read-only.

Arrays, however, are peculiar in that they are only "semi" read-only: they cannot be resized after being allocated, however individual entries can be changed. (see Array.IsReadOnly inconsistent depending on interface implementation for a discussion about this.) It's possible to take advantage of this fact to create a custom JsonConverter for arrays that, during reading, loads the JSON into an intermediate JToken, allocates an array of the correct size by querying the token's contents, adds the array to the serializer.ReferenceResolver, deserializes the contents into a list, then finally populates the array entries from the list:

public class ArrayReferencePreservngConverter : JsonConverter
{
    const string refProperty = "$ref";
    const string idProperty = "$id";
    const string valuesProperty = "$values";

    public override bool CanConvert(Type objectType)
    {
        return objectType.IsArray;
    }

    public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
    {
        if (reader.TokenType == JsonToken.Null)
            return null;
        else if (reader.TokenType == JsonToken.StartArray)
        {
            // No $ref.  Deserialize as a List<T> to avoid infinite recursion and return as an array.
            var elementType = objectType.GetElementType();
            var listType = typeof(List<>).MakeGenericType(elementType);
            var list = (IList)serializer.Deserialize(reader, listType);
            if (list == null)
                return null;
            var array = Array.CreateInstance(elementType, list.Count);
            list.CopyTo(array, 0);
            return array;
        }
        else
        {
            var obj = JObject.Load(reader);
            var refId = (string)obj[refProperty];
            if (refId != null)
            {
                var reference = serializer.ReferenceResolver.ResolveReference(serializer, refId);
                if (reference != null)
                    return reference;
            }
            var values = obj[valuesProperty];
            if (values == null || values.Type == JTokenType.Null)
                return null;
            if (!(values is JArray))
            {
                throw new JsonSerializationException(string.Format("{0} was not an array", values));
            }
            var count = ((JArray)values).Count;

            var elementType = objectType.GetElementType();
            var array = Array.CreateInstance(elementType, count);

            var objId = (string)obj[idProperty];
            if (objId != null)
            {
                // Add the empty array into the reference table BEFORE poppulating it,
                // to handle recursive references.
                serializer.ReferenceResolver.AddReference(serializer, objId, array);
            }

            var listType = typeof(List<>).MakeGenericType(elementType);
            using (var subReader = values.CreateReader())
            {
                var list = (IList)serializer.Deserialize(subReader, listType);
                list.CopyTo(array, 0);
            }

            return array;
        }
    }

    public override bool CanWrite { get { return false; } }

    public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
    {
        throw new NotImplementedException();
    }
}

The memory efficiency of this approach is not great, so for large collections it would be better to switch to a List<T>.

Then use it like:

var settings = new JsonSerializerSettings
{
    Converters = { new ArrayReferencePreservngConverter() },
    PreserveReferencesHandling = PreserveReferencesHandling.All
};
var a2 = JsonConvert.DeserializeObject<Test[]>(jsonString, settings);

Note the converter is completely generic and works for all arrays.

Sample fiddle showing successful deserialization of nested recursive self-references.

Answer 2

if someone utilizes this converter, might be considering to modify this:

 public override bool CanConvert(Type objectType)
 {
        if (objectType == typeof(byte[]))
            return false; // He would kill a Byte[] and you'll wonder, why the JSON Deserializer will return NULL on Byte[] :-)

        return objectType.IsArray;
 }

Answer 3

I think this code is nice but needs a refine

 var elementType = objectType.IsArray ? objectType.GetElementType() : objectType.GetGenericArguments()[0];

objectType.IsGenericType might be true so, we need to use GetGenericArguments()[0]