How to handle both a single item and an array for the same property using System.Text.Json?

Question

I am trying to deserialize some JSON that contains a value that is sometimes an array, and sometimes a single item. How can I do this with System.Text.Json and JsonSerializer? (This question is inspired by this question for Json.NET by Robert McLaws.)

I have received the following JSON:

[
  {
    "email": "[email protected]",
    "timestamp": 1337966815,
    "category": [
      "newuser",
      "transactional"
    ],
    "event": "open"
  },
  {
    "email": "[email protected]",
    "timestamp": 1337966815,
    "category": "olduser",
    "event": "open"
  }
]

And I want to deserialize it to a list of the following type:

class Item
{
    public string Email { get; set; }
    public int Timestamp { get; set; }
    public string Event { get; set; }

    public List<string> Category { get; set; }
}

Using the following code:

var options = new JsonSerializerOptions
{
    PropertyNamingPolicy = JsonNamingPolicy.CamelCase,
};
var list = JsonSerializer.Deserialize<List<Item>>(json, options);

However, when I do I get the following exception:

System.Text.Json.JsonException: The JSON value could not be converted to System.String. Path: > $[1].category | LineNumber: 13 | BytePositionInLine: 25.
   at System.Text.Json.ThrowHelper.ThrowJsonException_DeserializeUnableToConvertValue(Type propertyType)
   at System.Text.Json.JsonPropertyInfo.Read(JsonTokenType tokenType, ReadStack& state, Utf8JsonReader& reader)
   at System.Text.Json.JsonSerializer.ReadCore(JsonSerializerOptions options, Utf8JsonReader& reader, ReadStack& readStack)
   at System.Text.Json.JsonSerializer.ReadCore(Type returnType, JsonSerializerOptions options, Utf8JsonReader& reader)
   at System.Text.Json.JsonSerializer.Deserialize(String json, Type returnType, JsonSerializerOptions options)
   at System.Text.Json.JsonSerializer.Deserialize[TValue](String json, JsonSerializerOptions options)

The exception occurs because the value of "category" is sometimes a single string, and sometimes an array of strings. How can I deserialize such a property with System.Text.Json?

Answer 1

As inspired by this answer by Brian Rogers and other answers to How to handle both a single item and an array for the same property using JSON.net, you can create a generic JsonConverter<List<T>> that checks whether the incoming JSON value is an array, and if not, deserializes an item of type T and returns the item wrapped in an appropriate list. Even better, you can create a JsonConverterFactory that manufactures such a converter for all list types List<T> encountered in your serialization graph.

First, define the following converter and converter factory:

public class SingleOrArrayConverter<TItem> : SingleOrArrayConverter<List<TItem>, TItem>
{
    public SingleOrArrayConverter() : this(true) { }
    public SingleOrArrayConverter(bool canWrite) : base(canWrite) { }
}

public class SingleOrArrayConverterFactory : JsonConverterFactory
{
    public bool CanWrite { get; }

    public SingleOrArrayConverterFactory() : this(true) { }

    public SingleOrArrayConverterFactory(bool canWrite) => CanWrite = canWrite;

    public override bool CanConvert(Type typeToConvert)
    {
        var itemType = GetItemType(typeToConvert);
        if (itemType == null)
            return false;
        if (itemType != typeof(string) && typeof(IEnumerable).IsAssignableFrom(itemType))
            return false;
        if (typeToConvert.GetConstructor(Type.EmptyTypes) == null || typeToConvert.IsValueType)
            return false;
        return true;
    }

    public override JsonConverter CreateConverter(Type typeToConvert, JsonSerializerOptions options)
    {
        var itemType = GetItemType(typeToConvert);
        var converterType = typeof(SingleOrArrayConverter<,>).MakeGenericType(typeToConvert, itemType);
        return (JsonConverter)Activator.CreateInstance(converterType, new object [] { CanWrite });
    }

    static Type GetItemType(Type type)
    {
        // Quick reject for performance
        if (type.IsPrimitive || type.IsArray || type == typeof(string))
            return null;
        while (type != null)
        {
            if (type.IsGenericType)
            {
                var genType = type.GetGenericTypeDefinition();
                if (genType == typeof(List<>))
                    return type.GetGenericArguments()[0];
                // Add here other generic collection types as required, e.g. HashSet<> or ObservableCollection<> or etc.
            }
            type = type.BaseType;
        }
        return null;
    }
}

public class SingleOrArrayConverter<TCollection, TItem> : JsonConverter<TCollection> where TCollection : class, ICollection<TItem>, new()
{
    public SingleOrArrayConverter() : this(true) { }
    public SingleOrArrayConverter(bool canWrite) => CanWrite = canWrite;

    public bool CanWrite { get; }

    public override TCollection Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
    {
        switch (reader.TokenType)
        {
            case JsonTokenType.Null:
                return null;
            case JsonTokenType.StartArray:
                var list = new TCollection();
                while (reader.Read())
                {
                    if (reader.TokenType == JsonTokenType.EndArray)
                        break;
                    list.Add(JsonSerializer.Deserialize<TItem>(ref reader, options));
                }
                return list;
            default:
                return new TCollection { JsonSerializer.Deserialize<TItem>(ref reader, options) };
        }
    }

    public override void Write(Utf8JsonWriter writer, TCollection value, JsonSerializerOptions options)
    {
        if (CanWrite && value.Count == 1)
        {
            JsonSerializer.Serialize(writer, value.First(), options);
        }
        else
        {
            writer.WriteStartArray();
            foreach (var item in value)
                JsonSerializer.Serialize(writer, item, options);
            writer.WriteEndArray();
        }
    }
}

Then add the the converter factory to JsonSerializerOptions.Converters before deserialization:

var options = new JsonSerializerOptions
{
    Converters = { new SingleOrArrayConverterFactory() },
    PropertyNamingPolicy = JsonNamingPolicy.CamelCase,
};
var list = JsonSerializer.Deserialize<List<Item>>(json, options);

Or add a specific converter either to options or to your data model directly using JsonConverterAttribute:

class Item
{
    public string Email { get; set; }
    public int Timestamp { get; set; }
    public string Event { get; set; }

    [JsonConverter(typeof(SingleOrArrayConverter<string>))]
    public List<string> Category { get; set; }
}

If your data model uses some other type of collection, say ObservableCollection<string>, you can apply a lower level converter SingleOrArrayConverter<TCollection, TItem> as follows:

    [JsonConverter(typeof(SingleOrArrayConverter<ObservableCollection<string>, string>))]
    public ObservableCollection<string> Category { get; set; }

Notes:

• If you want the converter(s) to apply only during deserialization, pass canWrite: false to the parameterized constructor:

  Converters = { new SingleOrArrayConverterFactory(canWrite: false) }
  

  The converter will still get used, but will unconditionally generate a default serialization.

• The converter is not implemented for jagged 2d or nD collections such as List<List<string>>. It is also not implemented for arrays and read-only collections.

• According to Serializer support for easier object and collection converters #1562, because JsonConverter<T> lacks an async Read() method,

  A limitation of the existing [JsonConverter] model is that it must "read-ahead" during deserialization to fully populate the buffer up to the end up the current JSON level. This read-ahead only occurs when the async+stream JsonSerializer deserialize methods are called and only when the current JSON for that converter starts with a StartArray or StartObject token.

  Thus using this converter to deserialize potentially very large arrays may have a negative performance impact.

  As discussed in the same thread, the converter API may get redesigned in System.Text.Json - 5.0 to fully support async deserialization by converters for arrays and object, implying that this converter may benefit from being rewritten when .NET 5 (no longer labeled with "Core") is eventually released.

Demo fiddle here.