How to deserialize JSON with duplicate property names in the same object

Question

I have a JSON string that I expect to contain duplicate keys that I am unable to make JSON.NET happy with.

I was wondering if anybody knows the best way (maybe using JsonConverter? ) to get JSON.NET to change a JObject's child JObjects into to JArrays when it sees duplicate key names ?

// For example: This gives me a JObject with a single "JProperty\JObject" child.
var obj = JsonConvert.DeserializeObject<object>("{ \"HiThere\":1}");

// This throws:
// System.ArgumentException : Can not add Newtonsoft.Json.Linq.JValue to Newtonsoft.Json.Linq.JObject.
obj = JsonConvert.DeserializeObject<object>("{ \"HiThere\":1, \"HiThere\":2, \"HiThere\":3 }");

The actual JSON I am trying to deserialize is much more complicated and the duplicates are nested at multiple levels. But the code above demonstrates why it fails for me.

I understand that the JSON is not correct which is why I am asking if JSON.NET has a way to work around this. For argument's sake let's say I do not have control over the JSON. I actually do use a specific type for the parent object but the particular property that is having trouble will either be a string or another nested JSON object. The failing property type is "object" for this reason.

Answer 1

Interesting question. I played around with this for a while and discovered that while a JObject cannot contain properties with duplicate names, the JsonTextReader used to populate it during deserialization does not have such a restriction. (This makes sense if you think about it: it's a forward-only reader; it is not concerned with what it has read in the past). Armed with this knowledge, I took a shot at writing some code that will populate a hierarchy of JTokens, converting property values to JArrays as necessary if a duplicate property name is encountered in a particular JObject. Since I don't know your actual JSON and requirements, you may need to make some adjustments to it, but it's something to start with at least.

Here's the code:

public static JToken DeserializeAndCombineDuplicates(JsonTextReader reader)
{
    if (reader.TokenType == JsonToken.None)
    {
        reader.Read();
    }

    if (reader.TokenType == JsonToken.StartObject)
    {
        reader.Read();
        JObject obj = new JObject();
        while (reader.TokenType != JsonToken.EndObject)
        {
            string propName = (string)reader.Value;
            reader.Read();
            JToken newValue = DeserializeAndCombineDuplicates(reader);

            JToken existingValue = obj[propName];
            if (existingValue == null)
            {
                obj.Add(new JProperty(propName, newValue));
            }
            else if (existingValue.Type == JTokenType.Array)
            {
                CombineWithArray((JArray)existingValue, newValue);
            }
            else // Convert existing non-array property value to an array
            {
                JProperty prop = (JProperty)existingValue.Parent;
                JArray array = new JArray();
                prop.Value = array;
                array.Add(existingValue);
                CombineWithArray(array, newValue);
            }

            reader.Read();
        }
        return obj;
    }

    if (reader.TokenType == JsonToken.StartArray)
    {
        reader.Read();
        JArray array = new JArray();
        while (reader.TokenType != JsonToken.EndArray)
        {
            array.Add(DeserializeAndCombineDuplicates(reader));
            reader.Read();
        }
        return array;
    }

    return new JValue(reader.Value);
}

private static void CombineWithArray(JArray array, JToken value)
{
    if (value.Type == JTokenType.Array)
    {
        foreach (JToken child in value.Children())
            array.Add(child);
    }
    else
    {
        array.Add(value);
    }
}

And here's a demo:

class Program
{
    static void Main(string[] args)
    {
        string json = @"
        {
            ""Foo"" : 1,
            ""Foo"" : [2],
            ""Foo"" : [3, 4],
            ""Bar"" : { ""X"" : [ ""A"", ""B"" ] },
            ""Bar"" : { ""X"" : ""C"", ""X"" : ""D"" },
        }";

        using (StringReader sr = new StringReader(json))
        using (JsonTextReader reader = new JsonTextReader(sr))
        {
            JToken token = DeserializeAndCombineDuplicates(reader);
            Dump(token, "");
        }
    }

    private static void Dump(JToken token, string indent)
    {
        Console.Write(indent);
        if (token == null)
        {
            Console.WriteLine("null");
            return;
        }
        Console.Write(token.Type);

        if (token is JProperty)
            Console.Write(" (name=" + ((JProperty)token).Name + ")");
        else if (token is JValue)
            Console.Write(" (value=" + token.ToString() + ")");

        Console.WriteLine();

        if (token.HasValues)
            foreach (JToken child in token.Children())
                Dump(child, indent + "  ");
    }
}

Output:

Object
  Property (name=Foo)
    Array
      Integer (value=1)
      Integer (value=2)
      Integer (value=3)
      Integer (value=4)
  Property (name=Bar)
    Array
      Object
        Property (name=X)
          Array
            String (value=A)
            String (value=B)
      Object
        Property (name=X)
          Array
            String (value=C)
            String (value=D)