Adding backward compatibility support for an older JSON structure

Question

I have developed an app for android which stores a serialized domain model in a JSON file to the local storage. Now the thing is, sometimes I make changes to the domain model (new features) and want to have the option to easily load a previous structure of the JSON file from the local storage. How can I do this?

I thought of deserializing the object anonymously and using auto-mapper, but I want to hear others' ideas first before going this path.

If a code example of the domain model is needed (before and after), I'll provide. Thanks everyone.

Answer 1

How you support backward compatibility depends on how different your "before" and "after" models are going to be.

If you are just going to be adding new properties, then this should not pose a problem at all; you can just deserialize the old JSON into the new model and it will work just fine without errors.

If you are replacing obsolete properties with different properties, you can use techniques described in Making a property deserialize but not serialize with json.net to migrate old properties to new.

If you are making big structural changes, then you may want to use different classes for each version. When you serialize the models, ensure that a Version property (or some other reliable marker) is written into the JSON. Then when it is time to deserialize, you can load the JSON into a JToken, inspect the Version property and then populate the appropriate model for the version from the JToken. If you want, you can encapsulate this logic into a JsonConverter class.

---

Let's walk through some examples. Say we are writing an application which keeps some information about people. We'll start with the simplest possible model: a Person class which has a single property for the person's name.

public class Person  // Version 1
{
    public string Name { get; set; }
}

Let's create a "database" of people (I'll just use a simple list here) and serialize it.

List<Person> people = new List<Person>
{
    new Person { Name = "Joe Schmoe" }
};
string json = JsonConvert.SerializeObject(people);
Console.WriteLine(json);

That gives us the following JSON.

[{"Name":"Joe Schmoe"}]

Fiddle: https://dotnetfiddle.net/NTOnu2

---

OK, now say we want to enhance the application to keep track of people's birthdays. This will not be a problem for backward compatibility because we're just going to be adding a new property; it won't affect the existing data in any way. Here's what the Person class looks like with the new property:

public class Person  // Version 2
{
    public string Name { get; set; }
    public DateTime? Birthday { get; set; }
}

To test it, we can deserialize the Version 1 data into this new model, then add a new person to the list and serialize the model back to JSON. (I'll also add a formatting option to make the JSON easier to read.)

List<Person> people = JsonConvert.DeserializeObject<List<Person>>(json);
people.Add(new Person { Name = "Jane Doe", Birthday = new DateTime(1988, 10, 6) });
json = JsonConvert.SerializeObject(people, Formatting.Indented);
Console.WriteLine(json);

Everything works great. Here's what the JSON looks like now:

[
  {
    "Name": "Joe Schmoe",
    "Birthday": null
  },
  {
    "Name": "Jane Doe",
    "Birthday": "1988-10-06T00:00:00"
  }
]

Fiddle: https://dotnetfiddle.net/pftGav

---

Alright, now let's say we've realized that just using a single Name property isn't robust enough. It would be better if we had separate FirstName and LastName properties instead. That way we can do things like sort the names in directory order (last, first) and print informal greetings like "Hi, Joe!".

Fortunately, we know that the data has been reliably entered so far with the first name preceding the last name and a space between them, so we have a viable upgrade path: we can split the Name property on the space and fill the two new properties from it. After we do that, we want to treat the Name property as obsolete; we don't want it written back to the JSON in the future.

Let's make some changes to our model to accomplish these goals. After adding the two new string properties FirstName and LastName, we need to change the old Name property as follows:

• Make its set method set the FirstName and LastName properties as explained above;
• Remove its get method so that the Name property does not get written to JSON;
• Make it private so it is no longer part of the public interface of Person;
• Add a [JsonProperty] attribute so that Json.Net can still "see" it even though it is private.

And of course, we'll have to update any other code that uses the Name property to use the new properties instead. Here is what our Person class looks like now:

public class Person  // Version 3
{
    public string FirstName { get; set; }
    public string LastName { get; set; }
    public DateTime? Birthday { get; set; }

    // This property is here to support transitioning from Version 2 to Version 3
    [JsonProperty]
    private string Name
    {
        set
        {
            if (value != null)
            {
                string[] parts = value.Trim().Split(' ');
                if (parts.Length > 0) FirstName = parts[0];
                if (parts.Length > 1) LastName = parts[1];
            }
        }
    }
}

To demonstrate that everything works, let's load our Version 2 JSON into this model, sort the people by last name and then reserialize it to JSON:

List<Person> people = JsonConvert.DeserializeObject<List<Person>>(json);
people = people.OrderBy(p => p.LastName).ThenBy(p => p.FirstName).ToList();
json = JsonConvert.SerializeObject(people, Formatting.Indented);
Console.WriteLine(json);

Looks good! Here is the result:

[
  {
    "FirstName": "Jane",
    "LastName": "Doe",
    "Birthday": "1988-10-06T00:00:00"
  },
  {
    "FirstName": "Joe",
    "LastName": "Schmoe",
    "Birthday": null
  }
]    

Fiddle: https://dotnetfiddle.net/T8NXMM

---

Now for the big one. Let's say we want add a new feature to keep track of each person's home address. But the kicker is, people can share the same address, and we don't want duplicate data in that case. This requires a big change to our data model, because up until now it's just been a list of people. Now we need a second list for the addresses, and we need a way to tie the people to the addresses. And of course we still want to support reading all the old data formats. How can we do this?

First let's create the new classes we will need. We need an Address class of course:

public class Address
{
    public int Id { get; set; }
    public string Street { get; set; }
    public string City { get; set; }
    public string State { get; set; }
    public string PostalCode { get; set; }
    public string Country { get; set; }
}

We can reuse the same Person class; the only change we need is to add an AddressId property to link each person to an address.

public class Person
{
    public int? AddressId { get; set; }
    ...
}

Lastly, we need a new class at the root level to hold the lists of people and addresses. Let's also give it a Version property in case we need to make changes to the data model in the future:

public class RootModel
{
    public string Version { get { return "4"; } }
    public List<Person> People { get; set; }
    public List<Address> Addresses { get; set; }
}

That's it for the model; now the big issue is how do we handle the differing JSON? In versions 3 and earlier, the JSON was an array of objects. But with this new model, the JSON will be an object containing two arrays.

The solution is to use a custom JsonConverter for the new model. We can read the JSON into a JToken and then populate the new model differently depending on what we find (array vs. object). If we get an object, we'll check for the new version number property we just added to the model.

Here is the code for the converter:

public class RootModelConverter : JsonConverter
{
    public override bool CanConvert(Type objectType)
    {
        return objectType == typeof(RootModel);
    }

    public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
    {
        JToken token = JToken.Load(reader);
        RootModel model = new RootModel();
        if (token.Type == JTokenType.Array)
        {
            // we have a Version 3 or earlier model, which is just a list of people.
            model.People = token.ToObject<List<Person>>(serializer);
            model.Addresses = new List<Address>();
            return model;
        }
        else if (token.Type == JTokenType.Object)
        {
            // Check that the version is something we are expecting
            string version = (string)token["Version"];
            if (version == "4")
            {
                // all good, so populate the current model
                serializer.Populate(token.CreateReader(), model);
                return model;
            }
            else
            {
                throw new JsonException("Unexpected version: " + version);
            }
        }
        else
        {
            throw new JsonException("Unexpected token: " + token.Type);
        }
    }

    // This signals that we just want to use the default serialization for writing
    public override bool CanWrite
    {
        get { return false; }
    }

    public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
    {
        throw new NotImplementedException();
    }
}

To use the converter, we create an instance and pass it to the DeserializeObject method like this:

RootModelConverter converter = new RootModelConverter();
RootModel model = JsonConvert.DeserializeObject<RootModel>(json, converter);

Now that we have the model loaded, we can update the data to show that Joe and Jane live at the same address and serialize it back out again:

model.Addresses.Add(new Address
{
    Id = 1,
    Street = "123 Main Street",
    City = "Birmingham",
    State = "AL",
    PostalCode = "35201",
    Country = "USA"
});

foreach (var person in model.People)
{
    person.AddressId = 1;
}

json = JsonConvert.SerializeObject(model, Formatting.Indented);
Console.WriteLine(json);

Here is the resulting JSON:

{
  "Version": 4,
  "People": [
    {
      "FirstName": "Jane",
      "LastName": "Doe",
      "Birthday": "1988-10-06T00:00:00",
      "AddressId": 1
    },
    {
      "FirstName": "Joe",
      "LastName": "Schmoe",
      "Birthday": null,
      "AddressId": 1
    }
  ],
  "Addresses": [
    {
      "Id": 1,
      "Street": "123 Main Street",
      "City": "Birmingham",
      "State": "AL",
      "PostalCode": "35201",
      "Country": "USA"
    }
  ]
}

We can confirm the converter works with the new Version 4 JSON format as well by deserializing it again and dumping out some of the data:

model = JsonConvert.DeserializeObject<RootModel>(json, converter);
foreach (var person in model.People)
{
    Address addr = model.Addresses.FirstOrDefault(a => a.Id == person.AddressId);
    Console.Write(person.FirstName + " " + person.LastName);
    Console.WriteLine(addr != null ? " lives in " + addr.City + ", " + addr.State : "");
}

Output:

Jane Doe lives in Birmingham, AL
Joe Schmoe lives in Birmingham, AL

Fiddle: https://dotnetfiddle.net/4lcDvE