Populating a complex default value with Json.NET for an F# record label

Question

Let's say I have a first version of a record, which I am serializing and deserializing to persist the data it is representing. Now I want to add a new label to that record. As F# records are immutable, that label should be populated with a meaningful default value while the persisted old version of the record is being deserialized.

That works well for simple values using the System.ComponentModel.DefaultValueAttribute on that new label. But unfortunately I cannot figure out how to pass a complex value here. Something like another record, or a discriminated union.

I would buy whatever sane solution works. For example, I would also be happy to register some functions that provide default values for given types during deserialization.

open FsUnit
open NUnit.Framework
open Newtonsoft.Json

type Gender = | Female | Male | Other

type TenantV1 = { Name: string }
type TenantV3 =
  { Name: string;
    [<System.ComponentModel.DefaultValue("John")>]
    FirstName: string }
type TenantV4 =
  { Name: string;
    [<System.ComponentModel.DefaultValue(typeof<Gender>, "Other")>] 
    Gender: Gender }

let serializationSettings = JsonSerializerSettings()
serializationSettings.TypeNameHandling <- TypeNameHandling.All

let deserializationSettings = JsonSerializerSettings()
deserializationSettings.DefaultValueHandling <- DefaultValueHandling.Populate

[<Test>] // OK
let ``Serialize V1 + Deserialize V3 with simple Default`` () =
  let tenant : TenantV1 = { Name = "Doe" }
  let json = JsonConvert.SerializeObject(tenant, serializationSettings)
  let tenant = JsonConvert.DeserializeObject<TenantV3>(json, deserializationSettings)
  tenant.Name |> should equal "Doe"
  tenant.FirstName |> should equal "John"

[<Test>] // FAILS
let ``Serialize V1 + Deserialize V4 with complex Default`` () =
  let tenant : TenantV1 = { Name = "Doe" }
  let json = JsonConvert.SerializeObject(tenant, serializationSettings)
  let tenant = JsonConvert.DeserializeObject<TenantV4>(json, deserializationSettings)
  tenant.Name |> should equal "Doe"
  tenant.Gender |> should equal Other // NULL

Answer 1

While the obvious problem is versioning, there could be other problems. JSON is human readable, and a human might actually edit it and change something in a way to invalidate it (or your assumptions about it). I love doing a one line call to JsonConvert.DeserializeObject<>() but sometimes this is just not enough. I was storing user settings in JSON, and I did exactly as you suggest, "register some functions that provide default values for given types during deserialization".

So, I have a record type "UserSettings" which has a lot of different kind of values. I don't explicitly attach a version number, as that's not really needed to handle missing members. But obviously the version is changing anytime I add a new setting.

For each member where a default returned by the JSON deserializer is not acceptable, I define a function that will handle providing a value, and can also, if needed, do validation.

I deserialize the JSON text twice (though I only read it from disk once). First I safely (well, not completely safely, see below) deserialize into a dictionary, then collect those keys so I can later lookup whether a value is coming from the deserializer because it found it, or because it provided a default value (since it was not present).

Then I deserialize a second time, this time to an instance of UserSettings. Then I construct a new instance of UserSettings based on the one that the deserializer gave me, but for each member I call (if needed), the special coercion function for that member.

I've created a chopped down example, but it's still so long that I will use a gist.

https://gist.github.com/jimfoye/8a5e99291e863f55d0b1f3b351f50e6d

Note that sometimes a default returned by the deserializer is OK, sometimes it's not, sometimes the value needs to be range checked, or post processed, etc. Just put whatever you need in the function that handles that member.

All that code, and what happens if (for example) a value that is defined as bool has been changed by the user to something that can't be parsed as a bool? That will screw up the second call to DeserializeObject(). You could bulletproof this further by trying type conversions on the string values returned by the first call to DeserializeObject(). But I decided to just punt in this case (I trap for the exception, obviously, and just return a complete set of defaults).

Answer 2

Cribbing a bit from another related question:

Discriminated Unions compile to objects in IL, so you can't put them into attributes. You can put enums into attributes, though, since they're constants (they're numbers at run-time).

How can I use an F# discriminated union type as a TestCase attribute parameter?

Thus, I've gotten your tests to pass with the following tiny change.

type Gender = 
    | Female =0
    | Male =1
    | Other=2