net 8.0: migrating from serialization to source generators

Question

We're in the process of migrating an existing Blazor WASM (hosted on a ASP.NET Core App) to net 8.0 and we're having several issues with trimming (and how it works). One of them is ensuring that the trimmer doesn't remove the types used for interacting with the Web API consumed by the Blazor WASM App. Currently, serializing/deserializing relies on System.Text.Json (reflection base and no trimming). In theory, moving from reflection to source generators shouldn't be that hard. Unfortunately, it seems like this is one of those scenarios where theory doesn't quite match what happens in the real world.

For instance, suppose you've got the following helper generic type:

public sealed record Descriptor<T>(T Value, string Description);

In the project we're migrating, this type is used by several of the API endpoints for returning enum values/string pairs shown by several dropdowns that exist in different pages presented by the Blazor WASM UI.

Now, if I'm not mistaken, to serialize this type through source generation, then I must apply the JsonSerializableAttribute to a new partial JsonSerializerContext derived type. Futhermore, it seems like you can't specify an open generic type, so if you're using this generic type with enums Kind and KindB, then you'll have to at least have this:

[JsonSourceGenerationOptions(GenerationMode = JsonSourceGenerationMode.Serialization)]
[JsonSerializable(typeof(Descriptor<Kind>))]
[JsonSerializable(typeof(Descriptor<KindB>))]
public partial class DescriptorContext: JsonSerializerContext{}

Now, when you have several enum types, this isn't pretty. However, it seems like you also need add entries if you want to serialize collections of each type of descriptor:

[JsonSourceGenerationOptions(GenerationMode = JsonSourceGenerationMode.Serialization)]
[JsonSerializable(typeof(Descriptor<Kind>))]
[JsonSerializable(typeof(IEnumerable<Descriptor<Kind>>))]
[JsonSerializable(typeof(List<Descriptor<Kind>>))]
// removed entries for  KindB
public partial class DescriptorContext: JsonSerializerContext{}

In other words, without these entries, I'm unable to serialize/deserialize IEnumerable<Descriptor<Kind>> or List<Descriptor<Kind>>instances like the ones shown on the next example:

List<Descriptor<Kind>> items =
[
  new(Kind.Single, "Single"),
  new(Kind.Colective, "Colective")
];
string serialized = JsonSerializer.Serialize
(
  items,
  DescriptorContext.Default.ListDescriptorKind
);
List<Descriptor<Kind>> recovered = JsonSerializer.Deserialize
(
  serialized,
  DescriptorContext.Default.ListDescriptorKind
)!;

foreach (Descriptor<Kind> item in recovered)
{
  Console.WriteLine($"{item.Value} {item.Description}");
}

Is this correct? I mean, is it right to conclude that:

1. You can't specify an open type
2. You must specify all the types you want to serialize/deserialize through JsonSerializableAttribute, including closed generics like the one shown on the previous snippet (ex.: List<Descriptor<Kind>>)

I really hope I'm missing something here because if using source generators does require all these lines, then it's really hard to use them in any mid-size project... If you look at this example, I'd say that maintaining the JsonSerializableAttribute list is not simple.

The second issue we're facing is that we don't have the source code for the assembly which has the types used by the Web API that is consumed by the Blazor WASM app. Initially, Json.NET was used for serialization/deserialization and the API has several endpoints where the return type is a derived type of the type specified on the signature of the controller's method (polymorphism):

public async Task<ActionResult<Base>> DoSomething()
{
   // always returns an instance of a Base's derived type
}

Initially, serialization was done with Json.NET. However, we've ended up migrating to System.Text.Json when it officially supported inheritance. At the time, we've ended up resorting to custom DefaultJsonTypeInfoResolver so that the discriminator used is compatible with the one generated by default by Json.NET (property name is set to $type and its value is full assembly qualified name). This was required because the several of the types used in the API were also persisted to the database. Here's some demo code used for the DefaultJsonTypeInfoResolver:

public class JsonHierarchyTypeInfoResolver : DefaultJsonTypeInfoResolver
{
  public override JsonTypeInfo GetTypeInfo(Type type, JsonSerializerOptions options)
  {
    var jsonTypeInfo = base.GetTypeInfo(type, options);
    var jsonType = jsonTypeInfo.Type;

    if (jsonType == typeof(Equipamento))
    {
      jsonTypeInfo = GetTypeInfoForDevice(jsonTypeInfo);
    }
    // ... remaining code removed
  }

  private static JsonTypeInfo GetTypeInfoForDevice(JsonTypeInfo jsonTypeInfo)
  {
    jsonTypeInfo.PolymorphismOptions = new JsonPolymorphismOptions
    {
      TypeDiscriminatorPropertyName = "$type",
      IgnoreUnrecognizedTypeDiscriminators = true,
      UnknownDerivedTypeHandling = JsonUnknownDerivedTypeHandling.FailSerialization
    };

    foreach (var jsonDerivedType in _derivedDevicesTypes)
    {
      jsonTypeInfo.PolymorphismOptions.DerivedTypes.Add(
        new JsonDerivedType(jsonDerivedType,
          jsonDerivedType.AssemblyQualifiedName!));
    }

    return jsonTypeInfo;
  }
}

So, the final 2 questions are:

1. Is it possible to combine this resolver with the one generated by source generators? Or do they only work with polymorphic attributes (ex.: JsonDerivedTypeAttribute)
2. If no, then does it mean that I'll have to rewrite a new DTO assembly which uses polymorphic attributes in order to reuse the endpoint API based on polymorphism/inheritance (i.e., if I want the API's method signature to return A and then return A derived types)?

Answer 1

You have several related questions here, let's take them in order.

Do I need to add [JsonSerializable(typeof(T))] for every type I need to serialize -- including closed generics for every generic in which I might use T as a parameter, such as List<T> or IEnumerable<T>?

You must add [JsonSerializable(typeof(T))] for every root type you need to serialize. Thus you would need to add both of the following:

[JsonSerializable(typeof(IEnumerable<Descriptor<Kind>>))]
[JsonSerializable(typeof(List<Descriptor<Kind>>))]
public partial class DescriptorContext : JsonSerializerContext;

However, you don't need to add [JsonSerializable(typeof(T))] for non-root types -- i.e. the types of serialized members of the root types already added. Thus if you have added the above two includes, you actually don't need to add [JsonSerializable(typeof(Descriptor<Kind>))] or [JsonSerializable(typeof(Kind))] as well since these are already included by List<Descriptor<Kind>>. E.g. given the above context, you can already do:

var kindJson = JsonSerializer.Serialize(Kind.Single, DescriptorContext.Default.Kind);

But even if you only need to JsonSerializableAttribute for root types, this can indeed become quite burdensome for larger projects.

Is there any way to use an open generic type with JsonSerializableAttribute?

No, this is not implemented.

What's more, it might not be possible to implement. In Native AOT scenarios it may be impossible to call Type.MakeGenericType() to generate new closed types from open generic types precisely because, in the case when the generic parameters are value types, new code would need to be generated. For confirmation, see When is AOT + MakeGenericType safe?.

Is it possible to combine this resolver with the one generated by source generators?

Yes, you can use JsonTypeInfoResolver.Combine(IJsonTypeInfoResolver[]):

Combines multiple IJsonTypeInfoResolver sources into one.

I.e. you could do something like:

JsonSerializerOptions options = new()
{
    TypeInfoResolver = 
        JsonTypeInfoResolver.Combine(DescriptorContext.Default, new JsonHierarchyTypeInfoResolver()),
}

However combining resolvers doesn't allow you to modify the output of one using the other; instead the first resolver in the list to return a JsonTypeInfo "wins".

Do JsonSerializerContext resolvers only work with compile-time polymorphic attributes?

Luckily it is possible to modify the contracts returned by a source-generated context via a typeInfo modifier. However, be aware than JsonTypeInfoResolver.WithAddedModifier() returns a new IJsonTypeInfoResolver that encapsulates and adapts the existing resolver, rather than modifying the input resolver. (In fact JsonSerializerContext instances are immutable once constructed.) Thus you lose the convenient compile-time ability to use named JsonTypeInfo<T> properties like DescriptorContext.Default.ListDescriptorKind on your typed serialization context and must instead fetch an appropriate JsonTypeInfo<T> in runtime.

So if you want to add support for polymorphism in runtime to a source-generated context, first introduce the following generic utilities:

public static partial class JsonExtensions
{
    public const string NewtonsoftDiscriminator = "$type";
    public const string DefaultDiscriminator = "$type";
    
    public static JsonDerivedType ToNewtonsoftDerivedType(this Type type) => new JsonDerivedType(type, type.AssemblyQualifiedName!);
    
    public static Action<JsonTypeInfo> AddPolymorphismOptions(
        Type baseType, 
        IEnumerable<JsonDerivedType> derivedTypes,
        string typeDiscriminatorPropertyName = DefaultDiscriminator,
        bool ignoreUnrecognizedTypeDiscriminators = false,
        JsonUnknownDerivedTypeHandling unknownDerivedTypeHandling = JsonUnknownDerivedTypeHandling.FailSerialization) => (typeInfo) =>
    {
        // Adapted from https://stackoverflow.com/questions/77857543/how-can-i-add-jsonderivedtype-without-attributes-in-runtime-in-system-text-json
        if (baseType.IsSealed)
            throw new ArgumentException($"Cannot add JsonPolymorphismOptions to sealed base type {baseType.FullName}");
        if (typeInfo.Type.IsAssignableTo(baseType) && !typeInfo.Type.IsSealed)
        {
            typeInfo.PolymorphismOptions = new()
            {
                TypeDiscriminatorPropertyName = typeDiscriminatorPropertyName,
                IgnoreUnrecognizedTypeDiscriminators = ignoreUnrecognizedTypeDiscriminators,
                UnknownDerivedTypeHandling = unknownDerivedTypeHandling,
            };
            foreach (var derivedType in derivedTypes.Where(t => !t.DerivedType.IsAbstract && t.DerivedType.IsAssignableTo(typeInfo.Type)))
                typeInfo.PolymorphismOptions.DerivedTypes.Add(derivedType);         
        }
    };

    // API methods approved for .NET 11: https://github.com/dotnet/runtime/issues/118468
    // These will throw if the JsonTypeInfo<T> is not found.
    public static JsonTypeInfo<T> GetTypeInfo<T>(this JsonSerializerOptions options, T value) => 
        (JsonTypeInfo<T>)options.GetTypeInfo(typeof(T));
    public static JsonTypeInfo<T> GetTypeInfo<T>(this JsonSerializerOptions options) => 
        (JsonTypeInfo<T>)options.GetTypeInfo(typeof(T));    
}

Next, replace your JsonHierarchyTypeInfoResolver with some type that can discover and include the derived types of your base types (here Equipamento) such as:

public class JsonHierarchyTypeInfoModifier
{
    private JsonHierarchyTypeInfoModifier () { }
    readonly Lazy<IReadOnlyList<Type>> _derivedDevicesTypes = new(() => SearchDeviceDerivedTypes(typeof(Equipamento)).ToArray());
    public static JsonHierarchyTypeInfoModifier Instance { get; } = new();
    
    public Action<JsonTypeInfo> AddPolymorphismOptions() => (typeInfo) =>
    {
        JsonExtensions.AddPolymorphismOptions(
            typeof(Equipamento), 
            _derivedDevicesTypes.Value.Select(t => t.ToNewtonsoftDerivedType()),
            JsonExtensions.NewtonsoftDiscriminator,
            ignoreUnrecognizedTypeDiscriminators : true)(typeInfo);
        // ... remaining code removed
    };

    // TODO: Replace with your actual logic or use dependency injection
    static IEnumerable<Type> SearchDeviceDerivedTypes(Type baseType) => baseType.Assembly.GetTypes().Where(t => t.IsAssignableTo(baseType));
}

Then if you have some List<Equipamento> list that might contain instances of subtypes of Equipamento, you would serialize and deserialize as follows:

// Microsoft recommends caching and reusing options for best performance.
JsonSerializerOptions options = new()
{
    TypeInfoResolver = DescriptorContext.Default
        .WithAddedModifier(
            JsonHierarchyTypeInfoModifier.Instance.AddPolymorphismOptions()),
    // Add your other default options here:
    WriteIndented = true,
};

var json = JsonSerializer.Serialize(list, options.GetTypeInfo(list));

var list2 = JsonSerializer.Deserialize(json, options.GetTypeInfo<List<Equipamento>>());

Notes:

• JsonSourceGenerationMode.Serialization only supports serialization. For a list of limitations of this mode see Serialization-optimization (fast path) mode.

• See the related question Adding modifiers to System.Text.Json serialization when using source generation.