Different System.Text.Json serialization for different versions

[steveoh]

I have a .net framework api with 1 version. I am converting that .net framework api to dotnet 7 and adding a second version to fix some of the goofy functionality of the existing api. But I need to recreate some of the goofy functionality so the dotnet 7 code can seamlessly replace the framework code.

In the framework api, json.net was used and camelCase property names are littered all over the response models. If a request has a specific query string value passed in, then a response delegating handler sets some items in a dictionary and those keys were left in title case while the rest of the response was camelCase. e.g.

{
 "result": {
  "attributes": {
   "Location": {
    "x": 425608.67349758215,
    "y": 4513499.5088544935
   },
   "Score": 98.24,
   "Locator": "AddressPoints.AddressGrid",
   "MatchAddress": "326 E SOUTH TEMPLE ST, SALT LAKE CITY",
   "InputAddress": "326 east south temple, salt lake city",
   "AddressGrid": "SALT LAKE CITY",
   "ScoreDifference": -1,
   "Wkid": 26912,
   "Candidates": []
  },
  "geometry": {
   "x": 425608.67349758215,
   "y": 4513499.5088544935,
   "type": "point",
   "spatialReference": {
    "wkid": 26912
   }
  }
 },
 "status": 200
}

There are other dictionaries on other routes that are formatted based on input request parameters so the formatting has to be configurable per route and per version.

I can't figure out how to configure STJ to be aware of the version so I can replicate this dictionary key behavior. I can create a JsonNamingPolicy and set it on the DictionaryKeyPolicy but the policy isn't version or route aware.

Do you have any suggestions for how versioning can influence STJ serialization?

I'm currently reading this blog about using multiple serializer settings. But this situation is so specific to one route, on a specific version, when an certain input parameter is used.

I read some microsoft advice that creating a serializer per request really impacts performance. So try to reuse them.

This stack overflow talks about converting all my routes to return a JsonResult with settings specific to the route. This could work because the route is aware of the version and can set different settings. Does this fall into the trap of not reusing the serializer?

I know this is kind of a versioning question and kind of not a versioning question but I hope we can still have a discussion about different ways to solve this.

[commonsensesoftware]

This type of scenario makes a lot of sense. I don't get a ton of questions about it so I don't know if people have figured out there own devices or what. A custom InputFormatter and OutputFormatter seems to be the right choice to me.

The blog post you referenced has a lot of the mechanics you need. Rather than using a magic string key, you can just rely on the API version that's included in the request. The condition should be something like:

public override bool CanRead(InputFormatterContext context) =>
    base.CanRead(context) &&
    context.HttpContext.GetRequestedApiVersion() is ApiVersion apiVersion &&
    apiVersion == ApiVersion.Default;

public override bool CanWriteResult(OutputFormatterCanWriteContext context) =>
    base.CanWriteResult(context) &&
    context.HttpContext.GetRequestedApiVersion() is ApiVersion apiVersion &&
    apiVersion == ApiVersion.Default;

This should work for both input and output. It will be organically paired to the API version mapping you've already configured. At the point in the pipeline where a formatter comes into place, an API version should be set and available. The one and only exception is if the API is version-neutral, but then there's no real way to distinguish different APIs. That seems to not a problem you would face.

ApiVersion.Default is 1.0, but you can initialize an alternate version in your formatter constructor. That's where I would start.

[steveoh]

I'm glad it makes sense. It feels silly to have to deal with but it's a real world example. I implemented the blog post solution and it seems to be providing the expected results. I'll tidy it up to work on the version rather than an attribute like you suggested.

To sum it all up, I would use the extension method on IMvcBuilder to register another serializer option. This still requires a magic key though right?

.AddJsonOptions("1", options => {})

The public class ConfigureMvcJsonOptions : IConfigureOptions<MvcOptions> then gets those settings from the AddJsonOptions using the magic key and passes them through to an input and output formatter which is inserted into the formatter pipelines.

The input and output formatters check the http context for the version number and compare it to the default version which I can change in the formatter itself.

Do I have that all correct?

[commonsensesoftware]

That's pretty close. I think you will, or will at least want, a separate IConfigureOptions<MvcOptions>. You don't need the magic strings though. You know all the variations in JSON options and they are only useful in the formatters so using keyed services isn't helping much. If you really want or needed keyed services, then consider using the string value of the API version (e.g. ApiVersion.ToString()) for the key.

With a small refactoring, you can create a simple specification using a Delegate like so:

public class VersionedJsonInputFormatter : SystemTextJsonInputFormatter
{
    private readonly Func<ApiVersion, bool> isSatisifiedBy;

    public VersionedJsonInputFormatter(
        Func<ApiVersion, bool> isSatisifiedBy,
        JsonOptions options,
        ILogger<SpecificSystemTextJsonInputFormatter> logger)
        : base(options, logger) => this.isSatisifiedBy = isSatisifiedBy;

    public override bool CanRead(InputFormatterContext context) =>
        base.CanRead(context) &&
        context.HttpContext.GetRequestedApiVersion() is ApiVersion apiVersion &&
        isSatisifiedBy(apiVersion);
}

public class VersionedJsonOutputFormatter : SystemTextJsonOutputFormatter
{
    private readonly Func<ApiVersion, bool> isSatisifiedBy;

    public VersionedJsonOutputFormatter(
        Func<ApiVersion, bool> isSatisifiedBy,
        JsonSerializerOptions jsonSerializerOptions)
        : base(jsonSerializerOptions) => this.isSatisifiedBy = isSatisifiedBy;

    public override bool CanWriteResult(OutputFormatterCanWriteContext context) =>
        base.CanWriteResult(context) &&
        context.HttpContext.GetRequestedApiVersion() is ApiVersion apiVersion &&
        isSatisifiedBy(apiVersion);
}

Now putting it all together, you just need a configuration that adds formatters for your initial API version (ex: 1.0) and one that handles everything else.

public class ConfigureMvcJsonOptions : IConfigureOptions<MvcOptions>
{
    private readonly IOptionsMonitor<JsonOptions> jsonOptions;
    private readonly ILoggerFactory loggerFactory;

    public ConfigureMvcJsonOptions(
        IOptionsMonitor<JsonOptions> jsonOptions,
        ILoggerFactory loggerFactory)
    {
        this.jsonOptions = jsonOptions;
        this.loggerFactory = loggerFactory;
    }

    public void Configure(MvcOptions options)
    {
        var legacyOptions = new JsonSerializerOptions();

        // TODO: configure legacy JSON options

        var options = jsonOptions.Get(Options.DefaultName);
        var logger = loggerFactory.CreateLogger<VersionedJsonInputFormatter>();

        // add legacy formatters
        options.InputFormatters.Insert(
            0,
            new VersionedJsonInputFormatter(
                static apiVersion => apiVersion == ApiVersion.Default,
                legacyOptions,
                logger));

        options.OutputFormatters.Insert(
            0,
            new VersionedJsonOutputFormatter(
                static apiVersion => apiVersion == ApiVersion.Default,
                legacyOptions.JsonSerializerOptions));

        // add current formatters (at index 0 to make sure they are first)
        options.InputFormatters.Insert(
            0,
            new VersionedJsonInputFormatter(
                static apiVersion => apiVersion > ApiVersion.Default,
                jsonOptions,
                logger));

        options.OutputFormatters.Insert(
            0,
            new VersionedJsonOutputFormatter(
                static apiVersion => apiVersion > ApiVersion.Default,
                options.JsonSerializerOptions));
    }
}

You can change the specification to suite your needs. You could also add additional formatters for all the variations you need. Formatter selection is always O(n) so order the most heavily used formatters first.

[steveoh]

Thank you so much for this. This looks like an awesome pattern.

I'm getting a bit into the weeds but the input formatter expects a JsonOptions class and it has an empty constructor and the settings property only has a getter. Is there some factory method, or class that implements JsonOptions or otherwise for creating that?

I can probably get away with using the default one for the input or simply remove the input formatters since this is a one way street. But I'm still curious how someone would set the input formatter JsonOptions now.

[commonsensesoftware]

Hmm... I didn't realize that. That's unfortunate. It seems you have to create the JsonOptions and then tweak all default settings back to the way you want. Something like:

var legacyOptions = new JsonOptions();

// TODO: set all legacy JSON options, including reverting any unwanted defaults
legacyOptions.JsonSerializerOptions.AllowTrailingCommas = false;

// ...

The .NET 7 configuration looks like this:

https://github.com/dotnet/aspnetcore/blob/release/7.0/src/Mvc/Mvc.Core/src/JsonOptions.cs#L34

I noticed that it has changed a little bit over time. You may want to explicitly configure everything; otherwise check earlier versions in the release/6.0 (etc) branch or .NET 8+ in the main branch.

[steveoh]

I need to remember it's easy to go find the source instead of relying on onmisharp to show me the metadata.

[steveoh]

@commonsensesoftware am I able to get the HttpContext or ApiVersion information in a TypeInfoResolver Modifier?

I'd like to make a contract that is based on a [JsonIgnoreInVersion(2)] attribute.

[AttributeUsage(AttributeTargets.Property | AttributeTargets.Field, AllowMultiple = false)]
public class JsonIgnoreInVersionAttribute : JsonAttribute {.
    public JsonIgnoreInVersionAttribute(int version) {
        Version = version;
    }

    public int Version { get; }
}

var quirkOptions = new JsonSerializerOptions() {
    ...
    TypeInfoResolver = new DefaultJsonTypeInfoResolver {
        Modifiers = {
            new JsonIgnoreInVersionModifier(
	            // seems it can only be passed in here?
                _log.CreateLogger<JsonIgnoreInVersionModifier>()
            ).IgnorePropertiesConverter
         }
    }
};

[steveoh]

I meant the contract for the JsonSerializer. It's likely I am confused. But I thought the contract was modified once per type and cached.

The System.Text.Json library constructs a JSON contract for each .NET type, which defines how the type should be serialized and deserialized. The contract is derived from the type's shape, which includes characteristics such as its properties and fields and whether it implements the IEnumerable or IDictionary interface. Types are mapped to contracts either at run time using reflection or at compile time using the source generator.

My concerns is that v1 comes in and the contract is created at runtime and cached. A v2 request comes in and the type contract created by a v1 request would be used?

[commonsensesoftware]

I think I understand what you're trying to do. I think it may be too granular. I'm not sure you can or would want to make decisions about specific JSON attributes at runtime. What you want is a set of options for each model for each version. That shouldn't require Reflection or runtime inspection. You should be able to build those configurations explicitly or, if you wanted to be really fancy, you could create your own source code generator. The different options configurations can be hid behind a factory.

interface IJsonSerializerOptionsFactory
{
    JsonSerializerOptions GetOptions(Type contract, ApiVersion apiVersion);
}

The implementation would contain all your configurations and put it in the DI container. At runtime, you can use in the formatters with something like:

var apiVersion = httpContext.GetRequestedApiVersion()!;
var options = factory.GetOptions(modelType, apiVersion);
var model = await JsonSerializer.DeserializeAsync(modelType, options, cancellationToken);

and

var apiVersion = httpContext.GetRequestedApiVersion()!;
var options = factory.GetOptions(model.GetType(), apiVersion);
await JsonSerializer.SerializeAsync(httpContext.Response.Stream, model.GetType(), options, cancellationToken);

There's plenty of variations to the approach, but you want map Type+ApiVersion to get a specific set of JSON options. That should be possible to do at compile time since you know all of the combinations. You might even have to take what the default JSON source code generator does and tweak that. It isn't going to be API version aware.

[steveoh]

I really like this idea. Remove all the program AddJsonOptions and move everything into a factory which is resolved and used somewhere where the ApiVersion will always be available.

I have a combination of JsonConverterFactories, DefaultJsonTypeInfoResolver modifiers, the versioned output formatters, and some if version checks in some models because the differences between the outputs are so specific. It all seems very messy. It will be a wonderful day when the v1 api can be deprecated... after I retire probably.

So, since I'm a little dense, where are you suggesting the serialize and deserialize code goes? I'm guessing a custom JsonConverter in the read and write methods?

[commonsensesoftware]

A JsonConverter would work. You might also be able to get away with some trickery by using something like a TypeDelegator or custom TypeInfo that will combine the hash code of the Type and ApiVersion. That would prevent the JsonSerializer from caching a single instance for a specific Type. There is some other way to filter or control the behavior through the JsonSerializerOptions, that would probably be the way to go.

[commonsensesoftware]

"Simpler" is all relative. 😉

Using the Adapter pattern is probably the simplest, but likely the most annoying (can't we do better? 🤔). For example, if your model is:

public class Person
{
    public string FirstName { get; set; }
    public string LastName { get; set; }
    public string Email { get; set; }
}

Let's say that 1.0 doesn't have the Email member. You could then have:

internal sealed class PersonV1Adapter
{
    private readonly Person person;

    public PersonV1Adapter(Person person) => this.person = person;

    public string FirstName
    {
        get => person.FirstName;
        set => person.FirstName = value;
    }

    public string LastName
    {
        get => person.LastName;
        set => person.LastName = value;
    }
}

This will solve a lot of the different issues with type mapping for the serialization/deserialization. I believe this should still work with source code generators too. How you integrate it can vary, but the simplest implementation might be something like:

[ApiVersion(1.0)]
[Route("[controller]")]
public class PeopleController : ControllerBase
{
    private readonly IRepository<Person> repository;

    public PeopleController(IRepository<Person> repository) => this.repository = repository;

    [HttpGet]
    public async IAsyncEnumerable<PersonV1Adapter> Get(CancellationToken cancellationToken)
    {
        await foreach (var person in repository.GetAllAsync(cancellationToken))
        {
            yield return new PersonV1Adapter(person);
        }
    }

    [HttpGet("{id}")]
    public async Task<IActionResult> Get(string id, CancellationToken cancellationToken)
    {
        if (await repository.GetOneAsync(id, cancellationToken) is Person person)
        {
            return Ok(new PersonV1Adapter(person));
        }

        return NotFound();
    }
}

There's several variations that can be added or used. The adapter types don't need to be public. You could still use a factory to map ApiVersion to adapter, but the return type would probably just object. That's a bit strange for your code (though you shouldn't care), but that should match up to formatters without issue.

[steveoh]

I just switched to minimal api's which undid all of this work. Are minimal api's ready for this kind of serialization @commonsensesoftware?

[commonsensesoftware]

To achieve this with Minimal APIs, you would need to use custom results or tweak the response as shown in the documentation:

Configure JSON serialization options for an endpoint

It might look something like:

app.MapGet("/", (HttpContext httpContext, IJsonSerializerOptionsFactory factory) =>
{
    var apiVersion = httpContext.GetRequestedApiVersion()!;
    var options = factory.GetOptions(typeof(Todo), apiVersion);
    return Results.Json(new Todo { Name = "Walk dog", IsComplete = false }, options));
});

You can make it more succinct if you enable binding for ApiVersion like so:

services.AddApiVersioning().EnableApiVersionBinding();

This method will hopefully go away sometime in the future. It is required because you have to opt into specific DI features due to limitations for parameter binding in Minimal APIs. This should be updated in .NET 8 or beyond. The deprecation path will be a major version as a no-op and then finally it will probably just go away two or three major versions from now. Under the hood, it's the same lookup as the long form above, but more succinct. It would look like this:

app.MapGet("/", (IJsonSerializerOptionsFactory factory, ApiVersion apiVersion) =>
{
    var options = factory.GetOptions(typeof(Todo), apiVersion);
    return Results.Json(new Todo { Name = "Walk dog", IsComplete = false }, options));
});

In addition to this, you can still use adapters, etc if you want to. The overall process isn't all that wildly different. The key difference is that you don't go through a formatter abstraction.