DEV Community: João Antunes

Mapping ASP.NET Core minimal API endpoints with C# source generators

João Antunes — Tue, 31 Jan 2023 17:30:00 +0000

Intro

This will be a very simple post about how we can use C# source generators to map minimal API endpoints automagically.

The obvious way to do this automatic endpoint mapping, would be to do reflection based assembly scanning, which is a very common approach to do these kinds of things, like registering services, which, for example, we can do pretty easily with libraries like Scrutor.

Although reflection based assembly scanning works, there’s just a little something we can do better: performance. Reflection isn’t the fastest thing, so if we can avoid it and just have things put in place at compile time, there’s one less thing slowing down our application’s startup. Additionally, .NET Native AOT and reflection might not go too well together (still early days though), so it’s good to already start looking at possibilities.

I’m pretty late to the C# source generator party, but hey, better late than never 😅. Not only am I pretty late, but I’m pretty sure what I’m talking about in this post, has already been talked loads of times, but I wanted to try things out for myself, so, like many other of my posts, if for no one else, it’ll be relevant for future me 🙃.

One final note on the post, is that the code you’ll was just to try things out, so things should be improved for actual production use (e.g. better type safety and avoiding finding things with hardcoded strings). But that’s hopefully something you already assume when looking at code in blogs 🙂.

The API and hooking points

Let’s start with the API (which is as basic as it can be), as well as, more importantly, the integration points put in place for the source generator to hook into.

For starters, we have an IEndpoint interface, exposing a single static abstract method to implement (new feature in C# 11). This will allow the source generator to look for all implementations of the interface.

namespace Api;

public interface IEndpoint
{
    static abstract IEndpointRouteBuilder Map(IEndpointRouteBuilder endpoints);
}

Then, we have the most basic hello world implementation:

namespace Api;

public class HelloEndpoints : IEndpoint
{
    public static IEndpointRouteBuilder Map(IEndpointRouteBuilder endpoints)
    {
        endpoints.MapGet("/", () => "Hello World!");
        return endpoints;
    }
}

The whole point of this post, is that we don’t want to call HelloEndpoints.Map manually, so instead, in the Program.cs, we call a method RegisterEndpoints, which will be implemented by the source generator, to map all endpoints:

using Api;

var builder = WebApplication.CreateBuilder(args);
var app = builder.Build();

app.RegisterEndpoints();

app.Run();

This RegisterEndpoints method is defined as a partial extension method in a partial EndpointRegistrationExtensions class.

namespace Api;

public static partial class EndpointRegistrationExtensions
{
    public static partial IEndpointRouteBuilder RegisterEndpoints(this IEndpointRouteBuilder endpoints);
}

The importance of this class and method being partial, is that then the source generator can generate code for the same class, actually implementing the method. Source generators cannot modify existing code, only add more code, so this is a way to leave our type open for extensibility, allowing our source generator to contribute to it.

Bootstrap the source generator

I’m just going to quickly skim past this one, as this is one of the first things in the docs: we need to create a .NET Standard 2.0 class library, with references to the Microsoft.CodeAnalysis.CSharp and Microsoft.CodeAnalysis.Analyzers packages.

My Generators.csproj looks like the following:

<Project Sdk="Microsoft.NET.Sdk">

    <PropertyGroup>
        <TargetFramework>netstandard2.0</TargetFramework>
        <ImplicitUsings>enable</ImplicitUsings>
        <LangVersion>preview</LangVersion>
        <Nullable>enable</Nullable>
        <WarningsAsErrors>nullable</WarningsAsErrors>
    </PropertyGroup>

    <ItemGroup>
        <PackageReference Include="Microsoft.CodeAnalysis.CSharp" Version="4.3.1" PrivateAssets="all" />
        <PackageReference Include="Microsoft.CodeAnalysis.Analyzers" Version="3.3.3">
            <PrivateAssets>all</PrivateAssets>
            <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
        </PackageReference>
    </ItemGroup>

</Project>

There’s some extra stuff I didn’t mention (like nullable and language version bits), but that’s just about some of my preferences, not really source generator related.

With the project in place, we can reference it from the API project, like so:

<Project Sdk="Microsoft.NET.Sdk.Web">

    <PropertyGroup>
        <TargetFramework>net7.0</TargetFramework>
        <Nullable>enable</Nullable>
        <WarningsAsErrors>nullable</WarningsAsErrors>
        <ImplicitUsings>enable</ImplicitUsings>
    </PropertyGroup>

    <ItemGroup>
        <ProjectReference Include="..\Generators\Generators.csproj" OutputItemType="Analyzer" ReferenceOutputAssembly="false"/>
    </ItemGroup>

</Project>

Then we can create our source generator class. This class should be decorated with the Generator attribute, and inherit from ISourceGenerator, which will provide us a couple of methods to implement.

using Microsoft.CodeAnalysis;

namespace Generators;

[Generator]
public class EndpointRegisterExtensionsGenerator : ISourceGenerator
{
    public void Initialize(GeneratorInitializationContext context)
    {
        throw new NotImplementedException();
    }

    public void Execute(GeneratorExecutionContext context)
    {
        throw new NotImplementedException();
    }
}

Collecting required information

Before generating the endpoint mapping code, the source generator needs to do two things: find the EndpointRegistrationExtensions class we want to extend, as well as find all implementations of IEndpoint, so we can map them.

There are a couple of ways to do this: in the execute method, using the context parameter go through the all nodes and find what we want; implement an ISyntaxReceiver that we configure to be invoked by the runtime for each node it finds. I’m not sure I have a preference between them at this point, so for no particular reason, I went with the latter. If you want to learn more, Khalid wrote a post on the subject.

The ISyntaxReceiver implementation, which I named Collector and is an internal class of the source generator, looks like this:

private class Collector : ISyntaxContextReceiver
{
    private readonly List<ClassDeclarationSyntax> _endpoints = new();
    private ClassDeclarationSyntax? _partial;

    public IReadOnlyCollection<ClassDeclarationSyntax> Endpoints => _endpoints;

    public ClassDeclarationSyntax Partial
        => _partial ?? throw new InvalidOperationException("Could not collect partial class to implement");

    public void OnVisitSyntaxNode(GeneratorSyntaxContext context)
    {
        if (context.Node is not ClassDeclarationSyntax @class) return;

        if (@class.Identifier.ValueText == "EndpointRegistrationExtensions")
        {
            _partial = @class;
        }
        else
        {
            var classSymbol = context.SemanticModel.GetDeclaredSymbol(@class);
            if (classSymbol?.AllInterfaces.Any(i => i.ToDisplayString().EndsWith("IEndpoint")) ?? false)
            {
                _endpoints.Add(@class);
            }
        }
    }
}

As you can see, it’s not something particularly esoteric. We check if the node in the context is a class, then check if it’s one of the two things we’re looking for: the partial class we’ll extend, or an implementation of IEndpoint (yes, doing hardcoded string comparison isn’t a good idea, I warned you at the beginning of the post 😜). We then expose this information in properties for the source generator to use.

To configure the Collector to be used, we implement the source generator Initialize method like so:

[Generator]
public class EndpointRegisterExtensionsGenerator : ISourceGenerator
{
    private readonly Collector _collector = new();

    public void Initialize(GeneratorInitializationContext context)
        => context.RegisterForSyntaxNotifications(() => _collector);

        // ...
}

Generating the code

With the bulk of the work done, all that’s left is to generate the code. We do this in the source generator Execute method, which looks like the following:

[Generator]
public class EndpointRegisterExtensionsGenerator : ISourceGenerator
{
    // ...

    public void Execute(GeneratorExecutionContext context)
    {
        // Retrieve the populated receiver
        var collector = (Collector)context.SyntaxContextReceiver!;

        var @namespace = GetNamespace(collector.Partial);

        var endpointRegistrations = new StringBuilder();

        foreach (var endpointClass in collector.Endpoints)
        {
            endpointRegistrations.AppendLine($"{endpointClass.Identifier.ValueText}.Map(endpoints);");
        }

        var source = // lang=C#
            $$""""
// <auto-generated/>

namespace {{@namespace}}; 

public static partial class EndpointRegistrationExtensions
{
    public static partial IEndpointRouteBuilder RegisterEndpoints(this IEndpointRouteBuilder endpoints)
    { 
        {{endpointRegistrations}}
        return endpoints;
    }
}
"""";

        context.AddSource(
            $"{nameof(EndpointRegisterExtensionsGenerator)}.generated.cs", 
            SourceText.From(source, Encoding.UTF8));
    }

    private static string GetNamespace(SyntaxNode? node)
        => node switch
        {
            NamespaceDeclarationSyntax namespaceNode => namespaceNode.Name.ToString(),
            FileScopedNamespaceDeclarationSyntax fileScopedNamespaceNode => fileScopedNamespaceNode.Name.ToString(),
            { } => GetNamespace(node.Parent),
            _ => throw new InvalidOperationException("Could not find namespace")
        };

    // ...
}

Let’s go step by step, they should be mostly self explanatory from the code.

We start by casting the context’s SyntaxtContextReceiver property to our Collector, so we can access our collected data.

Then, we grab the namespace in which the EndpointRegistrationExtensions lives, so we add the partial to the same one.

We then compose the lines of code invoking the Map method on all IEndpoint implementations we found.

With these things ready, we compose the final code, which is just a plain string. The code is again using some recent C# features, namely raw string literals, to make it much more readable (plus that // lang=C# comment, is a JetBrains Rider feature that enables syntax highlighting within the string 🤯). This syntax highlighting is a bit messed up in the blog, as my blog engine doesn’t understand C# 11 🙃.

Finally, we invoke context.AddSource to add our code to the solution.

With all of this in place (assuming I didn't forget any step while writing this post), we can now run our API and everything will work as we hoped 🙂.

Outro

That’s it for this quick look at how we can implement automatic discovery and mapping of minimal API endpoints with C# source generators.

As I mentioned, this was a quick and dirty test of how things could be done, so the code would need some extra effort to be a bit more production ready.

In any case, it was a good exercise to have a better understanding of how source generators work and how we can use them to solve some common problems.

Relevant links:

Thanks for stopping by, cyaz! 👋

Beware of records, with expressions and calculated properties

João Antunes — Thu, 01 Sep 2022 17:30:00 +0000

Intro

I’ve been using C# records a lot since the feature was introduced, probably too much 😅.

Given their terseness, even ignoring the other records characteristics, as soon as I’m creating a type that I’m expecting to be immutable, I immediately start with a record. However this might not always be adequate, and in this post we’ll look at a very specific example where I introduced a bug in the code due to not taking into consideration all of the records characteristics.

A record with a calculated property

Let’s take the following record as an example:

public record SomeRecordWithCalculatedProperty(string SomeValue)
{
    public string SomeCalculatedValue { get; } = SomeValue + " *calculated*";
}

What’s going on here, is that we’re using the primary constructor to declare and initialize the SomeValue property, but we also have another property, SomeCalculatedProperty, that’s calculated using SomeValue. Seems simple enough, right?

Now let’s take a look at the following usage of this record:

var x = new SomeRecordWithCalculatedProperty("This is some value");
Console.WriteLine(x);
var y = x with {SomeValue = "This is another some value"};
Console.WriteLine(y);

So, we’re creating a record, printing it to the console, then using the with expression to create a copy of that record, changing the SomeValue property.

Now give it a think for a while, what do you think will be printed to the console?

It’s the following (new lines added for readability):

SomeRecordWithCalculatedProperty 
{ 
    SomeValue = This is some value,
    SomeCalculatedValue = This is some value *calculated*
}
SomeRecordWithCalculatedProperty 
{
    SomeValue = This is another some value, 
    SomeCalculatedValue = This is some value *calculated* 
}

Can you spot the problem?

SomeCalculatedValue in the second line still has the same value as in the first. This makes sense, as what happens when we use the with expression, is that the record is cloned and then the properties provided within the brackets are overwritten.

This shouldn’t be a surprise, but again, given I was just blindly using records for the terseness, it did, in fact, get my surprise when I was trying to figure out the issue 🙂.

Calculating the property on the fly

An immediate and easy way to fix this, is to, instead of calculating the property value and setting it, just calculate it on the fly, like so:

public record SomeRecordWithOnTheFlyCalculatedProperty(string SomeValue)
{
    public string SomeCalculatedValue => SomeValue + " *calculated*";
}

This solves the problem, and might be a valid solution in general. It has one potential issue though: every time we use SomeCalculatedValue, like the method that the property get accessor is, the value will be calculated. Depending on the way the property is used, it might not be a problem, or it might be, as we’re always repeating the same logic and creating new objects to return. In my case, I was using the property multiple times, and the calculation logic was a bit more complex than this simplified example, so it would be nice to avoid executing this code more often than actually needed.

What if we make it lazy

Spoiler alert, you’ll probably see the issue as soon as I drop the code snippet, but, what if we make it lazy, using a Lazy<T> type? The code would look the following:

public record SomeRecordWithLazilyCalculatedProperty(string SomeValue)
{
    private readonly Lazy<string> _someCalculatedValue = new(() => SomeValue + " *calculated*");

    public string SomeCalculatedValue => _someCalculatedValue.Value;
}

What do you think, is this the solution to our problems? Give it a thought before looking the the output below 😉.

SomeRecordWithLazilyCalculatedProperty
{
    SomeValue = This is some value,
    SomeCalculatedValue = This is some value *calculated*
}
SomeRecordWithLazilyCalculatedProperty
{
    SomeValue = This is another some value,
    SomeCalculatedValue = This is some value *calculated*
}

Yup, it’s the same. The problem is the same, but this time, instead of copying the property, it was the backing field that was copied.

Just if you’re curious, using a decompilation tool (I used sharplab.io), we can see the generated copy constructor, which takes care of the object cloning:

protected SomeRecordWithLazilyCalculatedProperty(
[System.Runtime.CompilerServices.Nullable(1)] SomeRecordWithLazilyCalculatedProperty original)
{
    <SomeValue>k__BackingField = original.<SomeValue>k__BackingField;
    _someCalculatedValue = original._someCalculatedValue;
}

So, if the lazy field is copied, which was already initialized as we had already printed the first record instance, its contents are the same in the new cloned instance.

Overriding the copy constructor

Let’s take another stab at fixing the issue. Let’s keep the lazy field, but we’ll override the copy constructor.

A copy constructor is a constructor that gets an instance of the same type as the sole parameter. It should be protected when the record isn’t sealed, private otherwise.

public record YetAnotherRecordWithLazilyCalculatedPropertyAndCopyCtor(string SomeValue)
{
    private readonly Lazy<string> _someCalculatedValue = new(() => SomeValue + " *calculated*");

    public string SomeCalculatedValue => _someCalculatedValue.Value;

    protected YetAnotherRecordWithLazilyCalculatedPropertyAndCopyCtor(
        YetAnotherRecordWithLazilyCalculatedPropertyAndCopyCtor original)
    {
        SomeValue = original.SomeValue;
        _someCalculatedValue = new(() => SomeValue + " *calculated*");
    }
}

As we can see, we’re overriding the copy constructor, and when the time comes to initialize _someCalculatedValue, instead of copying the value from the original, we’re creating a new instance (but the record definition isn’t looking so terse now 😭).

Does this solve our problem? The output is:

YetAnotherRecordWithLazilyCalculatedPropertyAndCopyCtor
{
    SomeValue = This is some value,
    SomeCalculatedValue = This is some value *calculated*
}
YetAnotherRecordWithLazilyCalculatedPropertyAndCopyCtor
{
    SomeValue = This is another some value,SomeCalculate
    dValue = This is another some value *calculated*
}

It does! Because we create a new instance of the lazy backing field, the value is only calculated when needed, which in this case is when we print to the console, having the new SomeValue in place.

Note that this wouldn’t work if it wasn’t for the lazy, as at the time the copy constructor is invoked, we still don’t know what the new SomeValue will be.

Other options

There are, of course, other options to solve this issue.

We could, for example, declare SomeValue outside of the primary constructor and make it have a getter only, instead of the getter and init setter that’s generated when using the primary constructor. The side effect would be that we no longer can use a with expression to clone the record and change SomeValue.

We could also manually implement the init setter of SomeValue, so it would force recalculation of SomeCalculatedValue.

I imagine there are more options that I’m not remembering right now, but you get the point.

Outro

To summarize, I’ve misused records and got myself head-scratching while trying to figure out what was going wrong. To be clear, it was a mess of my own doing, I’m not blaming the feature at all, but thought I might not be the only one caught off guard with something like this, so thought of sharing this post 🙂.

At the end of the day, just keep in mind the characteristics of records, how they differ from classes, and use the right tool for the job, don’t be like me and get bedazzled by one of the features, ignoring everything else that comes along with it.

Thanks for stopping by, cyaz! 👋

[Video] Outbox meets change data capture - hooking into the Write-Ahead Log (feat. .NET, PostgreSQL & Kafka)

João Antunes — Wed, 27 Jul 2022 18:45:00 +0000

Hey folks! 👋

📢📽️ New one sent right into the tubes!

Another video on using change data capture (often referred to as CDC), to hook up into the database transaction log, forwarding incoming entries to the outbox table.

Previously, we used Debezium for this, but I was left wondering, what if I want to implement something similar with .NET? Turns out it's not super hard, with the help of Npgsql, which provides us with APIs to hook into PostgreSQL Write Ahead Log, so we can read the incoming outbox messages and forward them to Kafka.

Here are a couple of related links:

Thanks for stopping by, cyaz! 👋

[Video] What's the point of async/await in an ASP.NET Core application?

João Antunes — Mon, 30 May 2022 17:00:00 +0000

Hey folks! 👋

📢 New one available through the tubes!

I’ve been noticing in recent chats that folks have difficulty explaining why do we use async/await, tasks and related, when building ASP.NET Core applications.

I hope that this video, very high-level and narrowly focused by design, can help in increasing the understanding of the importance of using these features.

Here are a couple of related links:

Thanks for stopping by, cyaz! 👋

[Video] Quick shout-out to DevToys

João Antunes — Tue, 03 May 2022 16:50:00 +0000

Hey folks!

Wanted to very briefly share a tool I’ve come across a couple of months ago: DevToys.

It centralizes in one app a bunch of tools commonly useful when developing software. It’s unfortunately Windows only (at least for now), but it’s pretty cool and you should give it a go if you work on Windows.

Additionally, it’s open source and built with .NET, so you can also contribute if you’re up for it!

For non-Windows alternatives check out the links below.

Here are a couple of related links:

Thanks for stopping by, cyaz! 👋

[Video] Polymorphic JSON Serialization (feat. .NET & System.Text.Json)

João Antunes — Wed, 13 Apr 2022 18:45:00 +0000

Hey folks!

Time for one more video on messing with System.Text.Json, this time to get polymorphic (de)serialization going.

This is a simple approach, that can certainly be further extended and improved, but has been working well in the projects I needed it.

Here are a couple of related links:

Thanks for stopping by, cyaz! 👋

OpenAPI extensions and Swashbuckle

João Antunes — Mon, 21 Feb 2022 17:25:00 +0000

Intro

Time for another quick post 🙂.

This time, let’s take a quick look at OpenAPI extensions (which I discovered were a thing last week 😛) and how to add them to our API description using ASP.NET Core and Swashbuckle.

What are OpenAPI extensions

OpenAPI extensions are what you’re maybe already inferring from the name, a way to extend an API description, with custom properties, to be able to describe things that aren’t supported by the OpenAPI specification.

These custom properties names need to be prefixed with x- (e.g. x-your-property), but only at the root level, i.e. if this property is an object, the object’s properties don’t need the prefix.

Context

As I mentioned earlier, I just discovered OpenAPI extensions was a thing, as I never really studied the OpenAPI specification in depth, have been learning as I need things, and I had never needed this before.

I came across this, as I’m currently working with Google Cloud, and am using the API Gateway in front of Cloud Run hosted APIs.

When setting things up, noticed there was a need to add a x-google-backend property to the API description.

The following, is an example from Google’s documentation.

# openapi2-run.yaml
swagger: '2.0'
info:
  title: API_ID optional-string
  description: Sample API on API Gateway with a Cloud Run backend
  version: 1.0.0
schemes:
- https
produces:
- application/json
x-google-backend:
  address: APP_URL

# ...

My first reaction was... WTF is this?! 🤣

Having never noticed this before, thought this was Google coming up with things because of reasons, and my first thought was just edit the downloaded Swashbuckle generated description and add the thing there ad-hoc.

Of course, this isn’t a great to do things, so I started looking into ways to add custom stuff using Swashbuckle. Mind you, given I didn’t know about OpenAPI extensions, also didn’t know the right terms to use in the search, but at some point, after scouring a bunch of blog posts and Stack Overflow entries, found out what was the name of what I needed, so from then on, things got easier 🙂.

Making it work with Swashbuckle

So, how do we do this with Swashbuckle? Not too difficult really (after you know what you’re looking for, of course 😛).

To add this x-google-backend property, we can create a document filter (class implementing IDocumentFilter) and add an extension to the API description document.

internal class GoogleOpenApiExtensions : IDocumentFilter
{
    public void Apply(OpenApiDocument swaggerDoc, DocumentFilterContext context)
    {
        swaggerDoc.AddExtension(
            "x-google-backend",
            new OpenApiObject
            {
                ["address"] = new OpenApiString("http://some.backend.url")
            });
    }
}

Then, if we look at the generated document, we can see it there (side note, this is based on ASP.NET Core’s web template).

{
  "openapi": "3.0.1",
  "info": {
    "title": "OpenApiExtensions",
    "version": "v1"
  },
  "paths": {
    "/": {
      "get": {
        "tags": [
          "OpenApiExtensions"
        ],

    // ...

  },
  "components": { },
  "x-google-backend": {
    "address": "http://some.backend.url"
  }
}

OpenAPI extensions are not supported only at document level, so we could add them in other places as well, like operations or schemas.

A non-sensical example could be the following:

internal class SomeOtherOpenApiExtensions : IOperationFilter, ISchemaFilter, IParameterFilter, IRequestBodyFilter
{
    public void Apply(OpenApiOperation operation, OperationFilterContext context)
    {
        operation.AddExtension(
            "x-some-operation-extension",
            new OpenApiObject
            {
                ["random-operation-metadata-array"] = new OpenApiArray
                {
                    new OpenApiString("awesome-operation"),
                    new OpenApiInteger(9000)
                },
                ["some-boolean"] = new OpenApiBoolean(true)
            });
    }

    public void Apply(OpenApiSchema schema, SchemaFilterContext context)
    {
        schema.AddExtension("x-some-schema-extension", new OpenApiString("hello!"));
    }

    public void Apply(OpenApiParameter parameter, ParameterFilterContext context)
    {
        // parameter.AddExtension ...
    }

    public void Apply(OpenApiRequestBody requestBody, RequestBodyFilterContext context)
    {
        // requestBody.AddExtension ...
    }
}

Which would result in the following document:

{
  "openapi": "3.0.1",
  "info": {
    "title": "OpenApiExtensions",
    "version": "v1"
  },
  "paths": {
    "/": {
      "get": {
        "tags": [
          "OpenApiExtensions"
        ],
        "responses": {
          "200": {
            "description": "Success",
            "content": {
              "text/plain": {
                "schema": {
                  "type": "string",
                  "x-some-schema-extension": "hello!"
                }
              }
            }
          }
        },
        "x-some-operation-extension": {
          "random-operation-metadata-array": [
            "awesome-operation",
            9000
          ],
          "some-boolean": true
        }
      }
    }
  },
  "components": { },
  "x-google-backend": {
    "address": "http://some.backend.url"
  }
}

Outro

As I said in the beginning, quick post, so it’s done! 🙂

We took a quick look at what are OpenAPI extensions, one example of how they can be used to include extra information about an API that isn’t part of the spec, as well as how to work with them using ASP.NET Core and Swashbuckle.

Links in the post:

Thanks for stopping by, cyaz! 👋

Array or object JSON deserialization (feat. .NET & System.Text.Json)

João Antunes — Mon, 31 Jan 2022 17:25:00 +0000

Intro

Ah, the joys of integrating with third-party APIs... We always end up having to hammer something to get things working 🤣.

This is a post about one of such situations, resorting to some JSON deserialization trickery (via JsonConverter) to be able to get things working.

Problem statement

So, I’m integrating with this API, which in a specific endpoint, for some reason, returns an object in which one of its properties, when empty, is an empty array, but when there’s data, it’s an object with an items property which in turn is the array.

Some examples, for better understanding:

When collection is empty:

{
    "someItems": []
}

When collection has entries:

{
    "someItems": {
        "items": [
            {
                "id": 123,
                "text": "some text"
            },
            {
                "id": 456,
                "text": "some more text"
            }
        ]
    }
}

Let’s just say that the deserializer wasn’t happy when this happened 🙂.

So, how do we get out of this mess? Enter JsonConverters.

Side note: I reported this behavior as a bug, which will be fixed at some point, but until then, I still need to get things working.

Implementing a JsonConverter

Let’s jump right into the code!

public class ArrayOrObjectJsonConverter<T> : JsonConverter<IReadOnlyCollection<T>>
{
    public override IReadOnlyCollection<T>? Read(
        ref Utf8JsonReader reader,
        Type typeToConvert,
        JsonSerializerOptions options)
        => reader.TokenType switch
        {
            JsonTokenType.StartArray => JsonSerializer.Deserialize<T[]>(ref reader, options),
            JsonTokenType.StartObject => JsonSerializer.Deserialize<Wrapper>(ref reader, options)?.Items,
            _ => throw new JsonException()
        };

    public override void Write(Utf8JsonWriter writer, IReadOnlyCollection<T> value, JsonSerializerOptions options)
        => JsonSerializer.Serialize(writer, (object?) value, options);

    private record Wrapper(T[] Items);
}

Fortunately, as we can see, it’s not that much code, so we’ll go through it quickly.

We’re inheriting from JsonConverter<IReadOnlyCollection<T>>, to implement the JSON converter for that specific type (I normally use IReadOnlyCollection when passing collections around instead of IEnumerable, to be sure the collection isn’t lazy unless I really want it to be).

As for the read implementation, we check what the first token of the object’s JSON representation is:

Array start token ([) - we deserialize it as an array
Object start token ({) - we deserialize it as the Wrapper type declared below, which fits the structure of the objects we’re receiving
Another thing - that’s unexpected, so blowing things up 💥

Note that the usage of T[] is important. If we used IReadOnlyCollection<T>, we’d get into an infinite loop, with the converter basically calling itself again and again. I used T[] here, but it could also be another type of collection, just can’t be IReadOnlyCollection, to avoid the loop.

Regarding writing, it wasn’t important in my case, as at least for now I only need to deserialize things, but anyway, created a simple implementation where it’s always serialized as a JSON array. We could easily adjust it a bit if we wanted to serialize in different ways, depending on some logic.

Again, note that the cast to object? is important, for the same reason we used T[] earlier, we end up in a loop because the value is of type IReadOnlyCollection<T>. Casting it like this, the serializer will care for the underlying type, being oblivious to the original type of the object reference. We could achieve the same in some other ways, like casting to IEnumerable<T>, or removing the cast altogether and pass the generic parameter like JsonSerializer.Serialize<object?>(writer, value, options).

Outro

That’s it for this quick post.

We saw how to make use of JsonConverters to customize (de)serialization of specific types, in this case to workaround a bug in an API, but it’s also useful for a lot of other situations.

I’ve been dabbling with JsonConverters a bunch lately, so I’ll probably share one or two more of these posts, to document my shenanigans.

Fortunately, it ended up being less complicated than I expected, so I was happy not to waste too much time hammering things to hide bugs.

Links in the post:

Thanks for stopping by, cyaz! 👋

[Video] Outbox meets change data capture (feat. .NET, PostgreSQL, Kafka and Debezium)

João Antunes — Mon, 17 Jan 2022 17:30:00 +0000

Hey folks!

Quick video with an interesting approach to implement the publisher part of the outbox pattern.

Using change data capture (often referred to as CDC), we hook up something to the database transaction log, forwarding incoming entries to the outbox table.

In this example, we’ll make use of Debezium, hooked up into a PostgreSQL database, forwarding messages to Kafka.

Here come a bunch of related links:

Thanks for stopping by, cyaz! 👋

Getting a complex type as a simple type from the query string in a ASP.NET Core API controller

João Antunes — Mon, 03 Jan 2022 18:00:00 +0000

Intro

This is a tale of a good amount of hours of wasted time, so I’m going to document it so I remember it in the future 🙂.

One of these days, while implementing a web API and respective OpenAPI/Swagger documentation, I wanted to treat a complex type as a simple one, passing it in the query string. I had done this in the past, but getting things from the route and with no Swagger involved, so I assumed it would work the same. Apparently not, and hence the story begins.

Side note: I’m using ASP.NET Core controllers in this specific case, but I’ll leave some notes regarding doing the same with minimal APIs towards the end.

Prologue: why treat a complex type as a simple type

Before getting into how I got things working, it’s probably worth to mention why is it even relevant to go through these troubles. I won’t take too long explaining the reasons though, as you probably already heard this a bunch of times: avoiding primitive obsession.

Andrew Lock wrote a series on avoiding primitive obsession, with a focus on ids, but others, like Jimmy Bogard and Vladimir Khorikov also wrote about it.

Very briefly, the idea is to avoid using primitive types for everything, so we have types representing specific concepts, centralize logic specific to them, as well make things more type safe in general.

With this in mind, I have a type like the following:



public record struct SomeWrapperType(int Value)
{
    public int Value { get; } = Value;

    public static bool TryParse(string? value, out SomeWrapperType result)
    {
        if (int.TryParse(value, out var parsed))
        {
            result = new SomeWrapperType(parsed);
            return true;
        }

        result = default;
        return false;
    }
}

As you can see, the type itself is just a wrapper around an int (hence the name 🙂). In this case, I don’t have specific logic, just wanted to encapsulate the int value as a detail, so I can pass SomeWrapperType around the application.

We also have a TryParse method to help with creating an instance of the type from a string (similar idea to int.TryParse or Guid.TryParse).

First try: create and configure model binder

As I briefly mentioned, in the past I did something similar, but getting the parameter from the route, instead of the query string, plus, and more relevant, no Swagger was involved. I assumed it would work the same, so tried the exact same strategy: created and configured a model binder.

As we’ll soon find out, not only didn’t it work as well as I hoped, but it’s even overkill, but we’ll get there. Before that, let’s quickly see the model binding bits.

Starting with the model binder, we have:



public class SomeWrapperTypeModelBinder : IModelBinder
{
    public Task BindModelAsync(ModelBindingContext bindingContext)
    {
        if (bindingContext == null)
        {
            throw new ArgumentNullException(nameof(bindingContext));
        }

        var modelName = bindingContext.ModelName;

        var valueProviderResult = bindingContext.ValueProvider.GetValue(modelName);

        if (valueProviderResult == ValueProviderResult.None)
        {
            return Task.CompletedTask;
        }

        var value = valueProviderResult.FirstValue;

        if (bindingContext.ModelType == typeof(SomeWrapperType)
            && SomeWrapperType.TryParse(value, out var result))
        {
            bindingContext.Result = ModelBindingResult.Success(result);
        }
        else
        {
            bindingContext.Result = ModelBindingResult.Failed();
        }

        return Task.CompletedTask;
    }
}

A bunch of boiler plate, but you can find the relevant part towards the end, in the last if, where it’s using the SomeWrapperType.TryParse to try to get an instance of the type from the provided string.

Additionally we need a model binder provider, to hook things up with ASP.NET Core:



public class SomeWrapperTypeModelBinderProvider : IModelBinderProvider
{
    public IModelBinder? GetBinder(ModelBinderProviderContext context)
    {
        if (context == null)
        {
            throw new ArgumentNullException(nameof(context));
        }

        if (typeof(SomeWrapperType).IsAssignableFrom(context.Metadata.ModelType))
        {
            return new BinderTypeModelBinder(typeof(SomeWrapperTypeModelBinder));
        }

        return null;
    }
}

And finally, configure it:



builder.Services.AddControllers(options =>
{
    options.ModelBinderProviders.Add(new SomeWrapperTypeModelBinderProvider());
});

The controller action, looks like this:



[HttpGet("basic")]
public IActionResult GetWithNoCustomization(SomeWrapperType wrapper)
    => Ok(wrapper.Value);

If we run this now, what do we get?

So... something’s not great... not only is it not being treated as a simple type, but it’s showing up as part of the body, which in a GET request, is unexpected at best.

Part of it makes sense, as I added a model binder, but as we’re finding out, that doesn’t really contribute to the metadata required to generate the OpenAPI/Swagger bits, so it still shows up like that.

If we look at the available schemas, we see SomeWrapperType in there, further showing that it’s being treated as a complex type.

Add type specific Swagger configuration

So, if the problem seems to be lack of metadata for docs generation, let’s try to add some.

When configuring Swashbuckle (which is the library I’m using here for Swagger stuff), we can add info about specific types. In this case, we can do the following:



builder.Services.AddSwaggerGen(options =>
{
    // ...
    options.MapType<SomeWrapperType>(() => new OpenApiSchema { Type = "string" });
});

With this in place, SomeWrapperType should be treated as a simple type, in this case, like a regular string. Let’s see what happened in the Swagger UI:

So, we have improvements, as now the type is being shown as string (and no longer shows up in the schemas section), but it’s still showing up in the request body.

Add FromQuery attribute

Things showing up in the request body that shouldn’t, should be as simple as adding the [FromQuery] to the parameter, right? Let’s try it!



[HttpGet("from-query")]
public IActionResult GetWithFromQuery([FromQuery]SomeWrapperType wrapper)
    => Ok(wrapper.Value);

Almost, but we’re still not there!

As we can see, we now don’t have a body, the parameter moved to the query string, but as we further inspect, the parameter should be called wrapper and be a string (as we configured with Swashbuckle). Unfortunately, we see the name Value and type int, both referring to SomeWrapperType's internal property.

Enter type converters

At this point, I was getting annoyed and going further down rabbit holes and model binding shenanigans, but remembered to check Andrew Lock’s series of posts on strongly typed ids, to check if he touched on this, which fortunately for my sanity, he did (minus Swagger specifics).

And the answer is: type converters, which “Provides a unified way of converting types of values to other types, as well as for accessing standard values and subproperties.".

Going back to SomeWrapperType, did the following:



[TypeConverter(typeof(SomeWrapperTypeTypeConverter))]
public record struct SomeWrapperType(int Value)
{
    public int Value { get; } = Value;

    public static bool TryParse(string? value, out SomeWrapperType result)
    {
        if (int.TryParse(value, out var parsed))
        {
            result = new SomeWrapperType(parsed);
            return true;
        }

        result = default;
        return false;
    }

    private class SomeWrapperTypeTypeConverter : TypeConverter
    {
        public override bool CanConvertFrom(ITypeDescriptorContext? context, Type sourceType)
            => sourceType == typeof(string) || base.CanConvertFrom(context, sourceType);

        public override object? ConvertFrom(ITypeDescriptorContext? context, CultureInfo? culture, object value)
        {
            var stringValue = value as string;
            if (TryParse(stringValue, out var result))
            {
                return result;
            }

            return base.ConvertFrom(context, culture, value);
        }
    }
}

As we can see above, created a class inheriting from TypeConverter, implemented the conversion from string to SomeWrapperType, and finally used it with the TypeConverterAttribute.

We can delete the model binder we created earlier, as it won’t be relevant when we’re using the type converter (and by now you see why I said the model binder was overkill, this is simpler).

Going back to Swagger UI...

... and it finally works as desired (and the FromQuery attribute isn’t needed).

A quick note that, even if we could get rid of the model binding bits (other than the type converter), the Swashbuckle configuration is still required, otherwise it’ll still show up as a complex type in the docs, even though it works as expected on the implementation side.

Quick aside: it “just works” with minimal APIs

As a final piece of info, as I’ve also played with the new ASP.NET Core minimal APIs (which seem to be controversial for reasons that I don’t understand), everything would “just work”, without implementing most of the things described here.

Using this new approach, the existence of a TryParse method with a signature like I showed earlier, as well as a BindAsync method, are detected and used for parameter binding (more information on the official docs).



app.MapGet(
    "/sample/minimal",
    (SomeWrapperType wrapper) => Results.Ok(wrapper.Value));

Outro

That does it for this mythical journey, full of twists and turns, where at the end we find out, a straight path was available right in our grasp 🤣.

Hope this can help someone avoid my mistakes.

In summary, for a complex type to be treat as a simple one when getting its value from the query string:

For controllers, derive from TypeConverter, implement conversion logic and decorate the type with the TypeConverterAttribute.
For minimal APIs, TryParse and BindAsync will do the trick.
To make things look nice in the Swagger UI, use MapType to configure the schema when using Swashbuckle.

Links in the post:

Thanks for stopping by, cyaz!

[Video] Checking out NDepend

João Antunes — Mon, 06 Dec 2021 17:15:00 +0000

Hey folks!

Let’s take a look at something that’s been around for years, but maybe flew under the radar: NDepend, a static code analysis tool tailored for .NET.

It’s not free, but it might be worth checking out, to see if your project would benefit from it. You can try it with the available 14-day trial (which was what I used to test it and record this video).

A couple of related links:

Thanks for stopping by, cyaz!

[Video] High-performance and compile-time logging source generation in .NET 6

João Antunes — Sun, 21 Nov 2021 16:30:00 +0000

Hey folks!

Quick little video about an interesting new logging related feature coming in .NET 6, compile-time logging source generation, as well as related high-performance logging features, which were already present but probably not very well known.

Here come a bunch of related links:

Thanks for stopping by, cyaz!