DEV Community: Maarten Balliauw

Discriminated Unions in C#

Maarten Balliauw — Mon, 18 Sep 2023 02:44:05 +0000

Discriminated unions have been a long-standing request for C#. While F# users have had discriminated unions for years, C# developers will have to wait a bit longer.

What discriminated unions allow you to do is tell the compiler (and other tooling like your IDE) that data can be one of a range of pre-defined types.

For example, you could have a method RegisterUser() that returns either a User, a UserAlreadyExists or InvalidUsername class. These classes don’t have to inherit from each other. You want to support 3 potential return types and tell the language about this, get compiler errors if you return a 4th type, and so on.

If you have used ASP.NET Core Minimal APIs, you may have seen the Results<> and TypedResults approach to return data from your API. Using this approach, you can define which object types may be returned from your API (using Results<>). Here’s a quick example of an API that can return an Ok or Unauthorized result.

app.MapGet("/items", async Task<Results<Ok<IEnumerable<ApiItem>>, Unauthorized>>(
  [FromRoute]int storeId,
  GroceryListDb db) => {
      // ... code here
      return TypedResults.Ok(items);
  });

The Results<> type essentially a discriminated union: the return value will be one of (in this case) two types, and the ASP.NET Core Minimal API engine can use that information to return the correct type.

Digging into the source code (and removing some ASP.NET Core-specifics), the Results class with support for 3 different types looks like this:

public sealed class Results<TResult1, TResult2, TResult3>
{
    private Results(object activeResult)
    {
        Result = activeResult;
    }

    public object Result { get; }

    public static implicit operator Results<TResult1, TResult2, TResult3>(TResult1 result) => new(result);

    public static implicit operator Results<TResult1, TResult2, TResult3>(TResult2 result) => new(result);

    public static implicit operator Results<TResult1, TResult2, TResult3>(TResult3 result) => new(result);
}

It should be quite straightforward to change this into a Results class that supports 2 types, or 5.

Using implicit operators, the Results class can be instantiated from any of the types that have supported conversions.

What’s cool is that you can drop this class into your own code, and use the Results class to have, for example, a method that can return either int, bool or string, but nothing else:

Results<int, bool, string> GetData() => "Hello, world!";

If you returned a type that is not supported, the IDE (and compiler) will tell you:

Even pattern matching is supported (if you do it on the property that holds the actual data):

var data = GetData();
var typeAsString = data.Result switch
{
    int => "int",
    bool => "bool",
    string => "string",
    _ => throw new NotImplementedException()
};

Console.WriteLine(typeAsString);

Results<int, bool, string> GetData() => "Hello, world!";

The downside however, is that when you’d change the GetData() method to return either of 4 types (instead of 3), you would not get a compilation error in the above switch expression. And let that be one of the advantages of discriminated unions: being able to get tooling support for these cases, informing you that you don’t have an exhaustive match on all types.

For ASP.NET Core Minimal APIs, the Results<> class works perfectly. It’s a discriminated union that only needs one side of the story (being able to get compiler errors when you return something you’re not supposed to). Consuming the result is part of the framework mechanics, and ideally you should never need to do an exhaustive comparison yourself.

If you’re outside ASP.NET Core Minimal APIs, you want to work with discriminated unions in your code, and you can’t wait for proper language support, there is good news for you! The OneOf package (docs) lets you work with discriminated unions, provides compiler errors when comparisons are not exhaustive, etc.

For me, the reason of writing this blog post was mainly that I wanted to show you the clever use of implicit operators in the Results<> class. I hope, however, that you got something more out of it as well: a short introduction to discriminated unions, and two alternatives (using F#, and the OneOf package) if you do want to use them in your code.

Running Large Language Models locally – Your own ChatGPT-like AI in C#

Maarten Balliauw — Thu, 15 Jun 2023 02:44:05 +0000

For the past few months, a lot of news in tech as well as mainstream media has been around ChatGPT, an Artificial Intelligence (AI) product by the folks at OpenAI. ChatGPT is a Large Language Model (LLM) that is fine-tuned for conversation. While undervaluing the technology with this statement, it’s a smart-looking chat bot that you can ask questions about a variety of domains.

Until recently, using these LLMs required relying on third-party services and cloud computing platforms. To integrate any LLM into your own application, or simply to use one, you’d have to swipe your credit card with OpenAI, Microsoft Azure, or others.

However, with advancements in hardware and software, it is now possible to run these models locally on your own machine and/or server.

In this post, we’ll see how you can have your very own AI powered by a large language model running directly on your CPU!

Towards open-source models and execution – A little bit of history…

A few months after OpenAI released ChatGPT, Meta released LLaMA. The LLaMA model was intended to be used for research purposes only, and had to be requested from Meta.

However, someone leaked the weights of LLaMA, and this has spurred a lot of activity on the Internet. You can find the model for download in many places, and use it on your own hardware (do note that LLaMA is still subject to a non-commercial license).

In comes Alpaca, a fine-tuned LLaMA model by Standford. And Vicuna, another fine-tuned LLaMA model. And WizardLM, and …

You get the idea: LLaMA spit up (sorry for the pun) a bunch of other models that are readily available to use.

While part of the community was training new models, others were working on making it possible to run these LLMs on consumer hardware. Georgi Gerganov released llama.cpp, a C++ implementation that can run the LLaMA model (and derivatives) on a CPU. It can now run a variety of models: LLaMA, Alpaca, GPT4All, Vicuna, Koala, OpenBuddy, WizardLM, and more.

There are also wrappers for a number of languages:

Let’s put the last one from that list to the test!

Getting started with SciSharp/LLamaSharp

Have you heard about the SciSharp Stack? Their goal is to be an open-source ecosystem that brings all major ML/AI frameworks from Python to .NET – including LLaMA (and friends) through SciSharp/LLamaSharp.

LlamaSharp is a .NET binding of llama.cpp and provides APIs to work with the LLaMA models. It works on Windows and Linux, and does not require you to think about the underlying llama.cpp. It does not support macOS at the time of writing.

Great! Now, what do you need to get started?

Since you’ll need a model to work with, let’s get that sorted first.

1. Download a model

LLamaSharp works with several models, but the support depends on the version of LLamaSharp you use. Supported models are linked in the README, do go explore a bit.

For this blog post, we’ll be using LLamaSharp version 0.3.0 (the latest at the time of writing). We’ll also use the WizardLM model, more specifically the wizardLM-7B.ggmlv3.q4_1.bin model. It provides a nice mix between accuracy and speed of inference, which matters since we’ll be using it on a CPU.

There are a number of more accurate models (or faster, less accurate models), so do experiment a bit with what works best. In any case, make sure you have 2.8 GB to 8 GB of disk space for the variants of this model, and up to 10 GB of memory.

2. Create a console application and install LLamaSharp

Using your favorite IDE, create a new console application and copy in the model you have just downloaded. Next, install the LLamaSharp and LLamaSharp.Backend.Cpu packages. If you have a Cuda GPU, you can also use the Cuda backend packages.

Here’s our project to start with:

With that in place, we can start creating our own chat bot that runs locally and does not need OpenAI to run.

3. Initializing the LLaMA model and creating a chat session

In Program.cs, start with the following snippet of code to load the model that we just downloaded:

using LLama;

var model = new LLamaModel(new LLamaParams(
    model: Path.Combine("..", "..", "..", "Models", "wizardLM-7B.ggmlv3.q4_1.bin"),
    n_ctx: 512,
    interactive: true,
    repeat_penalty: 1.0f,
    verbose_prompt: false));

This snippet loads the model from the directory where you stored your downloaded model in the previous step. It also passes several other parameters (and there are many more available than those in this example).

For reference:

n_ctx – The maximum number of tokens in an input sequence (in other words, how many tokens can your question/prompt be).
interactive – Specifies you want to keep the context in between prompts, so you can build on previous results. This makes the model behave like a chat.
repeat_penalty – Determines the penalty for long responses (and helps keep responses more to-the-point).
verbose_prompt – Toggles the verbosity.

Again, there are many more parameters available, most of which are explained in the llama.cpp repository.

Next, we can use our model to start a chat session:

var session = new ChatSession<LLamaModel>(model)
    .WithPrompt(...)
    .WithAntiprompt(...);

Of course, these ... don’t compile, but let’s explain first what is needed for a chat session.

The .WithPrompt() (or .WithPromptFile()) method specifies the initial prompt for the model. This can be left empty, but is usually a set of rules for the LLM. Find some example prompts in the llama.cpp repository, or write your own.

The .WithAntiprompt() method specifies the anti-prompt, which is the prompt the LLM will display when input from the user is expected.

Here’s how to set up a chat session with an LLM that is Homer Simpson:

var session = new ChatSession<LLamaModel>(model)
    .WithPrompt("""
        You are Homer Simpson, and respond to User with funny Homer Simpson-like comments.

        User:
        """)
    .WithAntiprompt(new[] { "User: " });

We’ll see in a bit what results this Homer Simpson model gives, but generally you will want to be more detailed in what is expected from the LLM. Here’s an example chat session setup for a model called “LocalLLM” that is helpful as a pair programmer:

var session = new ChatSession<LLamaModel>(model)
    .WithPrompt("""
        You are a polite and helpful pair programming assistant.
        You MUST reply in a polite and helpful manner.
        When asked for your name, you MUST reply that your name is 'LocalLLM'.
        You MUST use Markdown formatting in your replies when the content is a block of code.
        You MUST include the programming language name in any Markdown code blocks.
        Your code responses MUST be using C# language syntax.

        User:
        """)
    .WithAntiprompt(new[] { "User: " });

Now that we have our chat session, we can start interacting with it. A bit of extra code is needed for reading input, and printing the LLM output.

Console.WriteLine();
Console.Write("User: ");
while (true)
{
    Console.ForegroundColor = ConsoleColor.Green;
    var prompt = Console.ReadLine() + "\n";

    Console.ForegroundColor = ConsoleColor.White;
    foreach (var output in session.Chat(prompt, encoding: "UTF-8"))
    {
        Console.Write(output);
    }
}

That’s pretty much it. The chat session in the session variable is prompted using its .Chat() method, and all outputs are returned token by token, like any generative model.

You want to see this in action, right? Here’s the “Homer Simpson chat” in action:

The more useful “C# pair programmer chat”:

Pretty nice, no?

On my Windows laptop (i7-10875H CPU @ 2.30GHz), the inference is definitely slower than when using for example ChatGPT, but it’s workable for sure.

Wrapping up

Because of the hardware needs, using LLMs has always required third-party services and cloud platforms like OpenAI’s ChatGPT.

In this post, we’ve seen some of the history of open-source large language models, and how the models themselves as well as the surrounding community have made it possible to run these models locally.

I’m curious to hear what you will build using this approach!

Getting rid of warnings with nullable reference types and JSON object models in C#

Maarten Balliauw — Thu, 12 Jan 2023 02:44:05 +0000

In my blog series, Nullable reference types in C# - Migrating to nullable reference types, we discussed the benefits of enabling nullable reference types in your C# code, and annotating your code so the compiler and IDE can give you more reliable hints about whether a particular variable or property may need to be checked for being null before using it.

We ended the series with a curious case: how to annotate classes to deserialize JSON.

The issue is this: you’ll typically have several Data Transfer Objects (DTO)/Plain-Old CLR Objects (POCO) in your project that declare properties to deserialize the data into. You know for sure the data will be there after deserializing, so you declare these properties as non-nullable. Yet, the compiler (and IDE) insist on you either making it a nullable property or initializing the property.

How to go about that? There are several options, each with their own advantages and caveats. Let’s have a look.

Option 1: Make the property nullable

If you follow the compiler’s advice, you can update the property and make it nullable:

public class User
{
    [JsonProperty("name")]
    public string? Name { get; set; }
}

This will get rid of the warning, but you now have to check the Name property for potential null values everywhere it is used. If the JSON may contain null values, this is a great approach. However, when you know for sure there will always be a value, it adds a lot of overhead in your codebase.

Option 2: Add a `default!` (please don’t)

You could also keep the property as non-nullable, and initialize the property with default!. This effectively sets the default value to null but suppresses the warning.

public class User
{
    [JsonProperty("name")]
    public string Name { get; set; } = default!;
}

I highly recommend against doing this. If the deserialized JSON does not contain a value for the Name property, it will now hold a null value. The compiler and IDE are satisfied and will no longer warn you about this, meaning unexpected NullReferenceException may be thrown at runtime.

The goal of nullable reference types/nullable annotations is to provide you with a null safety net, and the above is sabotaging that safety net from the start.

Option 3: Add a primary constructor

If you’re using Newtonsoft.Json as your JSON framework of choice, you can add a primary constructor to your class that sets all non-nullable properties.

The JSON deserializer will pick this up, and calls the constructor instead of setting the properties directly:

public class User
{
    public User(string? name)
    {
        Name = name ?? "Unknown"; // or ArgumentNullException.ThrowIfNull(name)
    }

    [JsonProperty("name")]
    public string Name { get; init; }
}

What’s nice with this approach is that the nullability warning will be gone, and you’re modeling your C# representation very closely to the JSON you want to deserialize. If you’re certain no null will be in the JSON, a non-nullable property in C# makes sense.

In addition, you can either set a default value or throw an ArgumentNullException in the constructor. The last option may mean you’ll see an exception at runtime, but then that exception is there because the JSON data is not what you expected, and other action may be needed (such as logging an incident) instead of happily continuing to run your code.

Option 4: Annotations and default values

Instead of setting the property to default and suppressing the nullability warning, you can also set a proper default value. In the following example, the Name property is non-nullable and contains an expected default value when no value is deserialized from JSON:

public class User
{
    [JsonProperty("name")]
    public string Name { get; init; } = "Unknown";
}

If you’re using record classes, you can do this as well:

public record User(
    [property: JsonProperty("name")]
    string Name = "Unknown"
);

This is a really nice way to express classes that are just a representation of a JSON document.

Option 5: Use a `required` property

In C# 11, the required modifier was added as a way to indicate that a field or property must be initialized by all constructors or by using an object initializer.

Given the compiler expects the property to always be initialized and contain a value, this means the nullability warning is no longer there. It helps make sure your own code always has to initialize such properties, and that it’s safe to assume no null reference will be present at runtime.

public class User
{
    [JsonProperty("name")]
    public required string Name { get; set; }
}

Personally, I like this approach the most. It clearly sets expectations, without providing the compiler and IDE with false information.

Do keep in mind it is important that the JSON document you are deserializing always contains a value and is not null. The required modifier is enforced at compile time, and not at runtime. If a null reference is set by the JSON framework you are using, there’s no guarantee NullReferenceException can’t occur.

If you expect null in some cases, annotating the property as nullable (string?) and performing null checks where applicable is the recommended approach.

Mastodon on your own domain without hosting a server

Maarten Balliauw — Sat, 05 Nov 2022 02:44:05 +0000

Like many in the past week, I have been having a serious look at Mastodon as an alternative to Twitter.

Mastodon is a social network that is distributed across many servers that have their own smaller communities, and federate with other servers to provide a more “global” social network.

There are many servers out there that you can choose from. Alternatively, you can also self-host your Mastodon server, or use one of many hosted instances, “Mastodon as a service”.

In recent hours, I have seen many people wanting to host their own servers, which is great fun! Self-hosting also has the added benefit of being able to have a Mastodon account on your own domain, and you own your data.

Now, I don’t really care about that (yet?). I ran my own mail server back in the day and am very happy with someone running it for me now. The same goes with Mastodon: I trust the folks at Mastodon.online, the server I joined, to do a much better job at this than I will ever do.

However, there is one thing I would like my own server for: discoverability. Much like with e-mail, I want folks to have an easy address to find me, and one that I can keep giving out to everyone even if later I switch to a different Mastodon server. A bit like e-mail forwarding to your ISP’s e-mail service.

The good news is: you can use your own domain and share it with other folks. It will link to your actual account.

Go on, try it. Search for @maarten@balliauw.be, and you will find my @maartenballiauw@mastodon.social.

How to discover Mastodon account via custom domain

Reading “how to implement a basic ActivityPub server”, there are a couple of things that stand out:

Mastodon (and others) use ActivityPub as their protocol to communicate between “actors”.
Actors are discovered using WebFinger, a way to attach information to an email address, or other online resource.

Since discovery is what I was after, WebFinger seemed like the only thing I would need to implement.

WebFinger lives on /.well-known/webfinger on a server. For Mastodon, your server will be queried for accounts using an endpoint that looks like this:

GET /.well-known/webfinger?resource=acct:accountname@server

And indeed, if I look at my Mastodon server’s webfinger for my account, I get a response back!

GET https://mastodon.online/.well-known/webfinger?resource=acct:maartenballiauw@mastodon.online

{
  "subject": "acct:maartenballiauw@mastodon.online",
  "aliases": [
    "https://mastodon.online/@maartenballiauw",
    "https://mastodon.online/users/maartenballiauw"
  ],
  "links": [
    {
      "rel": "http://webfinger.net/rel/profile-page",
      "type": "text/html",
      "href": "https://mastodon.online/@maartenballiauw"
    },
    {
      "rel": "self",
      "type": "application/activity+json",
      "href": "https://mastodon.online/users/maartenballiauw"
    },
    {
      "rel": "http://ostatus.org/schema/1.0/subscribe",
      "template": "https://mastodon.online/authorize_interaction?uri={uri}"
    }
  ]
}

Sweet!

The next thing I tried was simply copy-pasting this JSON output to my own server under .well-known/webfinger, and things magically started working.

In other words, if you want to be discovered on Mastodon using your own domain, you can do so by copying the contents of https://<your mastodon server>/.well-known/webfinger?resource=acct:<your account>@<your mastodon server> to https://<your domain>/.well-known/webfinger.

One caveat: this approach works much like a catch-all e-mail address. @anything@yourdomain.com will match, unless you add a bit more scripting to only show a result for resources you want to be discoverable.

Bonus: Discovering folks from Twitter

Discoverability, at this stage, is one of the things that matter to get a proper social graph going. Over the past days, there were a couple of tools I found very useful in finding Twitter folks on Mastodon:

Twitodon learns about which Twitter account matches a Mastodon account, from folks using this service.
Fedifinder and Debirdify scan Twitter accounts and checks if there is a Mastodon account in their profile data.

Good luck! And give @maarten@balliauw.be a follow if you make the jump to Mastodon.

Edit: Seems there is a GitHub issue which requests custom domains as well.

Rate limiting in web applications - Concepts and approaches

Maarten Balliauw — Mon, 03 Oct 2022 02:44:05 +0000

Your web application is running fine, and your users are behaving as expected. Life is good!

Is it, though…? Users are probably using your application in ways you did not expect. Crazy usage patterns resulting in more requests than expected, request bursts when users come back to the office after the weekend, and more!

These unexpected requests all pose a potential threat to the health of your web application and may impact other users or the service as a whole. Ideally, you want to put a bouncer at the door to do some filtering: limit the number of requests over a given timespan, limiting bandwidth, …

Last week, I covered how to use the ASP.NET Core rate limiting middleware in .NET 7.

In this post, let’s take a step back and explore the simple yet wide realm of rate limiting. We’ll go over how to decide which resources to limit, what these limits should be, and where to enforce these limits.

As a (mostly) .NET developer myself, I’ll use some examples and link some resources that use ASP.NET Core. The general concepts however will also apply to other platforms and web frameworks.

Introduction to rate limiting

Before we dive into the details, let’s start with an introduction about why you would want to apply rate limiting, and what it is.

Why rate limiting?

Let’s say you are building a web API that lets you store todo items. Nice and simple: a GET /api/todos that returns a list of todo items, and a POST /api/todos and PUT /api/todos/{id} that let you create and update a specific todo item. What could possibly go wrong with using these three endpoints?

Off the top of my head:

The mobile app another team is building accidentally causes an infinite loop that keeps calling POST, and tries to create a new todo item 10.000 times over the course of a few seconds before it crashes. That’s a lot of todo items in the database that should not be there.
You depend on an external system that throttles you for a number of requests. Frequent requests from one user to your API result in reaching that external limit, making your API unavailable for all your users.
Someone is brute-forcing the GET method, trying to get todo items for all your users. You have security in place, so they will never get in without valid credentials, but your database has to run a query to check credentials 10 times per second. That’s rough on this small 0.5 vCPU database instance that seemed good on paper.
An aggressive search engine spider accidentally adding 20.000 items into a shopping cart that is stored in memory.

There are probably more things that could go wrong, but you get the picture. You, your team, or external factors may behave in ways you did not expect.

That profile picture upload that usually gets small images uploaded? Guaranteed someone will try to upload a 500MB picture of the universe at some point.

When you build an application, there’s a very real chance that you don’t know how it will be used, and what potential abuse may look like. You are sharing CPU, memory and database usage among your users. One bad actor, whether intentional or accidental, can break or make your application slow, spoiling the experience for other users.

What is rate limiting?

Rate limiting, or request throttling, is an approach to reduce the fall-out of unexpected or unwanted traffic patterns to your application.

Typically, web applications implement rate limiting by setting an allowance on the number of requests for a given timeframe. If you are a streaming service, you may want to limit the outgoing bandwidth per user over a given time. Up to you!

The ultimate goal of imposing rate limits is to reduce or even eliminate traffic and usage of your application that is potentially damaging. Regardless of the traffic being accidental or malicious.

What should I rate limit?

I will give you a quote that you can use in other places:

Rate limit everything.

– Maarten Balliauw

With everything, I mean every endpoint that uses resources that could slow down or break your application when exhausted or stressed.

Typically, you’ll want to rate limit endpoints that make use of the CPU, memory, disk I/O, the database, external APIs, and the likes.

Huh. That does mean everything, even your internal (health) endpoints! You’ll want to prevent resource exhaustion, and make usage of shared resources more fair to all your users.

Naive rate limiting

The title of this section already hints at it: don’t use the approach described in this section, but do read through it to get into the mindset of what we are trying to accomplish…

If you wanted to add rate limiting to your ASP.NET Core web application, how would you do it?

Most probably, you will end up with a solution along these lines:

Add a database table Events, with three columns:
- UserIdentifier – who do we limit
- ActionIdentifier – what do we limit
- When – event timestamp so we can apply a query
Implement a request delegate to handle rate limits

The request delegate could look something like the following, storing events and then counting the number of events over a period of time:

app.Use(async (http, next) =>
{
    var eventsContext = app.Services.GetRequiredService<EventsContext>();

    // Determine identifier
    var userIdentifier = http.User.Identity?.IsAuthenticated == true
        ? http.User.Identity.Name!
        : "anonymous";

    // Determine action
    var actionIdentifier = http.Request.Path.ToString();

    // Store current request
    eventsContext.Events.Add(new Event
    {
        UserIdentifier = userIdentifier,
        ActionIdentifier = actionIdentifier,
        When = referenceTime
    });
    await eventsContext.SaveChangesAsync();

    // Check if we are rate limited (5 requests per 5 seconds)
    var referenceTime = DateTime.UtcNow;
    var periodStart = referenceTime.AddSeconds(-5);
    var numberOfEvents = eventsContext.Events
        .Count(e => e.UserIdentifier == userIdentifier && e.ActionIdentifier == actionIdentifier && e.When >= periodStart);

    // Rate limited - respond 429 status code
    if (numberOfEvents > 5)
    {
        http.Response.StatusCode = 429;
        return;
    }

    // Not rate limited
    await next.Invoke();
});

That should be it, right? RIGHT?!?

Well… Let’s start with the good. This would be very flexible in defining various limits and combinations of limits. It’s just code, and the logic is up to you!

However, every request is at least 2 queries to handle potential rate limiting. The Events table will grow. And fast! So you will need to remove events at some point.

The database server will suffer at scale. Imposing rate limits to protect shared resources, has now increased the load on this shared resource!

Ideally, the measurements and logic for your rate limiting solution should not add this additional load. A simple counter per user identifier and action identifier should be sufficient.

Rate limiting algorithms

Luckily for us, smart people have thought long and hard about the topic of rate limiting, and came up with a number of rate limiting algorithms.

Quantized buckets / Fixed window limit

An easy algorithm for rate limiting, is using quantized buckets, also known as fixed window limits. In short, the idea is that you keep a counter for a specific time window, and apply limits based on that.

An example would be to allow “100 requests per minute” to a given resource. Using a simple function, you can get the same identifier for a specific period of time:

public string GetBucketName(string operation, TimeSpan timespan)
{
    var bucket = Math.Floor(
        DateTime.UtcNow.Ticks / timespan.TotalMilliseconds / 10000);

    return $"{operation}_{bucket}";
}

Console.WriteLine(GetBucketName("someaction", TimeSpan.FromMinutes(10)));
// someaction_106062120 <-- this will be the key for +/- 10 minutes

You could keep the generated bucket name + counter in a dictionary, and increment the counter for every request. based on the counter, you can then apply the rate limit. When a new time window begins, a new bucket name is generated and the counter can start from 0.

This bucket name + counter can be stored in a C# dictionary, or as a named value on Redis that you can easily increment (and expires after a specific time so Redis does the housekeeping for you).

There is a drawback to quantized buckets / fixed window limits… They are not entirely accurate.

Let’s say you want to allow “4 requests per 10 seconds”. Per 10-second window, you allow only 4 requests. If all of those requests come in at the end of the previous window and the start of the current window, there’s a good chance the expected limit is going to be exceeded.

The limit of 4 requests is true per fixed window, but not per sliding window…

Does this matter? As always, “it depends”.

If you want to really lock things down and don’t want to tolerate a potential overrun, then yes, this matters. If your goal is to impose rate limits to prevent accidental or intentional excessive resource usage, perhaps this potential overrun does not matter.

In the case where you do need a sliding window limit, you could look into sliding window limit approaches. These usually combine multiple smaller fixed windows under the hood, to reduce the chance of overrunning the imposed limits.

Token buckets

Widely used in telecommunications to deal with bandwidth usage and bandwidth bursts, are token buckets. Token buckets control flow rate, and they are called buckets because buckets and water are a great analogy!

“Imagine a bucket where water is poured in at the top and leaks from the bottom. If the rate at which water is poured in exceeds the rate at which it leaks water out, the bucket overflows and no new requests can be handled until there’s capacity in the bucket again.”

If you don’t like water, you could use tokens instead:

“Imagine you have a bucket that’s completely filled with tokens. When a request comes in, you take a token out of the bucket. After a predetermined amount of time, new tokens are added to the bucket. If you take tokens out faster than they are added, the bucket will be empty at some point, and no new requests can be handled until new tokens are added.”

In code, this could look like the following. The GetCallsLeft() method returns how many tokens are left in the bucket.

public int GetCallsLeft() {
  if (_tokens < _capacity) {
    var referenceTime = DateTime.UtcNow;
    var delta = (int)((referenceTime - _lastRefill).Ticks / _interval.Ticks);

    if (delta > 0) {
      _tokens = Math.Min(_capacity, _tokens + (delta * _capacity));
      _lastRefill = referenceTime;
    }
  }
  return _tokens;
}

One benefit of token buckets is that they don’t suffer the issue we saw with quantized buckets. If too many requests come in, the bucket overflows (or is empty if you prefer the water analogy) and requests are limited.

Another benefit is that they allow bursts in traffic: if your bucket allows for 60 tokens per minute (replenished every second), clients can still burst up to 60 requests for the duration of 1 second, and thereafter the flow rate becomes 1 request per second (because of this replenishment flow).

There are other variations of the algorithms we have seen, but generally speaking they will correspond to either quantized buckets or token buckets.

Deciding rate limits

Now that we have seen the basic concepts of rate limiting, let’s have a look at the decisions to be made before implementing rate limiting in your applications.

Which resources to rate limit?

Deciding which resources to rate limit is easy. Here’s a quote from a famous blog author:

Rate limit everything.

– Maarten Balliauw

Your application typically employs a “time-sharing model”. Much like a time-sharing vacation property, you don’t want your guests to be hindered by other guests, and ideally come up with a fair model that allows everyone to use the vacation property in a fair way.

Rate limiting should be applied to every endpoint that uses resources that could slow down or break your application when exhausted or stressed. Given every request uses at least the CPU and memory of your server, and potentially also disk I/O, the database, external APIs and more, you’ll want to apply rate limiting to every endpoint.

What are sensible limits?

Deciding on sensible limits is hard, and the only good answer here is to measure what typical usage looks like.

Measurement brings knowledge! A good approach to decide on sensible limits is to:

Find out the current # of requests for a certain resource in your application.
Implement rate limiting, but don’t block requests yet. When a limit is hit, log it. This will let you fine-tune the numbers.
Iterate on measurements and logs, and when you are certain you know what the limit should be, start enforcing it.

As an extra tip, make sure to constantly monitor rate limiting events, and adjust when needed. Perhaps a newer version of your mobile app makes more requests to your API, and this is expected traffic.

Too strict limits will annoy your users. Remember, you don’t want to police the number of requests. You want fair usage of resources. You don’t call the police when two toddlers fight over a toy. If they both need the toy, maybe it’s fine to have multiple toys or have them play at different times, so they don’t have to fight over it.

Will you allow bursts or not?

Depending on your application and endpoint, having one rate limit in place will be enough. For example, a global rate limit of 600 requests per minute may be perfect for every endpoint in your application.

However, sometimes you may want to allow bursts. For example, when your mobile app starts, it performs some initial requests in rapid succession to get the latest data from your API, and after that it slows down.

To handle these bursts, you may want to implement a “laddering” approach, and have multiple different limits in place:

Limit	Operation A	Operation B	Operation C
Per second	10	10	100
Per minute	60	60	500
Per hour	3600	600	500

In the above table, a client could make 10 requests per second to Operation A. 10 per second would normally translate to 36000 request per hour, but maybe at the hourly level, only 3600 is a better number.

Again, measure, and don’t prematurely add laddering. There’s a good chance a single limit for all endpoints in your application may be sufficient.

What will be the partition/identifier?

Previously, we used a user identifier + action/operation identifier to impose rate limits. There are many other request properties you can use to partition your requests:

Partition per endpoint
Partition per IP address
- Keep in mind users may be sharing an IP address, e.g. when behind a NAT/CGNAT/proxy
Partition per user
- Keep in mind you may have anonymous users, how will you distinguish those?
Partition per session
- What if your user starts a new session for every request?
- What if your user makes use of multiple devices with separate sessions?
Partition per browser
Partition per header (e.g. X-Api-Token), …

Also here, “it depends” on your application. A global rate limit per IP address may work for your application. More complex applications may need a combination of these, e.g. per-endpoint rate limiting combined with the current user.

Decide on exceptions

Should rate limiting apply to all requests? Well… yes! We already discussed all endpoints in your application should be rate limited.

A better question would be whether the same limits should apply for all types of users. As usual, the answer to this question will depend on your application. There is no silver bullet, but here are some examples to think about.

Good candidates to have different rate limits in place:

Your automated monitoring - the last thing you want is nightly PagerDuty alerts because of your monitoring system being rate limited.
- Counter point: maybe you do want to have a rate limit in place, so your monitoring can check rate limits are enabled?
Your admin/support team - your support team may make a lot of requests to your application to help out users, so it’s best to not get in their way.
- Counter point: maybe a rate limit does make sense, so a disgruntled employee can’t go and scrape lots of data or add swear words into lots of places with an automated script.
Web crawlers - your marketing folks won’t be happy if your app is not visible in search engines!
- Counter point: there are aggressive crawlers, and you also don’t want them to get in the way of your users. There are some robots.txt entries many spiders respect, but a rate limit could be needed.
“Datacenter IP ranges” - If you have a mobile app targeted at consumers, does traffic coming from AWS, Azure and other big hosters make sense? If you expect mostly “residential” and mobile traffic, perhaps you can reduce automated traffic from other sources with a more strict rate limit. There’s a list of datacenter IP ranges that you can use for this.
- Counter point: VPN providers out there are often going to be in these IP ranges. Legitimate users may use datacenter IP addresses in those cases. Also at the time of writing, my dad’s Starlink subscription runs over what looks like a Google Compute Engine IP address.

An additional exception could be certain groups of customers. If your API is your product, it could be part of your business model to allow e.g. users of your “premium plan” to have different limits.

Also here, measuring will help you make an informed decision. If you see excess traffic from web crawlers, a tighter rate limit may be needed. If you see your support folks unable to help users, maybe a less strict rate limit for them makes more sense.

Responding to limits

What should happen when a request is being rate limited? You could “black hole” the request and silently abort it, but it’s much nicer to communicate what is happening, and why.

One example I like is StackOverflow. When using their website and posting responses to many questions in rapid succession, there’s a good chance their rate limiter may ask you to prove you are human:

This is pretty slick. Potential issues with a broken application posting multiple answers rapidly are avoided by rate limiting. Potential scripts and bots will also be rate limited, and their service happily hums along.

Another good example is GitHub. First of all, they document their rate limits so that you can account for these limits in any app you may be building that uses their API. Second, any request you make will get a response with information about how many requests are remaining, and when more will be available:

$ curl -I https://api.github.com/users/octocat
> HTTP/2 200
> Date: Mon, 01 Jul 2013 17:27:06 GMT
> x-ratelimit-limit: 60
> x-ratelimit-remaining: 56
> x-ratelimit-used: 4
> x-ratelimit-reset: 1372700873

In addition, when a rate limit is exceeded, you’ll get a response that says what happened and why, and where to find more information.

> HTTP/2 403
> Date: Tue, 20 Aug 2013 14:50:41 GMT
> x-ratelimit-limit: 60
> x-ratelimit-remaining: 0
> x-ratelimit-used: 60
> x-ratelimit-reset: 1377013266

> {
> "message": "API rate limit exceeded for xxx.xxx.xxx.xxx. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)",
> "documentation_url": "https://docs.github.com/rest/overview/resources-in-the-rest-api#rate-limiting"
> }

If you have mixed types of users, you could inspect the Accept header and return different responses based on whether text/html is requested (likely a browser) and when application/json is requested (likely an API client).

Other services have documented their limits as well. For example, NuGet lists limits for each endpoint and also shows you what the response would look like when a limit is reached.

Try and always communicate why a client is being limited, and when to retry. A link to the documentation may be enough.

This is of course not mandatory, but if you’re offering an API to your users, it does help in providing a great developer experience.

429, 403, 503, … status codes

From the GitHub example, you may have seen the status code returned when rate limits are exceeded is 403 (Forbidden). Other services return a 503 (Service unavailable), and others return a 429 status code (Too Many Requests).

There’s no strict rule here, but it does look like many services out there follow a convention of using 429 Too Many Requests.

Where to store rate limit data and counters?

When you search for information about rate limiting, there’s a good chance you’ll come across questions about where to store rate limit data and counters. More than once, you’ll see questions related to using your database, Redis or other distributed cached.

Keep it simple. Do you really need 100% accurate counters that all instances of your application share? Or is it enough to apply “10-ish requests per second per user” on every instance of your application and be done with it?

If your rate limit is part of your revenue model, for example when you sell API access with specific resource guarantees, then you’ll probably want to look into shared and accurate counters. When your goal is to ensure fair use of shared resources in your application, storing counters per instance may be more than enough.

Where to apply rate limiting?

In an ideal world, the consumer of your application would know about rate limits and apply them there, before even attempting a request. This would mean your server and application will never even have to process the request.

Unfortunately, we’re not living in an ideal world, and clients will send requests to your application. How far will you let traffic flow?

If you think of web-based applications (including APIs and the likes), there are several places where rate limits could be applied.

Maybe you are using a Content Delivery Network (CDN) that acts as a reverse proxy for your application, and they can rate limit? Or perhaps the framework you are using has some rate limiting infrastructure that can be used?

The closer to your application you add rate limiting, the more knowledge you will have about the user. If your partitioning requires deep knowledge about user privileges etc., your application may be the only place where rate limiting can be applied.

When you partition based on IP address and the Authentication header, a CDN or reverse proxy could handle rate limiting as they don’t need extra data for every request.

The closer to your application you add rate limiting, the more resources will be spent. If you’re running a serverless application and rate limit on a CDN or reverse proxy, you won’t be billed for execution of your serverless function. If you need more information about the user, then your serverless function may need to apply rate limiting (but also costs money).

Depending on what makes sense for your application, here are some resources:

In your application
- Stefan Prodan’s AspNetCoreRateLimit (highly recommend, it has lots of options for every aspect discussed in this blog post)
- ASP.NET Core Rate Limiting middleware in .NET 7
Reverse proxy
Content Delivery Network / API Gateway

Monitoring and circuit breakers

Applications change, usage patterns change, and as such, rate limits will also need to change. Perhaps your rules are too strict and hurting your users more than your application resources. Perhaps the latest deployment introduced a bug that is making excess calls to an API, and this needs to be fixed?

Keep an eye on your rate limiting, keep track of who gets rate limited, when and why. Use custom metrics to build dashboards on # of rate limiting actions kicking in to help during incident troubleshooting.

Also make sure you can adapt quickly if needed, by having circuit breakers in place. If with a new deployment all of your users experience rate limiting for some reason, having an emergency switch to just turn off rate limits will be welcome. Perhaps on/off is too coarse, and your circuit breaker could be in making rate limits dynamic and allowing for updates using a configuration file.

Wrapping up

Whether intentional or accidental, users of your application will bring along unexpected usage patterns. Excess requests, request bursts, automated scripts, brute-force requests - all of these are going to happen at some point.

These types of usage may pose a potential threat to your application’s health, and one abusive user could impact several others. Your application runs on shared resources, and ideally you want them to be shared in a fair manner.

This is where rate limiting comes in, and I hope I was able to give you a comprehensive overview of all the things you can and have to consider when implementing a rate limiting solution.

The concept of “it depends” definitely applies when building a rate limiting solution. Small and simple may be enough, and many of the considerations in this post will only apply for larger applications. But do consider to “rate limit everything” to make resource sharing more fair.

ASP.NET Core rate limiting middleware in .NET 7

Maarten Balliauw — Mon, 26 Sep 2022 02:44:05 +0000

Rate limiting is a way to control the amount of traffic that a web application or API receives, by limiting the number of requests that can be made in a given period of time. This can help to improve the performance of the site or application, and to prevent it from becoming unresponsive.

Starting with .NET 7, ASP.NET Core includes a built-in rate limiting middleware, which can be used to rate limit web applications and APIs. In this blog post, we’ll take a look at how to configure and use the rate limiting middleware in ASP.NET Core.

What is rate limiting?

Every application you build is sharing resources. The application runs on a server that shares its CPU, memory, and disk I/O, on a database that stores data for all your users.

Whether accidental or intentional, users may exhaust those resources in a way that impacts others. A script can make too many requests, or a new deployment of your mobile app has a regression that calls a specific API too many times and results in the database being slow. Ideally, all of your users get access to an equal amount of shared resources, within the boundary of what your application can support.

Let’s say the database used by your application can safely handle around 1000 queries per minute. In your application, you can set a limit to only allow 1000 requests per minute to prevent the database from getting more requests.

Instead of one global “1000 requests per minute” limit, you could look at your average application usage, and for example set a limit of “100 requests per user per minute”. Or chain those limits, and say “100 requests per user per minute, and 1000 requests per minute”.

Rate limits will help to prevent the server from being overwhelmed by too many requests, and still makes sure that all users have a fair chance of getting their requests processed.

Rate limiting in ASP.NET Core

If your application is using .NET 7 (or higher), a rate limiting middleware is available out of the box. It provides a way to apply rate limiting to your web application and API endpoints.

Note: Under the hood, the ASP.NET Core rate limiting middleware uses the System.Threading.RateLimiting subsystem. If you’re interested in rate limiting other resources, for example an HttpClient making requests, or access to other resources, check it out!

Much like other middlewares, to enable the ASP.NET Core rate limiting middleware, you will have to add the required services to the service collection, and then enable the middleware for all request pipelines.

Let’s add a simple rate limiter that limits all to 10 requests per minute, per authenticated username (or hostname if not authenticated):

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddRateLimiter(options =>
{
    options.GlobalLimiter = PartitionedRateLimiter.Create<HttpContext, string>(httpContext =>
        RateLimitPartition.GetFixedWindowLimiter(
            partitionKey: httpContext.User.Identity?.Name ?? httpContext.Request.Headers.Host.ToString(),
            factory: partition => new FixedWindowRateLimiterOptions
            {
                AutoReplenishment = true,
                PermitLimit = 10,
                QueueLimit = 0,
                Window = TimeSpan.FromMinutes(1)
            }));
});

// ...

var app = builder.Build();

// ...

app.UseRouting();
app.UseRateLimiter();

app.MapGet("/", () => "Hello World!");

app.Run();

Too much at once? I agree, so let’s try to break it down.

The call to builder.Services.AddRateLimiter(...) registers the ASP.NET Core middleware with the service collection, including its configuration options. There are many options that can be specified, such as the HTTP status code being returned, what should happen when rate limiting applies, and additional policies.

For now, let’s just assume we want to have one global rate limiter for all requests. The GlobalLimiter option can be set to any PartitionedRateLimiter. In this example, we’re adding a FixedWindowLimiter, and configure it to apply “per authenticated username (or hostname if not authenticated)” - the partition. The FixedWindowLimiter is then configured to automatically replenish permitted requests, and permits “10 requests per minute”.

Further down the code, you’ll see a call to app.UseRateLimiter(). This enables the rate limiting middleware using the options specified earlier.

If you run the application and refresh quickly, you’ll see at some point a 503 Service Unavailable is returned, which is when the rate limiting middleware does its thing.

Configure what happens when being rate limited

Not happy with that 503 being returned when rate limiting is enforced? Let’s look at how to configure that!

Many services settled on the 429 Too Many Requests status code. In order to change the status code, you can set the RejectionStatusCode option:

builder.Services.AddRateLimiter(options =>
{
    options.RejectionStatusCode = 429;

    // ...
});

Additionally, there’s an OnRejected option you can set to customize the response that is sent when rate limiting is triggered for a request. It’s a good practice to communicate what happened, and why a rate limit applies. So instead of going with the default of returning “just a status code”, you can return some more meaningful information. The OnRejected delegate gives you access to the current rate limit context, including the HttpContext.

Here’s an example that sets the response status code to 429, and returns a meaningful response. The response mentions when to retry (if available from the rate limiting metadata), and provides a documentation link where users can find out more.

builder.Services.AddRateLimiter(options =>
{
    options.OnRejected = async (context, token) =>
    {
        context.HttpContext.Response.StatusCode = 429;
        if (context.Lease.TryGetMetadata(MetadataName.RetryAfter, out var retryAfter))
        {
            await context.HttpContext.Response.WriteAsync(
                $"Too many requests. Please try again after {retryAfter.TotalMinutes} minute(s). " +
                $"Read more about our rate limits at https://example.org/docs/ratelimiting.", cancellationToken: token);
        }
        else
        {
            await context.HttpContext.Response.WriteAsync(
                "Too many requests. Please try again later. " +
                "Read more about our rate limits at https://example.org/docs/ratelimiting.", cancellationToken: token);
        }
    };

    // ...
});

Given you have access to the current HttpContext, you also have access to the service collection. It’s a good practice to keep an eye on who, when and why a rate limit is being enforced, and you could log that by grabbing an ILogger from context.HttpContext.RequestServices if needed.

Note: Be careful with the logic you write in your OnRejected implementation. If you use your database context and run 5 queries, your rate limit isn’t actually helping reduce strain on your database. Communicate with the user and return a meaningful error (you could even use the Accept header and return either JSON or HTML depending on the client type), but don’t consume more resources than a normal response would require.

Speaking of communicating about what and why, the ASP.NET Core rate limiting middleware is a bit limited (pun not intended). The metadata you have access to is sparse (“retry after” is pretty much the only useful metadata returned).

Additionally, if you would want to return statistics about your limits (e.g. like GitHub does), you’ll find the ASP.NET Core rate limiting middleware does not support this. You won’t have access to the “number of requests remaining” or other metadata. Not in OnRejected, and definitely not if you want to return this data as headers on every request.

If this is something that matters to you, I advise to check out Stefan Prodan’s AspNetCoreRateLimit, which has many (many!) more options available. Or chime in on this GitHub issue.

Types of rate limiters

In our example, we’ve used the FixedWindowLimiter to limit the number of requests in a time window.

There are more rate limiting algorithms available in .NET that you can use:

Concurrency limit is the simplest form of rate limiting. It doesn’t look at time, just at number of concurrent requests. “Allow 10 concurrent requests”.
Fixed window limit lets you apply limits such as “60 requests per minute”. Every minute, 60 requests can be made. One every second, but also 60 in one go.
Sliding window limit is similar to the fixed window limit, but uses segments for more fine-grained limits. Think “60 requests per minute, with 1 request per second”.
Token bucket limit lets you control flow rate, and allows for bursts. Think “you are given 100 requests every minute”. If you make all of them over 10 seconds, you’ll have to wait for 1 minute before you are allowed more requests.

In addition, you can “chain” rate limiters of one type of various types, using the PartitionedRateLimiter.CreateChained() helper.

Maybe you want to have a limit where one can make 600 requests per minute, but only 6000 per hour. You could chain two FixedWindowLimiter with different options.

builder.Services.AddRateLimiter(options =>
{
    options.GlobalLimiter = PartitionedRateLimiter.CreateChained(
        PartitionedRateLimiter.Create<HttpContext, string>(httpContext =>
            RateLimitPartition.GetFixedWindowLimiter(httpContext.ResolveClientIpAddress(), partition =>
                new FixedWindowRateLimiterOptions
                {
                    AutoReplenishment = true,
                    PermitLimit = 600,
                    Window = TimeSpan.FromMinutes(1)
                })),
        PartitionedRateLimiter.Create<HttpContext, string>(httpContext =>
            RateLimitPartition.GetFixedWindowLimiter(httpContext.ResolveClientIpAddress(), partition =>
                new FixedWindowRateLimiterOptions
                {
                    AutoReplenishment = true,
                    PermitLimit = 6000,
                    Window = TimeSpan.FromHours(1)
                })));

    // ...
});

Note that the ResolveClientIpAddress() extension method I use here is just an example that checks different headers for the current client’s IP address. Use a partition key that makes sense for your application.

Queue requests instead of rejecting them: `QueueLimit`

On most of the rate limiters that ship with .NET, you can specify a QueueLimit next to the PermitLimit. The QueueLimit specifies how many incoming requests will be queued but not rejected when the PermitLimit is reached.

Let’s look at an example:

PartitionedRateLimiter.Create<HttpContext, string>(httpContext =>
    RateLimitPartition.GetFixedWindowLimiter(httpContext.ResolveClientIpAddress(), partition =>
        new FixedWindowRateLimiterOptions
        {
            AutoReplenishment = true,
            PermitLimit = 10,
            QueueLimit = 6,
            QueueProcessingOrder = QueueProcessingOrder.OldestFirst,
            Window = TimeSpan.FromSeconds(1)
        })));

In the above example, clients can make 10 requests per second. If they make more requests per second, up to 6 of those excess requests will be queued and will seemingly “hang” instead of being rejected. The next second, this queue will be processed.

If you expect small traffic bursts, setting QueueLimit may provide a nicer experience to your users. Instead of rejecting their requests, you’re delaying them a bit.

I’d personally not go with large QueueLimit, and definitely not for long time windows. As a consumer of an API, I’d rather get a response back fast. Even if it’s a failure, as those can be retried. A few seconds of being in a queue may make sense, but any longer the client will probably time out anyway and your queue is being kept around with no use.

Create custom rate limiting policies

Next to the default rate limiters, you can build your own implementation of IRateLimiterPolicy<TPartitionKey>. This interface specifies 2 methods: GetPartition(), which you’ll use to create a specific rate limiter for the current HttpContext, and OnRejected() if you want to have a custom response when this policy is rejecting a request.

Here’s an example where the rate limiter options are partitioned by either the current authenticated user, or their hostname. Authenticated users get higher limits, too:

public class ExampleRateLimiterPolicy : IRateLimiterPolicy<string>
{
    public RateLimitPartition<string> GetPartition(HttpContext httpContext)
    {
        if (httpContext.User.Identity?.IsAuthenticated == true)
        {
            return RateLimitPartition.GetFixedWindowLimiter(httpContext.User.Identity.Name!,
                partition => new FixedWindowRateLimiterOptions
                {
                    AutoReplenishment = true,
                    PermitLimit = 1_000,
                    Window = TimeSpan.FromMinutes(1),
                });
        }

        return RateLimitPartition.GetFixedWindowLimiter(httpContext.Request.Headers.Host.ToString(),
            partition => new FixedWindowRateLimiterOptions
            {
                AutoReplenishment = true,
                PermitLimit = 100,
                Window = TimeSpan.FromMinutes(1),
            });
    }

    public Func<OnRejectedContext, CancellationToken, ValueTask>? OnRejected { get; } =
        (context, _) =>
        {
            context.HttpContext.Response.StatusCode = 418; // I'm a 🫖
            return new ValueTask();
        };
}

And instead of rejecting requests with a well-known status code, this policy rejects requests with a 418 status code (“I’m a teapot”).

Policies for rate limiting groups of endpoints

So far, we’ve covered global limits that apply to all requests. There’s a good chance you want to apply different limits to different groups of endpoints. You may have endpoints that you don’t want to rate limit at all.

This is where policies come in. In your configuration options, you can create different policies using the .Add{RateLimiter}() extension methods, and then apply them to specific endpoints or groups thereof.

Here’s an example configuration adding 2 fixed window limiters with different settings, and a different policy name ("Api" and "Web").

builder.Services.AddRateLimiter(options =>
{
    options.AddFixedWindowLimiter("Api", options =>
    {
        options.AutoReplenishment = true;
        options.PermitLimit = 10;
        options.Window = TimeSpan.FromMinutes(1);
    });

    options.AddFixedWindowLimiter("Web", options =>
    {
        options.AutoReplenishment = true;
        options.PermitLimit = 10;
        options.Window = TimeSpan.FromMinutes(1);
    });

    // ...
});

Before we look at how to apply these policies, let’s first cover an important warning…

Warning: The .Add{RateLimiter}() extension methods partition rate limits based on the policy name. This is okay if you want to apply global limits per group of endpoints, but it’s not when you want to partition per user or per IP address or something along those lines.

If you want to add policies that are partitioned by policy name and any aspect of an incoming HTTP request, use the .AddPolicy(..) method instead:
options.AddPolicy("Api", httpContext =>
RateLimitPartition.GetFixedWindowLimiter(httpContext.ResolveClientIpAddress(),
partition => new FixedWindowRateLimiterOptions
{
AutoReplenishment = true,
PermitLimit = 10,
Window = TimeSpan.FromSeconds(1)
}));

With that out of the way, let’s see how you can apply policies to certain endpoints.

Rate limiting policies with ASP.NET Core Minimal API

When using ASP.NET Core Minimal API, you can enable a specific policy per endpoint, or per group of endpoints:

// Endpoint
app.MapGet("/api/hello", () => "Hello World!").RequireRateLimiting("Api");

// Group
app.MapGroup("/api/orders").RequireRateLimiting("Api");

Similarly, you can disable rate limiting per endpoint or group:

// Endpoint
app.MapGet("/api/hello", () => "Hello World!").DisableRateLimiting();

// Group
app.MapGroup("/api/orders").DisableRateLimiting();

Rate limiting policies with ASP.NET Core MVC

When using ASP.NET Core MVC, you can enable and disable policies per controller or action.

[EnableRateLimiting("Api")]
public class Orders : Controller
{
    [DisableRateLimiting]
    public IActionResult Index()
    {
        return View();
    }

    [EnableRateLimitingAttribute("ApiListing")]
    public IActionResult List()
    {
        return View();
    }
}

You’ll find this works similar to authorization and authorization policies.

ASP.NET Core rate limiting with YARP proxy

In your application, you may be using YARP, to build a reverse proxy gateway sitting in front of various backend applications. For example, you may run YARP to listen on example.org, and have it proxy all requests going to this domain while mapping /api and /docs to different web apps running on diffreent servers.

In such scenario, rate limiting will also be useful. You could rate limit each application separately, or apply rate limiting in the YARP proxy. Given both YARP and ASP.NET Core rate limiting are middlewares, they play well together.

As an example, here’s a YARP proxy that applies a global rate limit of 10 requests per minute, partitioned by host header:

using System.Threading.RateLimiting;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddRateLimiter(options =>
{
    options.RejectionStatusCode = 429;
    options.GlobalLimiter = PartitionedRateLimiter.Create<HttpContext, string>(httpContext =>
        RateLimitPartition.GetFixedWindowLimiter(
            partitionKey: httpContext.Request.Headers.Host.ToString(),
            factory: partition => new FixedWindowRateLimiterOptions
            {
                AutoReplenishment = true,
                PermitLimit = 10,
                QueueLimit = 0,
                Window = TimeSpan.FromMinutes(1)
            }));
});

builder.Services.AddReverseProxy()
    .LoadFromConfig(builder.Configuration.GetSection("ReverseProxy"));

var app = builder.Build();
app.UseRateLimiter();
app.MapReverseProxy();
app.Run();

Just like with ASP.NET Core Minimal API and MVC apps, you can use the AddRateLimiter() extension method to configure rate limits, and AddReverseProxy() to register the YARP configuration.

To then register the configured middlewares in your application, use the UseRateLimiter() and MapReverseProxy() can be used.

Wrapping up

By limiting the number of requests that can be made to your application, you can reduce the load on your server and have more fair usage of resources among your users. ASP.NET Core provides an easy way to implement rate limiting in your applications. By using the built-in middleware, you can easily configure rate limiting for your application.

In this post, I wanted to give you some insights about how you can use the ASP.NET Core rate limiting middleware. It’s not as complete as Stefan Prodan’s AspNetCoreRateLimit, but there are enough options available to add rate limiting to your application.

In a future blog post, I’ll cover more concepts around rate limiting. Stay tuned!

How to test ASP.NET Core Minimal APIs

Maarten Balliauw — Sat, 04 Jun 2022 02:44:05 +0000

How do you test that your ASP.NET Core Minimal API behaves as expected? Do you need to deploy your application? Can you write tests with frameworks like xUnit, NUnit, or MSTest?

In this post, you will learn the basics of testing ASP.NET Core Minimal APIs. You’ll get started with testing a “hello world” endpoint, and then test a more complex API that returns JSON data. You’ll finish with customizing the ASP.NET Core service collection, so you can customize services for your unit tests and integration tests.

By the end of this post, you will have a good understanding of how to make sure your ASP.NET Core Minimal APIs behave as expected and can be deployed to production, even on Fridays!

This post was originally published on the Twilio blog on June 06, 2022: How to test ASP.NET Core Minimal APIs

Prerequisites

An OS that supports .NET (Windows/macOS/Linux)
A .NET IDE (such as JetBrains Rider)
.NET 6.0 SDK or later

You can find the source code for this tutorial on GitHub. Use it as a reference if you run into any issues.

Create a test project

To get started, you will need to create a solution with two projects: an ASP.NET Core Minimal API that will contain the application, and a unit test project that will contain the tests. In this blog post, you will use xUnit as the testing framework.

You can create this solution in your favorite .NET IDE, or using the .NET CLI. In the command line or terminal window, navigate to the folder you want your project to be created in, and run the following commands:

dotnet new web -o MyMinimalApi
dotnet new xunit -o MyMinimalApi.Tests
dotnet add MyMinimalApi.Tests reference MyMinimalApi
dotnet new sln
dotnet sln add MyMinimalApi
dotnet sln add MyMinimalApi.Tests

You now have a MyMinimalApi.sln file, and two projects (MyMinimalApi.csproj for the ASP.NET Core Minimal API, and MyMinimalApi.Tests.csproj for the unit tests) with some template code. The test project also has a project reference to the Minimal API project.

To run the Minimal API application, you can use the .NET CLI and specify the project to run:

dotnet run --project MyMinimalApi

The tests can be run using the following .NET CLI command:

dotnet test

There’s not a lot of useful code in these projects yet. The Minimal API project contains a Program.cs file with an endpoint that returns the string “Hello World!”:

var builder = WebApplication.CreateBuilder(args);

var app = builder.Build();

app.MapGet("/", () => "Hello World!");

app.Run();

The test project (MyMinimalApi.Tests.csproj) contains a template unit test file UnitTest1.cs that you will replace later in this article.

Update the test project

Before you can start testing your Minimal API, you will need to make some updates to the test project. The unit tests need to be able to use the ASP.NET Core framework, so you’ll have to bring that in somehow. The easiest way to do this is by adding a reference to the Microsoft.AspNetCore.Mvc.Testing package. This package also comes with several helper classes that are invaluable when writing unit tests later on.

Add this package using your favorite IDE, or use the .NET CLI:

dotnet add MyMinimalApi.Tests package Microsoft.AspNetCore.Mvc.Testing

The MyMinimalApi.Tests.csproj file now looks like this:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net6.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>

    <IsPackable>false</IsPackable>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.AspNetCore.Mvc.Testing" Version="6.0.0" />
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.1.0" />
    <PackageReference Include="xunit" Version="2.4.1" />
    <PackageReference Include="xunit.runner.visualstudio" Version="2.4.3">
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
      <PrivateAssets>all</PrivateAssets>
    </PackageReference>
    <PackageReference Include="coverlet.collector" Version="3.1.2">
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
      <PrivateAssets>all</PrivateAssets>
    </PackageReference>
  </ItemGroup>

  <ItemGroup>
    <ProjectReference Include="..\MinimalAPI\MinimalAPI.csproj" />
  </ItemGroup>

</Project>

You can now start writing unit tests for your Minimal API.

“Hello World” and the ASP.NET Core test server

In the Minimal API project, Program.cs already defines a “Hello World!” endpoint. You will test this endpoint first. Before you can do this, you will need to add the following public partial class definition at the bottom of Program.cs:

public partial class Program { }

The reason why you need this partial class definition, is that by default the Program.cs file is compiled into a private class Program, which can not be accessed by other projects. By adding this public partial class, the test project will get access to Program and lets you write tests against it.

In the MyMinimalApi.Tests project, rename the UnitTest1.cs file to HelloWorldTests.cs and update the code:

namespace MyMinimalApi.Tests;

using Microsoft.AspNetCore.Mvc.Testing;

public class HelloWorldTests
{
    [Fact]
    public async Task TestRootEndpoint()
    {

    }
}

The TestRootEndpoint() test will have to do a couple of things:

Start the ASP.NET Core Minimal API
Create an HTTP client for to connect to the application
Send an HTTP request to the / endpoint
Verify the response

Earlier in this post, you have added a reference to the Microsoft.AspNetCore.Mvc.Testing package. This package contains the WebApplicationFactory<T>, which is an important building block for testing ASP.NET Core applications.

The WebApplicationFactory<T> class creates an in-memory application that you can test. It handles bootstrapping of your application, and provides an HttpClient that you can use to make requests.

Update the code in the TestRootEndpoint() method:

[Fact]
public async Task TestRootEndpoint()
{
    await using var application = new WebApplicationFactory<Program>();
    using var client = application.CreateClient();

    var response = await client.GetStringAsync("/");

    Assert.Equal("Hello World!", response);
}

The code uses WebApplicationFactory<Program>. Here’s the reason you had to add that public partial class! You can use other public classes from the Minimal API project as well, but I personally prefer Program as it’s there in every project.

You can run this test using the .NET CLI, and look at the results:

> dotnet test

Microsoft (R) Test Execution Command Line Tool Version 17.2.0 (x64)
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.

Passed!  - Failed:     0, Passed:     1, Skipped:     0, Total:     1, Duration: < 1 ms - MyMinimalApi.Tests.dll (net6.0)

The test you created has just started your Minimal API application using the WebApplicationFactory<Program>, and uses an HttpClient that was returned by application.CreateClient(). Using this client, the test makes an HTTP GET request to the / endpoint. In this example, you used the GetStringAsync("/") method to do this. The test then asserts the response matches what is expected.

Congratulations, you have just created your first test for an ASP.NET Core Minimal API!

Update the Minimal API project

Let’s spice things up a little! In most APIs, endpoints will work with JSON payloads in requests and responses. An API endpoint may return different results depending on the request that is being made. It may return a 200 OK status code on success, and a 400 Bad Request status code with more details in the response body when the request was not valid.

In this section, you will add such an endpoint to the Minimal API. This endpoint will also perform validation of the request, using the MiniValidation package.

Add this package using your favorite IDE, or use the .NET CLI:

dotnet add MyMinimalApi package MiniValidation --prerelease

Info: MiniValidation is a library intended to bring model validation to ASP.NET Core Minimal APIs. It currently only has pre-release packages available. When a stable version lands you should consider dropping the --prerelease version.

When that is installed, add a Person class to your Minimal API. This class will be used as a request payload later on.

public class Person
{
    [Required, MinLength(2)]
    public string? FirstName { get; set; }

    [Required, MinLength(2)]
    public string? LastName { get; set; }

    [Required, DataType(DataType.EmailAddress)]
    public string? Email { get; set; }
}

Note that the Person class adds validation attributes from the System.ComponentModel.DataAnnotations namespace. Add using System.ComponentModel.DataAnnotations; to the top of your Program.cs file to include the namespace . The MiniValidation packages you added earlier can process these attributes and validate the request is well-formed.

The Minimal API will also need to be able to store the Person in a data store. While modeling this data store is not in the scope of this article, you can define an IPeopleService interface to interact with the data store, and a PeopleService class that implements this interface:

public interface IPeopleService
{
    string Create(Person person);
}

public class PeopleService: IPeopleService
{
    public string Create(Person person)
        => $"{person.FirstName} {person.LastName} created.";
}

Info: In real projects, the PeopleService could use Entity Framework Core or other storage mechanisms to do something more useful.

It’s now time to register the IPeopleService with the ASP.NET Core service collection, so your API endpoint can make use of it. Add it as a scoped service to make sure a new instance of PeopleService is created each time a request comes in:

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddScoped<IPeopleService, PeopleService>();

// ...

You are doing great! As a final step in this section, you will implement the actual API endpoint in your Minimal API. This endpoint will listen for POST requests on /people, and accept a Person object in the request body. After the endpoint validates the incoming request, the API either uses the IPeopleService to store the object in the database, or returns a validation result.

app.MapPost("/people", (Person person, IPeopleService peopleService) =>
    !MiniValidator.TryValidate(person, out var errors)
        ? Results.ValidationProblem(errors)
        : Results.Ok(peopleService.Create(person)));

Add using MiniValidation; to your using statements at the top of Program.cs class so you can use the MiniValidator class.

Just to make sure, here’s what your Program.cs should now look like:

using System.ComponentModel.DataAnnotations;
using MiniValidation;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddScoped<IPeopleService, PeopleService>();

var app = builder.Build();

app.MapGet("/", () => "Hello World!");

app.MapPost("/people", (Person person, IPeopleService peopleService) =>
    !MiniValidator.TryValidate(person, out var errors)
        ? Results.ValidationProblem(errors)
        : Results.Ok(peopleService.Create(person)));

app.Run();

public partial class Program { }

public interface IPeopleService
{
    string Create(Person person);
}

public class PeopleService : IPeopleService
{
    public string Create(Person person)
        => $"{person.FirstName} {person.LastName} created.";
}

public class Person
{
    [Required, MinLength(2)]
    public string? FirstName { get; set; }

    [Required, MinLength(2)]
    public string? LastName { get; set; }

    [Required, DataType(DataType.EmailAddress)]
    public string? Email { get; set; }
}

If you want to, you can run the Minimal API and test the /people endpoint from your terminal`.

First, start your Minimal API using dotnet run --project MyMinimalApi and look for the localhost URL in the output.

If you have the curl command available in your terminal, run:

bash curl -X POST --location "https://localhost:7230/people" \ -H "Content-Type: application/json" \ -d "{ \"FirstName\": \"Maarten\" }"

Or if you're using PowerShell, run:

powershell Invoke-WebRequest
-Uri https://localhost:7230/people -Method Post
-ContentType "application/json" -Body '{"FirstName": "Maarten"}'

Replace the https://localhost:7230 with the localhost URL that the dotnet run command printed to the console.

The response should be a 400 Bad request, since the LastName and Email properties are required:

`
HTTP/1.1 400 Bad Request
Content-Type: application/problem+json
Date: Fri, 03 Jun 2022 09:04:56 GMT
Server: Kestrel
Transfer-Encoding: chunked

{
"type": "https://tools.ietf.org/html/rfc7231#section-6.5.1",
"title": "One or more validation errors occurred.",
"status": 400,
"errors": {
"LastName": [
"The LastName field is required."
],
"Email": [
"The Email field is required."
]
}
}
`

After you confirm the endpoint works, you will convert this request into a test!

Test different payloads and HTTP methods

Your Minimal API now has a /people endpoint. It has two possible response types: a 200 OK that returns a string value, and a 400 Bad Request that returns problem details as a JSON payload.

In the MyMinimalApi.Tests project, add a PeopleTests.cs file that contains the following code:

`csharp
using System.Net;
using System.Net.Http.Json;
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Mvc.Testing;

namespace MyMinimalApi.Tests;

public class PeopleTests
{
[Fact]
public async Task CreatePerson()
{
}

[Fact]
public async Task CreatePersonValidatesObject()
{
}

}
`

The PeopleTests class now contains 2 test methods that you will need to implement:

CreatePerson() to test the 200 OK scenario
CreatePersonValidatesObject() to test the 400 Bad Request scenario

You will start with the CreatePerson() test method. The test will again make use of the WebApplicationFactory<Program> to create an in-memory HTTP client that you can use to validate the API.

`csharp
[Fact]
public async Task CreatePerson()
{
await using var application = new WebApplicationFactory();

var client = application.CreateClient();

}
`

Next, you will use the client to send a JSON payload to the /people endpoint. You can use the PostAsJsonAsync() method to send a JSON payload to the Minimal API under test. Finally, you can use the xUnit Assert class to validate the response status code and the response content.

Update the CreatePerson() test like below:

`csharp
[Fact]
public async Task CreatePerson()
{
await using var application = new WebApplicationFactory();

var client = application.CreateClient();

var result = await client.PostAsJsonAsync("/people", new Person
{
    FirstName = "Maarten",
    LastName = "Balliauw",
    Email = "maarten@jetbrains.com"
});

Assert.Equal(HttpStatusCode.OK, result.StatusCode);
Assert.Equal("\"Maarten Balliauw created.\"", await result.Content.ReadAsStringAsync());

}
`

You can run this test using the .NET CLI, and confirm your Minimal API works as expected.

bash dotnet test

The CreatePersonValidatesObject() test is next. Like in the CreatePerson() test method, you will begin with creating a request to the in-memory Minimal API. Only this time, you will send an empty Person object.

Since all of its properties will be null or empty, the test should get back a 400 Bad Request. You can assert this is indeed the case. What’s more, you can also use the result.Content.ReadFromJsonAsync<>() method to deserialize the validation problems, and verify they are as expected.

Update the CreatePersonValidatesObject() test like below:

`csharp
[Fact]
public async Task CreatePersonValidatesObject()
{
await using var application = new WebApplicationFactory();

var client = application.CreateClient();

var result = await client.PostAsJsonAsync("/people", new Person());

Assert.Equal(HttpStatusCode.BadRequest, result.StatusCode);

var validationResult = await result.Content.ReadFromJsonAsync<HttpValidationProblemDetails>();
Assert.NotNull(validationResult);
Assert.Equal("The FirstName field is required.", validationResult!.Errors["FirstName"][0]);

}
`

I will leave the validation of the other properties as an exercise for you.

Again, try running this test using the .NET CLI, and confirm your Minimal API works as expected.

bash dotnet test

Well done! You have now written tests that validate JSON payloads accepted and returned by your Minimal API!

Customizing the service collection

There’s one more thing… The Minimal API you created contains a PeopleService that, in a more real-life project, could need a database connection. This could be okay for some tests, and unnecessary for others.

The tests that you have written so far all have been validating the responses of the Minimal API. There’s no real need for the “real” implementation of IPeopleService, so let’s see how you can swap it out with a test implementation!

In the MyMinimalApi.Tests project, create a new file TestPeopleService.cs with the following code:

csharp public class TestPeopleService : IPeopleService { public string Create(Person person) => "It works!"; }

The TestPeopleService class implements IPeopleService just like the real implementation does, but the Create method returns a simple string value.

Next, you will update the test methods to configure the WebApplicationFactory<Program> with a service override for IPeopleService, wiring it to TestPeopleService instead. You can do this in a number of ways: using the WithWebHostBuilder() and ConfigureServices() methods, or by implementing a custom WebApplicationFactory<T>. In this tutorial, you will use the first approach to change the IPeopleService to be a TestPeopleService.

Update the CreatePerson test with the following code:

`csharp
[Fact]
public async Task CreatePerson()
{
await using var application = new WebApplicationFactory()
.WithWebHostBuilder(builder => builder
.ConfigureServices(services =>
{
services.AddScoped();
}));

var client = application.CreateClient();

var result = await client.PostAsJsonAsync("/people", new Person
{
FirstName = "Maarten",
LastName = "Balliauw",
Email = "maarten@jetbrains.com"
});

Assert.Equal(HttpStatusCode.OK, result.StatusCode);
Assert.Equal("\"It works!\"", await result.Content.ReadAsStringAsync());
}
`

To use services.AddScoped, add using Microsoft.Extensions.DependencyInjection; to your using statements at the top of the file.

Note that in the code sample, the final Assert.Equal is now testing for the string that is returned by TestPeopleService.

Depending on how many customizations you want to make to your Minimal API under test, you can move the WithWebHostBuilder() and ConfigureServices() methods out, and override the WebApplicationFactory<T> class. This has the advantage of having one place where you customize the service collection.

For example, you can create a TestingApplication class and override the CreateHost method to customize the service collection:

`csharp
class TestingApplication : WebApplicationFactory
{
protected override IHost CreateHost(IHostBuilder builder)
{
builder.ConfigureServices(services =>
{
services.AddScoped();
});

   return base.CreateHost(builder);

}
}
`

You can use it in tests by replacing new WebApplicationFactory<Program> with new TestingApplication():

`csharp
[Fact]
public async Task CreatePerson()
{
await using var application = new TestingApplication();

var client = application.CreateClient();

var result = await client.PostAsJsonAsync("/people", new Person
{
FirstName = "Maarten",
LastName = "Balliauw",
Email = "maarten@jetbrains.com"
});

Assert.Equal(HttpStatusCode.OK, result.StatusCode);
Assert.Equal("\"It works!\"", await result.Content.ReadAsStringAsync());
}
`

If you want to start customizing the Minimal API during tests, make sure to explore the various methods of WebApplicationFactory<T> that you can override to configure your application for the tests you are writing.

Conclusion

That’s it! You just built several tests for an ASP.NET Core Minimal API, and validated it behaves as expected. You started out with testing a basic endpoint that returned a string, and then saw how to work with different HTTP methods and payloads on the request and response. You even customized the ASP.NET Core service collection with custom services for your tests.

Whether you are writing unit tests, integration tests or both, you should now have a good understanding of how to go about using the test server and customizing the service collection for many scenarios.

If you’re hungry for more, check out the Microsoft docs on integration testing.

Techniques and tools to update your C# project - Migrating to nullable reference types - Part 4

Maarten Balliauw — Tue, 03 May 2022 02:44:05 +0000

Previously, we saw how you can help the compiler’s flow analysis understand your code, by annotating your code for nullability.

In this final post of our series, we’ll have a look at the techniques and tools that are available to migrate to using nullable reference types in an existing code base.

Pick your approach: there is no silver bullet

As we have seen in a previous post, it can be an overwhelming experience to go all-in and enable the nullable annotation context for all projects in your solution.

Generally speaking, it’s a good idea to fully enable the nullable annotation context for new projects. This gets you the benefits of better static flow analysis from the start.

For existing projects, the choice is yours:

For smaller projects, you can set <Nullable>enable</Nullable> at the project level and plow through.
For larger projects, you may want to leave the project level set to disable, and add #nullable enable file by file.
Alternatively, you can set warnings as the project level default, so you’ll see warnings where the compiler’s flow analysis infers potential null references. You can then add #nullable enable file by file, and gradually add the right annotations and nullable attributes.

Regardless of the setting you choose at the project level, you’ll be in the mode of working through all warnings incrementally.

There is no silver bullet. There are, however, some techniques and tools that will help you reach the end goal of having a fully annotated codebase.

Start at the edges and work inwards

Where to begin? What worked well for me on various code bases, was to start at the edges.

Try and find the classes in your project that have zero dependencies on other reference types, apart from some strings. Data Transfer Objects (DTOs) / Plain-Old CLR Objects (POCOs) almost always fall under this category.

DTOs/POCOs are often used in many places throughout your project. Updating nullability for these classes means that nullability flows through the rest of your projects, and makes usages more reliable project-wide. So even if annotating one property at the edge seems like a small thing to do, it will flow through and be meaningful in the bigger picture.

Here’s an example:

public class LocationInfo
{
    public string Country { get; set; }
    public string Location { get; set; }

    public LocationInfo(string country, string location)
    {
        Country = country;
        Location = location;
    }
}

This LocationInfo class only has two properties. Converting this class to using C# nullable reference types may be easy!

Add annotations or redesign your code

Let’s enable nullable reference types for this LocationInfo class!

Add #nullable enable to the class file
In the IDE, use Find Usages on every property, and determine if the properties are potentially set to null anywhere.
- If there’s a value specified at every usage, keep the class as-is.
- If a null reference is passed in, you may need to annotate the property with ?
In the IDE, use Find Usages on the constructor, and determine if the constructor parameters are potentially set to null anywhere.

Going through usages, I found this particular case in my code base:

if (_databaseReader.TryCity(address, out var result) && result != null)
{
    return new LocationInfo(
      result.Country.Name,
      result.City.Name);
}

I really want to recommend using ReSharper (R#) or JetBrains Rider once more. As mentioned before, both tools ship years of experience with nullable flow analysis, and it shows.

Both R# and Rider caught that result.Country.Name and result.City.Name may be null, even with the nullable warning context set to disabled at the project level:

This is one of those cases where you’ll have to decide on the approach to take…

Should you annotate the LocationInfo constructor parameters?
Should you keep the LocationInfo constructor parameters as non-nullable and update the call site?

In this case there is only one call site that potentially passes a null reference, so let’s keep the LocationInfo constructor parameters non-nullable and update the call site instead:

if (_databaseReader.TryCity(address, out var result) &&
    result != null &&
    result.Country.Name != null &&
    result.City.Name != null)
{
    return new LocationInfo(
        result.Country.Name,
        result.City.Name);
}

return LocationInfo.Unknown;

The call site is updated with more thorough null checks, and a redesigned API:

When no null values are present, we still return LocationInfo with non-nullable properties.
When any values are null, we return LocationInfo.Unknown - a static property with both properties set to "Unknown". No need for null checks anywhere LocationInfo is used, there’s always going to be a value.

Much like with async/await, nullable annotations will flow through your entire project. If we annotated the properties of LocationInfo as being nullable, we’d have to do null checks in our entire project. Instead, we chose to redesign our code and set a boundary of how far potential null references can flow. In this case, not far at all.

Once again, keep in mind there’s no silver bullet. In some cases, adding a nullable annotation will be the way to go, in other cases a small (or big) redesign may be better.

Don’t be afraid of `null`

Before we continue, there’s something important to keep in mind. The goal of migrating to C# nullable reference types, is to gain more confidence in the flow analysis provided by the compiler and the IDE. We’re not here to completely get rid of all null usages in our code!

As part of migrating to nullable reference types, you will be annotating some reference types with ?, sometimes you’ll be suppressing warnings with !, and sometimes, you’ll end up redesigning bits of your code.

Returning or passing around null is totally fine. Using C# nullable reference types and annotations makes doing so more reliable, with fewer chances of NullReferenceException being thrown unexpectedly. We’re building a safety net.

Nullable warning suppressions should be temporary

As part of migration, you may sprinkle some null-forgiving operators through your project’s code. When you’re not sure a reference type should be nullable or not, you can suffix the usage with the dammit-operator, !, and suppress any nullability warnings for that code path.

What’s nice about nullable warning suppressions, is that they disables flow analysis for a certain code path. You can use it to see the effect of what would happen to a code path if a reference that currently can be null would be redefined as non-nullable.

In some cases, you will indeed need to suppress null, and its usage is valid. In most cases, however, consider nullable warning suppressions a code smell. Using ! should be a temporary thing, use it with care. Since it disables flow analysis, it could hide nullability issues in your project - the exact issue you set out to improve upon!

Tool tip: At any time during a migration, you can use ReSharper or JetBrains Rider to find all nullable warning suppressions. Use Alt+Enter on any suppression, and search for other suppressions in the current file, project, or solution.

Where is this value coming from? Where is it being used?

In many cases, Find Usages will be sufficient to get an idea of the direct usages of a class, constructor, method, or property. In other cases, you may need more information.

ReSharper (R#) and JetBrains Rider come with value tracking and call tracking to help you out here.Visual Studio 2022 also has a Track Value Source command, but your mileage with it will vary.

With value tracking, you can follow the entire flow of a specific value and determine where it is originating from and where it is being used.

Not sure if this TrackingAccount property should be annotated? The Inspect | Value Origin action will track all places where a value for this property can be assigned, and provides a tool window to jump to every location.

Note that it’s also possible to do the inverse, and analyze where the value from this TrackingAccount property is used.

JetBrains Annotations to C# nullable annotations

In a previous post, we discussed JetBrains Annotations already. If you’re working on a project where these annotations were already in use before C# introduced nullable reference types, you are in luck when migrating to C#’s version!

When you enable the nullable context, ReSharper and JetBrains Rider will help you with the migration. You’ll get hints on whether certain annotations are still needed.

#nullable enable

[NotNull]
private static string ReadColumnFromCsv(
    CsvReader csv,
    [CanBeNull] string columnName,
    [NotNull] string defaultValue = "")
{
    return !string.IsNullOrEmpty(columnName)
        ? csv[columnName]
        : defaultValue;
}

The ReadColumnFromCsv returns a non-nullable string, which means the [NotNull] annotation can be safely removed. The Remove redundant attribute quick fix is one Alt+Enter away!

Similarly, the [CanBeNull] string columnName parameter declaration can be updated. The original [CanBeNull] annotation can be removed, and converted to string? columnName.

If you’re using the 2022.1 version of ReSharper or JetBrains Rider, there is a new Migrate to #nullable enable quick fix that does a few things at once:

It inserts all [NotNull] and [CanBeNull] attributes inherited from base members such as implemented interfaces. JetBrains annotations can be inherited (unlike C#’s annotations), so they are pulled in.
It infers annotations, by looking at your code’s branches. Are you returning null? A nullable return type will be inferred.
It converts all JetBrains Annotations in th current file to C# annotations.

Tip: You can run most of these quick fixes on your entire file, project or solution in one go.

Determine nullability based on `null` checks

When you’re annotating your code, there are often clear hints in your code about what its nullability should be.

Here’s a quiz: in the following ReadColumnFromExcel method, what should the nullability of the columnName parameter be?

#nullable enable

public static string ReadColumnFromExcel(
    Dictionary<int, string> data,
    Dictionary<string, int> mappings,
    string columnName,
    string defaultValue = "")
{
    if (columnName != null)
    {
        if (mappings.TryGetValue(columnName, out var columnIndex)
            && data.TryGetValue(columnIndex, out var columnData))
        {
            return columnData ?? defaultValue;
        }
    }

    return defaultValue;
}

If you answered string? columnName, you are right!

The first line of code in this method is checking if columnName != null, which means it should be annotated as nullable. There will be lots of these cases in the project you are migrating, and they usually provide a great hint in terms of annotating a parameter or property.

ReSharper and JetBrains Rider will detect these cases for you, and offer to fix the annotation(s) for you.

What about third-party libraries and external code?

When you are consuming third-party libraries, you’re in for a treat! Looking at the top downloaded packages on NuGet.org, not all of them are annotated. I’m sure if you look at the long tail of packages, there will be many more libraries that are not annotated at all!

Libraries with C# annotations

If you’re lucky, the library you are consuming has been fully annotated. There’s not much to say in this case: the C# compiler and all IDEs will pick up these annotations, and give you design- and compile-time hints. Great!

Libraries with JetBrains Annotations

If you’re consuming a library that ships its JetBrains Annotations, and you are using ReSharper or JetBrains Rider, you’re in luck as well.

ReSharper and JetBrains Rider will automatically recognize annotations found in the JetBrains.Annotations namespace. Sometimes, libraries ship a custom namespace. The IDE will recognize these attributes, but you’ll still need to enable them in the settings.

Here’s an example with Hangfire. This project ships their annotations in the Hangfire.Annotations namespace:

After enabling it, the IDE considers the annotations in flow analysis:

While many libraries use JetBrains Annotations, not all of them ship them in their NuGet package.

Tip: If you have a library that is annotated with JetBrains annotations, make sure to ship them along with your code and make the life of many developers more enjoyable.

Libraries without annotations

If you are consuming libraries that are not annotated with either C#’s or JetBrains’ nullable annotations, you’ll have to do lots of null checks. Unless you dive into their source code, there is no way the compiler’s flow analysis can give you reliable hints.

If you’re using ReSharper or JetBrains Rider, you can enable pessimistic analysis to help uncover the places where you’ll need extra null checks.

Pessimistic analysis

By default, ReSharper and JetBrains Rider analyze your code in optimistic mode. In this mode, you will only see warnings about potentially dereferencing null if you explicitly checked it for null in the code path, or if it’s annotated as nullable.

The opposite mode is pessimistic. Unless a value is annotated as non-nullable, the IDE will expect you do a null check. The web help has more info about both modes.

Here’s an example. This ReadColumnFromCsv returns a non-null string. In pessimistic mode, you’ll see a warning when returning csv[columnName]. The CsvReader’s indexer has no nullable annotations, and therefore pessimistic analysis treats it as a potential null reference.

private static string ReadColumnFromCsv(
    CsvReader csv,
    string? columnName,
    string defaultValue = "")
{
    return !string.IsNullOrEmpty(columnName)
        ? csv[columnName] // Considered nullable in pessimistic mode
        : defaultValue;
}

To get rid of this warning, you’ll have to return the defaultValue when csv[columnName] is null:

  return !string.IsNullOrEmpty(columnName)
- ? csv[columnName]
+ ? csv[columnName] ?? defaultValue
      : defaultValue;

Pessimistic analysis is not a mode I would recommend as the default in your projects. It’s more pessimistic than the compiler is!

It may be of help when migrating to C# nullable reference types, as it will definitely uncover some cases where you need additional null checks. Especially when working with third-party libraries that may not (yet) be annotated!

Deserializing JSON

If you have played with nullable reference types already, you may have found that there is no good way to get rid of all nullability warnings.

Typically, you will have several classes that will be used when deserializing JSON data, something like this:

public class User
{
    [JsonProperty("name")]
    public string Name { get; set; }
}

With C# nullable reference types enabled, a warning will be shown for the Name property: Non-nullable property is uninitialized. Consider declaring it as nullable.

Let’s look at how we can fix these warnings…

Make the property nullable - Bad!

Following the compiler’s advice, you can update the property and make it nullable:

public class User
{
    [JsonProperty("name")]
    public string? Name { get; set; }
}

Done! No more warnings! However, you now have to check for Name != null anywhere you consume this property…

Add a default value and suppress the warning - Bad!

Another option would be to suppress the warning, and change the User class to the following:

public class User
{
    [JsonProperty("name")]
    public string Name { get; set; } = default!;
}

Done! No more warnings! However, you are now lying to the compiler. When consuming the Name property, you may get a null reference after all.

Add a primary constructor (Newtonsoft.Json) - Good!

With Newtonsoft.Json, you can add a primary constructor that covers all properties of your class. The JSON deserializer will pick this up, and calls the constructor instead of setting the properties directly:

public class User
{
    public User(string? name)
    {
        Name = name ?? "Unknown"; // or throw ArgumentNullException
    }

    [JsonProperty("name")]
    public string Name { get; init; }
}

With this approach, you’ll get rid of nullability warnings without shooting yourself in the foot. If you don’t expect a null value from the JSON, stay close to that expectation and declare the property as non-nullable. In the constructor, you can set a default value, or throw an ArgumentNullException. Don’t blindly accept and propagate null.

Annotations and default values - Good!

Another approach to our problem would be setting a proper default value. In case no value is deserialized, and assuming the JSON deserializer doesn’t explicitly pass in null in such case, the property will be non-nullable and contain an expected default value:

public class User
{
    [JsonProperty("name")]
    public string Name { get; init; } = "Unknown";
}

This can also be accomplished using record classes, which is quite elegant for objects that are solely used for JSON deserialization:

public record User(
    [property: JsonProperty("name")]
    string Name = "Unknown"
);

An alternative would be to use a backing field, and make use of the [AllowNull] attribute that lets callers set null, while being certain that they will never get null when reading from this property.

public class User
{
    private readonly string _name;

    [AllowNull]
    [JsonProperty("name")]
    public string Name
    {
        get => _name;
        init => _name = value ?? "Unknown";
    }
}

Personally, I would not recommend this specific approach. It gets rid of all warnings, but it’s cumbersome to maintain with that backing field. And more importantly, since the [AllowNull] attribute works for the setter only, it can be confusing for consumers of your class.

To summarize, there are many solutions to making your JSON deserialization more reliable with C# nullable reference types. Whether you choose one of the approaches I have listed, or come up with another approach, don’t lie to the compiler, and don’t make life hard on yourself and your team by flowing potential null references around unnecessarily.

Remember that nullable warning suppressions are a code smell, and should be used with care - especially when you suppress warnings on values that are clearly null.

Be careful with Entity Framework

For most frameworks and libraries, nullable reference types and annotations are just hints to the IDE and compiler. That is, until you encounter a framework that uses the annotations for other things, such as Entity Framework.

From the documentation:

A property is considered optional if it is valid for it to contain null. If null is not a valid value to be assigned to a property then it is considered to be a required property. When mapping to a relational database schema, required properties are created as non-nullable columns, and optional properties are created as nullable columns.

In other words: if you update the nullability of an entity in C# code, a migration will be created that changes nullability in the database as well!

But what is considered “updating the nullability”? Adding or removing the nullable annotation (?) is one such change, and merely adding #nullable enable (or enabling nullability project-wide) is another.

In other words: if you enable nullability in a project or file, your database may change.

The Entity Framework documentation covers the various approaches to declaring entity properties and playing nice with C# nullable reference types.

Conclusion

In this final post of our series, we have covered various techniques and tools that are available to migrate your existing projects to use C# nullable reference types.

The benefit (and goal) of annotating your code, is that you get design-time and compile-time hints, from the IDE and the compiler. When adding annotations, steer clear of lying to the compiler. Suppressions are an anti-pattern in all but some cases. When needed, update the design of your API instead of passing around null values without there being a need , other than making a warning go away.

Thank you for reading!

P.S.: I realize this post was also a bit of a love letter for ReSharper (R#) and JetBrains Rider. Both tools are of great help when migrating and annotating your code.

If you have migrated (or are in the process of migrating), what tools are you using? What help do they offer? I would love to hear about those in the comments!

Annotating your C# code - Migrating to nullable reference types - Part 3

Maarten Balliauw — Mon, 25 Apr 2022 02:44:05 +0000

In the previous post, we looked at some internals of C# nullable reference types, and the nullable annotation context.

Today, let’s look at the many options for annotating your code and various ways to help the flow analysis understand your code. As a result, you (and anyone consuming your libraries) will get better and more reliable hints from the IDE and the C# compiler.

The need for more fine-grained annotations

So far, we’ve only been annotating our code with ? to inform flow analysis that a reference type can be null when nullable annotations are enabled. This one annotation may not be enough for all scenarios…

Let’s start with a piece of code that takes a title (like the one from this blog post) and makes a “slug” out of it, a representation of the title that can be used in URLs. It’s a very, very naive implementation, so don’t go using it in production. But it’s sufficient as an example. Also note the #nullable disable directive - the nullable annotation context is disabled in this example.

#nullable disable

public static string Slugify(string value)
{
    if (value == null)
    {
        return null;
    }

    return value.Replace(" ", "-").ToLowerInvariant();
}

The Slugify method can return a string (or null), and accepts a string parameter that can also be a string instance, or null. Let’s convert this method to use nullable reference types.

#nullable enable

public static string? Slugify(string? value)
{
    if (value == null)
    {
        return null;
    }

    return value.Replace(" ", "-").ToLowerInvariant();
}

So far, so good. The logic remained the same, and support for null on both the input and output of this method remains.

Now let’s use this method, pass it a known non-null string, and look at it in the IDE (or compile it).

var slug = Slugify("This is a test");

Console.WriteLine(slug.Length); // Warning - CS8602 Dereference of a possibly null reference.

We know that when we give the Slugify() method a non-null value, a non-null string is returned. However, the compiler’s flow analysis does not know this and flags dereferencing .Length as a warning. It uses the information available from the nullable context, which says both in- and output can be null.

Luckily, there are several attributes available that can help us in cases where the compiler can’t figure things out. In this case, we can use the NotNullIfNotNullAttribute to give the compiler a hint. We can tell it the return value is not null, if a specific named argument is also not null:

[return: NotNullIfNotNull("value")]
public static string? Slugify(string? value)
{
    // ...
}

var slug = Slugify("This is a test");

Console.WriteLine(slug.Length); // All good! Known to be not null

Let’s look at another example. After doing a null check, the C# compiler knows that a specific reference can no longer be null. A quick refresher:

var s = DateTime.Now.Day == 1 ? "" : null;

if (s == null) return;

Console.WriteLine(s.Length); // no warning here, s is known to be not null

The C# compiler only knows that a null check was performed if you write a null check. Yet, it does seem to know string.IsNullOrEmpty() is also doing such a check.

var s = DateTime.Now.Day == 1 ? "" : null;

if (string.IsNullOrEmpty(s)) return;

Console.WriteLine(s.Length); // no warning here, s is known to be not null

How does this work? The string.IsNullOrEmpty() method is annotated with the NotNullWhenAttribute.

public static bool IsNullOrEmpty([NotNullWhen(false)] string? value)
{
    return (value == null || 0 == value.Length) ? true : false;
}

The NotNullWhenAttribute lets the C# compiler know that the reference passed in the value parameter is not null when this method returns false.

There are quite a few of these attributes, with good descriptions and examples in the docs.

Preconditions

Preconditions are used when writing to a parameter, field, or property setter.

In the following example, the Username property getter can not return null, while the property setter does allow null. The AllowNull attribute lets you specify a parameter, field, or property setter accepts null:

private string _username = Guid.NewGuid().ToString();

[AllowNull]
public string Username
{
    get => _username;
    set => _username = value ?? Guid.NewGuid().ToString();
}

Alternatively, DisallowNull lets you specify that a parameter, field, or property setter won’t accept null, even if it’s declared as nullable.

Post-conditions

Post-conditions are used when reading from a field, property, or return value.

In the following example, the MiddleName property getter may return null, while the property setter requires a non-null value. The MaybeNull attribute lets you specify a field, property, or return value may be null:

private string? _middleName = default;

[MaybeNull]
public string MiddleName
{
    get => _middleName;
    set => _middleName = value;
}

Alternatively, NotNull lets you specify that a field, property, or return value won’t return null, even if it’s declared as nullable.

The MaybeNull and NotNull attributes are also useful in combination with generics, as we’ll see later in this post.

Conditional post-conditions

Conditional post-conditions can be used to make method arguments dependent on a method return value.

We’ve already seen NotNullWhen and NotNullIfNotNull earlier, and there is also MaybeNullWhen.

MaybeNullWhen - A non-nullable argument may be null when the method returns the specified bool value.
NotNullWhen - A nullable argument won’t be null when the method returns the specified bool value.
NotNullIfNotNull - A return value isn’t null if the argument for the specified parameter isn’t null.

Helper methods

With helper methods, you can specify whether a certain field is initialized after execution.

In this Person class, the _id field initially is null, even though it’s defined as being non-nullable. The EnsureIdentifier() method is called in the constructor, and makes sure _id does get a value.

public class Person
{
    private string _id;

    public Person() => EnsureIdentifier();

    private void EnsureIdentifier()
    {
        _id = Guid.NewGuid().ToString();
    }
}

However, when compiling, you’ll see a warning emitted:CS8618 Non-nullable field ‘_id’ must contain a non-null value when exiting constructor. Consider declaring the field as nullable.

This can be resolved by adding the MemberNotNull annotation on EnsureIdentifier(), telling the compiler that this method sets the _id field when it returns.

public class Person
{
    private string _id;

    public Person() => EnsureIdentifier();

    [MemberNotNull(nameof(_id))]
    private void EnsureIdentifier()
    {
        _id = Guid.NewGuid().ToString();
    }
}

MemberNotNull lets you specify a member won’t be null when the method returns. There’s also MemberNotNullWhen, which lets you specify the listed member won’t be null when the method returns the specified bool value.

Failure conditions

Lastly, there are two attributes that you can use to specify when no result will be returned, and further analysis is not needed:

DoesNotReturn - The method or property always throws an exception, and thus never returns.
DoesNotReturnIf - If the associated bool parameter has the specified value, the method or property always throws an exception, and thus never returns.

For example, this WriteName() method calls ThrowArgumentNullException() when name is null. Since that method is annotated with DoesNotReturn, the compiler’s flow analysis now knows that the Console.WriteLine below will only run when name is not null.

[DoesNotReturn]
private void ThrowArgumentNullException(string argumentName)
    => throw new ArgumentNullException(argumentName);

public void WriteName(string? name)
{
    if (name == null)
    {
        ThrowArgumentNullException(nameof(name));

        // any code here will never execute
    }

    Console.WriteLine(name.Length);
}

In summary, by annotating your methods with these attributes, the C# compiler can provide you (and anyone who consumes your code) with more accurate warnings. And again, the docs contain good examples on how to use these attributes.

JetBrains Annotations

Introduced many years ago, JetBrains has many annotation attributes that can be used to provide hints to the IDE, even before C# 8 was a thing. If you’re using ReSharper or JetBrains Rider, you can sprinkle these attributes through your code base and get better navigation, type hints, language injections, string formatting, and more.

JetBrains Annotations have been around for years, and many existing code bases have them in use. For example, Hangfire and Unity are shipping their code with JetBrains Annotations, helping their users with information about nullability even before the compiler supported it.

Here’s a summary of the JetBrains Annotations attributes related to nullability:

Attribute	What does it mean?
`CanBeNull`	Indicates that the value of the marked element could be `null` sometimes, so checking for `null` is required before its usage.
`NotNull`	Indicates that the value of the marked element can never be `null`.
`ItemNotNull`	Can be applied to symbols of types derived from `IEnumerable` as well as to symbols of `Task` and `Lazy` classes to indicate that the value of a collection item, of the `Task.Result` property or of the `Lazy.Value` property can never be `null`.
`ItemCanBeNull`	Can be applied to symbols of types derived from `IEnumerable` as well as to symbols of `Task` and `Lazy` classes to indicate that the value of a collection item, of the `Task.Result` property or of the `Lazy.Value` property can be `null`.
`ContractAnnotation`	Let you define expected outputs for given inputs. For example, you can specify that when `null` is passed as an argument, a method returns `null`, and more.

The ContractAnnotation is really powerful: it provides a powerful syntax to describe the input and output conditions of a method.

A simple example would be this TryDeserialize method, where a ContractAnnotation can be added to convey that when the json parameter is null, the method returns false and person will be null.

[ContractAnnotation("json:null => false,person:null")]
public bool TryDeserialize(string json, out Person? person)
{
    // ...
}

I find the ContractAnnotation more clear than C#’s own attributes, as they follow the natural flow: if the input is null, the output is false. C#’s own attributes are a bit backward. For example NotNullWhen tells the compiler that when the output is false, the input must have been null.

Now you may ask, why am I mentioning these annotations? Especially with native annotations in C#8 and beyond? There are a number of reasons.

First, teams that have been using these tools in the past will have an easier migration path towards using C#’s nullable annotations. The JetBrains tools have several automated quick fixes to handle this migration for you, as we’ll see in the next part of this series.

Second, you may be consuming a library that ships JetBrains Annotations with it. If you’re using ReSharper or Rider, the IDE will use those annotations to help you determine whether reference types can be null (or not), even if you’re using the C# annotations.

I won’t say they were first, but… JetBrains were first in paving the road towards better expressing nullability in C# code bases. You can clearly see the similarities between JetBrains flow based analysis and the solution for nullability the C# compiler settled on.

Having annotated libraries out there is of great value when migrating your own code base. Do keep in mind that the C# compiler doesn’t recognize the JetBrains Annotations, but you can use them with Rider and ReSharper to get improved nullability analysis on top of any C# 8 annotations.

Tip: The JetBrains Annotations have several other utilities. For example, you can use them to treat string parameters as ASP.NET MVC controllers/actions, and many more.

Expressing nullability with generics

The nullable annotations can also be applied to generics. Here are a few examples:

List<string> is a collection that contains non-null string elements.
List<string?> is a collection that may contain null elements.
List<string?>? is a collection that may be null, and may contain null elements.

But what with List<T>… Is T nullable or not? Is it a value type or a reference type?

You can use the MaybeNull attribute to tell the compiler about whether a generic method may return null:

[return: MaybeNull]
public T Find<T>(IEnumerable<T> sequence, Func<T, bool> predicate)
{
    // ... find by predicate, or return null ...
}

You can also use generic type constraints to inform the compiler about the capabilities a type argument must have. With C# nullable reference types, there’s a new generic constraint to prevent a generic parameter from being nullable: notnull.

Thanks to the where T : notnull in the following example, WriteToConsole() can only be called with a non-null T item:

#nullable enable

public static void WriteToConsole<T>(T item)
    where T : notnull
{
    Console.WriteLine(item);
}

string? s = null;
WriteToConsole(s); // CS8714 Nullability of type argument 'string?' doesn't match 'notnull' constraint.

The notnull constraint limits the type parameter to be either a value type or a non-nullable reference type. Use the class constraint to ensure a non-nullable reference type, or the class? constraint to allow a nullable reference type.

Nullable reference types in referenced code, libraries and frameworks

Migrating to using nullable reference types becomes easier when your dependencies have nullable annotations in place, whether the native C# attributes or using JetBrains Annotations. This is true for third-party dependencies, including the .NET framework, but it’s also true for first-party dependencies developed by your team or company.

With .NET 6, most if not all of the framework code has been annotated, giving you design-time and compile-time help when interacting with framework code. Having annotations in the framework helps in annotating your own code base. In the following example, Directory.GetParent() may return null, which means your own function will need to also return a nullable string, do a null check, throw an Exception, or work around this possibly null reference in another way.

Once again, it’s important to realize nullable reference types and annotations are not enforced at runtime.

Nullable reference types are a C#-only feature. The .NET Common Language Runtime (CLR) does not distinguish between string and string?, you don’t get runtime safety when it comes to nullability!

Consumers of your code may also use an older C# language compiler or IDE, or have disabled nullable reference types and happily invoke your code that expects a non-nullable parameter with null.

Because of all of this, you still need to do null checks, even with nullable reference types enabled.

#nullable enable

public string Reverse(string original)
{
    if (original == null)
    {
        throw new ArgumentNullException(nameof(original));
    }

    // or:
    // ArgumentNullException.ThrowIfNull(original);

    return new string(original.Reverse().ToArray());
}

Nothing stops you from calling Reverse(null). To guard against null at runtime, you’ll need to check for null and throw ArgumentNullException (or circumvent the null value in another way).

In summary, even with nullable reference types enabled, you still need to do null checks on untrusted and/or external input.

Do you need to redesign your API?

An additional benefit to enabling nullable reference types, is that it may expose API design flaws.

When you have to annotate an API as being nullable because there is one case where it can return null, maybe you will want to re-think that design. Do you want to keep allowing null? Do you want to instead make the API non-nullable, and handle that null case in another way?

Much like with async/await, nullable annotations will need to be present in your entire project to get the most value out of them. But also like with async/await, you may want to set boundaries on where certain values can be null or non-null, and redesign your API accordingly.

Conclusion

In this post, we have seen the many options for annotating your code for nullability. Using these annotations helps the compiler’s flow analysis in understanding your code. Thanks to annotated code, you and folks consuming your code get better and more reliable hints from the IDE and the C# compiler.

In the next and final post in this series, we’ll look at some techniques and tools that are available to migrate an existing code base to using nullable reference types.

Internals of C# nullable reference types - Migrating to nullable reference types - Part 2

Maarten Balliauw — Tue, 19 Apr 2022 02:44:05 +0000

In the previous post, we saw that with nullable reference types enabled, you get better static flow analysis when working on your code. While nullable reference types don’t give you runtime safety, the design-time and compile-time help is priceless!

In this post, we’ll look at some internals of how C# nullable reference types work, and how the C# compiler and IDE use the nullable annotation context.

Under the hood - Intermediate Language (IL)

We’ve already seen there is a difference between nullability for value types and reference types.

Value types are wrapped in a Nullable<T> that gives you a .HasValue property to determine if you can use the wrapped value, or should consider it as null/no value.

Reference types are always nullable, and with nullable reference types (NRT) enabled, you can provide extra context to the IDE and the compiler. With NRT enabled, the C# compiler treats reference types as non-nullable by default, and you can add syntax to annotate them as being nullable.

Consider the following piece of code. Two methods, one returning a nullable integer, the other returning a nullable string.

int? GetInt() => 1;
string? GetString() => "";

These two methods are compiled into the following Intermediate Language (IL):

.method private hidebysig instance valuetype [System.Runtime]System.Nullable`1<int32>
  GetInt() cil managed
{
  .maxstack 8

  IL_0000: ldc.i4.1
  IL_0001: newobj instance void valuetype [System.Runtime]System.Nullable`1<int32>::.ctor(!0)
  IL_0006: ret
}

.method private hidebysig instance string
  GetString() cil managed
{
  .custom instance void System.Runtime.CompilerServices.NullableContextAttribute::.ctor([in] unsigned int8)
    = (01 00 02 00 00)
  .maxstack 8

  IL_0000: ldstr ""
  IL_0005: ret
}

Let’s break this down.

int? GetInt() => 1; is compiled into a .method that returns a value type [System.Runtime]System.Nullable1. The method body pushes the value1onto the stack, and creates a newSystem.Nullable1<int32> that takes the first element from the stack (the 1 that was just pushed). Finally, this new object is returned.
string? GetString() => ""; is compiled into a .method that returns a string. The method body pushes a new object reference to a string literal stored in the assembly metadata (see this post about string literals for more background), and returns it.

The difference between value types and reference types is very clear in IL: the method that returns a value type, returns a Nullable<int>. The method that returns a reference type returns, well, a reference type. There’s no trace of null vs. non-null!

Or is there… In the IL code, GetString() defines a .custom attribute of type NullableContextAttribute, passing the value (01 00 02 00 00) to the attribute constructor. This is an 8-bit integer value (which exists in IL), and it translates to 2.

`NullableContextAttribute` and `NullableAttribute`

The NullableContextAttribute is used to provide code that consumes this method with some extra metadata, that can then be used by flow analysis.

The C# compiler can add this attribute on types and methods, and supports 3 values:

0 - Oblivious - Use the default, pre-C#8 behaviour (everything is maybe null)
1 - Not annotated - Consider the scope as not annotated by default (in other words, every reference type is non-nullable by default)
2 - Annotated - Consider the scope as annotated by default (in other words, every reference type has an implicit ? slapped onto it)

In our previous example, the value of 2 tells the flow analysis that the method return is annotated, in other words, it has a ? annotation.

Let’s rewrite our GetString() method to be non-nullable:

string GetString() => "";

The NullableContextAttribute will now be created with a value of 1:

.custom instance void System.Runtime.CompilerServices.NullableContextAttribute::.ctor([in] unsigned int8)
  = (01 00 01 00 00 ) // int8(1)

In other words: we have a nullable context enabled, and by default everything in scope is considered to not be annotated (in other words, non-nullable).

One more? What about two! Here are two new methods to explore. In C#:

string GetStringA(string? a, string b, string? c) => "";
string? GetStringB(string? a, string b, string? c) => "";

In IL (just the signatures):

.method public hidebysig instance string
  GetStringA(string a, string b, string c) cil managed
{
  .param [1]
    .custom instance void System.Runtime.CompilerServices.NullableAttribute::.ctor([in] unsigned int8)
      = (01 00 02 00 00 ) // int8(2)
  .param [3]
    .custom instance void System.Runtime.CompilerServices.NullableAttribute::.ctor([in] unsigned int8)
      = (01 00 02 00 00 ) // int8(2)
  .maxstack 8

  // ...
}

.method public hidebysig instance string
  GetStringB(string a, string b, string c) cil managed
{
  .custom instance void System.Runtime.CompilerServices.NullableContextAttribute::.ctor([in] unsigned int8)
    = (01 00 02 00 00 ) // int8(2)
  .param [2]
    .custom instance void System.Runtime.CompilerServices.NullableAttribute::.ctor([in] unsigned int8)
      = (01 00 01 00 00 ) // int8(1)
  .maxstack 8

  // ...
}

For GetStringA(), the compiler did not emit NullableContextAttribute. Instead, it annotated the method parameters: param [1] and param [3] are annotated with NullableAttribute and a value of 2. In other words, both parameters should be treated as being annotated (with a ?). By convention, the presence of a NullableAttribute without NullableContextAttribute also causes everything else to be treated as not annotated (in other words, non-nullable).

For GetStringB(), the compiler emitted two attributes. By default, a NullableContextAttribute with value 2 is applied. In other words, every reference type in this method is annotated with a ?. Except for one: param [2] (string b) got a NullableAttribute that has value 1 (no annotation, so non-nullable).

That’s some proper compiler magic going on right there! The C# compiler tries to emit as few attributes as possible, so that flow analysis in the IDE or in a consuming assembly does not have to process too much metadata: just a default nullable context, and any exceptions to that default.

Nullable annotation context

So far, we’ve seen how the nullable context on a type or method changes the Intermediate Language (IL) that is emitted. This nullable context gives consuming code an idea of how to treat nullable reference types.

The next step is telling flow analysis and the compiler what you want to do with that information, by setting the nullable annotation context. The nullable annotation context has 4 settings, and can be defined project-wide, or for every file separately:

disable - makes the compiler behave like it did pre-C# 8.0 (no NRT). You can not use ? on reference types.
enable - enables null reference analysis and all language features.
warnings - enables null reference analysis, and shows warnings when code might dereference null.
annotations - enables language features and lets you use ?, but does not enable null reference analysis.

Defining nullable annotation context project-wide

To define the nullable annotation context project-wide, you can use the <Nullable>...</Nullable> property in your project file (or any MSBuild file that is included). Here’s an example of a project that sets the nullable annotation context to enable for the entire project:

<Project Sdk="Microsoft.NET.Sdk">

    <PropertyGroup>
        <OutputType>Exe</OutputType>
        <TargetFramework>net6.0</TargetFramework>

        <Nullable>enable</Nullable>
    </PropertyGroup>

</Project>

Defining nullable annotation context per file

You can also define the nullable annotation context in individual files, using the #nullable preprocessor directive:

#nullable enable - enable nullable annotation context and nullable warning context
#nullable disable - disable nullable annotation context and nullable warning context
#nullable restore - set the nullable annotation context and nullable warning context to the project-level setting
#nullable enable/disable/restore warnings - enable/disable/restore just the nullable warning context
#nullable enable/disable/restore annotations - enable/disable/restore just the nullable annotation context

It’s also possible to use the #nullable directive multiple times per file, and to enable/disable a specific context at various places in the same file. This may be useful when migrating large files to using nullable reference types, as you can split bits that you have already migrated from those you haven’t yet.

#nullable disable

// Warning: CS8632 - The annotation for nullable reference types should only be used in code within a '#nullable' annotations context.
string? SomeMethod() => "";

#nullable enable

// No warning - ? is allowed because nullable context is enabled
string? SomeOtherMethod() => "";

Which nullable annotation context should you use?

Good question! As always, “it depends”.

For new projects, it’s a good idea to fully enable the nullable annotation context at the project level. This way, you get the benefits of better static flow analysis from the first line of code you start writing.

For existing projects and code bases, you’ll have to start with deciding what the default setting for your project will be. The end goal of migrating to nullable reference types, is to be able to have nullable warnings and annotations enabled in all projects. But that may not be the best default to start your migration with. Let’s look at some options.

`disable` as the default

One way to start migration, is by setting disable as the default (or not adding <Nullable /> to your project file at all). Doing so gives you the opportunity to go through your project and add #nullable enable file by file, without drowning in warnings. When your entire project uses nullable reference types, you can switch to enable.

`enable` as the default

Another way to start migration, is to go all-in and set enable as the project default - this is the end goal so why not set it from the start? This option may (and most probably will) give you lots of warnings to work through. You’ll have to either add nullable annotations, add the null-suppressing operator to some statements, or add #nullable disable for some files as you work your way through the code base.

`warnings` as the default

With warnings as the project default, null reference analysis is enabled and warnings are shown when your code might dereference null. You’ll see warnings where the compiler’s flow analysis infers potential null references, and you can account for this. Either by adding null checks, or by adding #nullable enable to a file (or a section of a file) and applying ?.

The warnings context is a good way to start exploring an existing codebase and improve null checks. It will make your project safer in terms of nullability, but unless you change the nullable annotation context, you’ll get no benefit of additional annotations in your code base.

Tip: The warnings context helps improve existing code bases with better flow analysis. To make sure you and your fellow developers address these warnings, you can treat these nullable warnings as errors. Set the following two properties in your project file:
&lt;Nullable&gt;warnings&lt;/Nullable&gt;
&lt;WarningsAsErrors&gt;Nullable&lt;/WarningsAsErrors&gt;

`annotations` as the default

When you set annotations as the default, you can start using ? in your project. Flow analysis will be disabled, though, so you won’t get any warnings about potential nullability issues.

In my opinion, this mode does not make a lot of sense. Yes, you can use the language features, but you get no assistance from the IDE or compiler when something is not right.

Conclusion

In this post, we’ve seen some internals of C# nullable reference types. The compiler generates NullableContextAttribute and NullableAttribute usages when compiling C# code to Intermediate Language (IL), providing metadata about nullability.

The metadata added by the compiler is used in various ways, depending on the nullable annotation context you specify in your project or separate files.

In the next post, we’ll look beyond ? and cover the many options for annotating your code and helping out flow analysis to give you better and more reliable results.

Nullable reference types in C# - Migrating to nullable reference types - Part 1

Maarten Balliauw — Mon, 11 Apr 2022 02:44:05 +0000

The C# nullability features introduced in C#8 help you minimize the likelihood of encountering that dreaded System.NullReferenceException. Nullability syntax and annotations give hints on whether a type can be nullable or not. Better static analysis is available to catch unhandled nulls while developing your code. What’s not to like?

Introducing explicit nullability into an existing code bases is quite an effort. There’s much more to it than just sprinkling some ? and ! throughout your code. It’s not a silver bullet either: you’ll still need to check non-nullable variables for null.

Throughout a series of blog posts, we’ll learn more about C# nullability, and cover some techniques and approaches that worked for me when migrating an existing codebase to using nullable reference types.

Let’s start at the beginning.

Reference types, value types, and null

You may have heard of null being the “billion-dollar mistake”. In 2009, Tony Hoare, the creator of the ALGOL W programming language, apologized for his billion-dollar mistake: null references. His original goal was to ensure that using any reference would be absolutely safe, validated with compiler checks. But supporting a null reference was easy to do, so he did - leading to innumerable errors, vulnerabilities, and system crashes in many systems out there. Other languages copied this idea, resulting in null references probably causing a billion dollars of pain and damage ever since.

Nullable reference types have always been part of C#: a reference type can be either a reference, or null. Consider the following example:

string s = GetValue();
Console.WriteLine($"Length of '{s}': {s.Length}");

When s is not null, a message will be written to the console. But what happens when s is null? Exactly: a NullReferenceException is thrown when accessing the Length property of… null.

We can add a check for null before accessing this property:

string s = GetValue();
Console.WriteLine(s != null
    ? $"Length of '{s}': {s.Length}"
    : "String is null.");

How do you know whether you need that check? If GetValue() never returns null, how would you know if you need to check s when accessing its properties?

For value types, such as int, bool, decimal, structs like DateTime or custom implementations, etc., you can make use of Nullable<T> to make those value types “nullable”, and have a safe way of accessing them.

Note: value types can’t really be null - they are values, not references. With Nullable<T>, you’re wrapping the value to be able to keep track of this additional null state.

There is some compiler magic that converts int? into Nullable<int>. When accessing a nullable value type, you know it will either be null or contain a value, and that a check has to be made. With a nullable DateTime, we could write the above example as follows:

DateTime? s = GetValue();
Console.WriteLine(s.HasValue
    ? $"The date is: {s.Value:O}"
    : "No date was given.");

The key difference here is intent. With our string before, you have no idea if null is to be expected. It can never happen, it may happen - you only know by looking into the GetValue() function (or take the safe approach and always add null checks). With value types, the intent is more clear. A DateTime can never be null, whereas a Nullable<DateTime> tells you that a check for null (or .HasValue) is required.

What are nullable reference types?

With C#8, nullable reference types (NRT) were introduced and help convey the intent that a reference can be null or will never be null.

Let’s start by pointing out that reference types have always been nullable, as we saw in the introduction. They have been since the first version of C# - string s = null is perfectly fine, and every reference type really always has been a nullable reference type.

Why am I pointing this out? Reference types have been nullable forever, and now we’re flipping that idea by thinking of them as non-nullable by default, and adding syntax to annotate them as being nullable. The annotations help while writing and compiling your code, but don’t provide a runtime safety net. Let’s have a look at what that means.

With nullable reference types enabled, the earlier example tells the compiler that we expect s to not be null (otherwise we’d declare string? s), and we can safely remove the null check…

string s = GetValue();
Console.WriteLine($"Length of '{s}': {s.Length}");

string? GetValue() => null;

…or can we? Surprisingly, you can compile and run the above code, even with GetValue() returning a string that is null. You’ll just see a NullReferenceException is thrown when accessing the Length property of… null.

So what are C#8 nullable reference types? I like to call them nullable annotations - NRT’s annotate your code with what their nullability can be.

When you compile the above code, or look at it in an IDE, you’ll see several warnings that mention you are doing something that may haunt you when you run the application:

CS8600 - Converting null literal or possible null value to non-nullable type.
CS8602 - Dereference of a possibly null reference.

Thanks, IDE! Thanks, C# compiler! While this may all blow up at runtime, at least I get enough warnings to go fix this code. And that fixing can be done almost automatically, with a quick-fix.

After applying these fixes, we’re essentially back at our initial example, but this time annotated with what we expect. Again, the IDE and compiler will help us out, and if we change that GetValue() function to return a non-nullable string, the tooling will tell us we’re doing redundant null checks, and offer to remove them.

string? s = GetValue();
Console.WriteLine($"Length of '{s}': {s.Length}");

string GetValue() => "";

Flow analysis

There are some interesting things when it comes to nullability and flow analysis.

For example, var is always considered nullable. The C# language team decided to always treat var as nullable, and to rely on flow analysis to determine when the IDE or compiler should warn.

In our example, the nullability of s is determined by the annotations of the GetValue() function.

var s = GetValue();
Console.WriteLine($"Length of '{s}': {s.Length}");

string GetValue() => "";

Many IDEs will render inlay type hints to help you determine what a var could be, based on flow analysis. Here’s a quick example. Note the inlay hint on var s change when nullability of the GetValue() function changes. Also note the squiggly at s.Length, where flow analysis determines whether dereferencing Length is potentially going to get you into trouble.

Cool, eh? Things become even cooler when you do null checks. Here are two methods:

void MethodA(string? value) {
    Console.WriteLine(value.Length); // CS8602 - Dereference of a possibly null reference.
}

void MethodB(string? value) {
    if (value == null) return;

    Console.WriteLine(value.Length); // No warning, null check happened before
}

In MethodA(), we’re not doing a null check for the value parameter, and flow analysis will warn about dereferencing a possibly null reference. In MethodB(), flow analysis determined we are doing a null check, and is fine with accessing .Length since there’s no chance of this line of code being hit with a null reference.

The null-forgiving operator

Before wrapping up, there’s one more topic I want to get into. The null-forgiving operator (also known as the null-suppression operator, or the dammit-operator), !.

When using nullable reference types, an expression can be postfixed with ! to tell the IDE and compiler’s static flow analysis to “ignore” the null state of the expression.

Here are two examples, where ! is telling the flow analysis that a nullable state can be ignored:

string? a = null;
Console.WriteLine(a!.Length);

string? b = null!;
Console.WriteLine(b.Length);

Of course, this is not ideal - both of the above examples will throw NullReferenceException when running the app.

There are valid use cases, though. Flow analysis may not able to detect that a nullable value is actually non-nullable. You can suppress those cases, if you know a reference is not going to be null. In the following example, IsValidUser() checks for null, so we can suppress the null warning we’d get otherwise in the Console.WriteLine call:

var user = _userManager.GetUserById(id);
if (IsValidUser(user)) {
    Console.WriteLine($"Username: {user!.Username}");
}

public static bool IsValidUser(User? user)
    => !string.IsNullOrEmpty(user?.Username);

Note: there are better ways to solve some of these suppressions, using special attributes. I’ll cover these in the next post.

Another case is when migrating a code base that does not yet use nullable reference types. You may want to (temporarily) suppress warnings on a certain code path, and not yet care about nullability while you are working on another part of the code base.

Yet another case could be with unit tests. Maybe you do want to check what happens if someone passes null into a function that expects a non-null reference. In those situations, you could pass null! and see what happens.

Do keep in mind the null-forgiving operator effectively disables flow analysis, and could easily hide nullability bugs in your code base. Use it with care, and consider the use of the null-suppressing operator a code smell.

Conclusion

With nullable reference types enabled, you get improved static flow analysis on your code that helps determine if a variable may be null before dereferencing it. Variable annotations can be used to explicitly declare the intended null-state for that variable - adding ? to convey a null reference is possible.

Nullable reference types don’t give you runtime safety when it comes to nullability, but design-time and compile-time help is great to have!

In the next post, we’ll look at some internals and the nullable annotation context.

A journey towards SpeakerTravel - Building a service from scratch

Maarten Balliauw — Mon, 08 Nov 2021 02:44:05 +0000

For close to two years now, I’ve had SpeakerTravel up & running. It’s a tool that helps conference organizers to book flights for speakers. You invite speakers, they pick their flight of choice (within a budget the organizer can specify), and the organizer can then approve and book the flight with a single click.

In this post, I want to go a bit into the process of building this tool. Why I started it in the first place, how it works, a look at it from the business side, and maybe a follow-up post that covers any questions you may have after reading.

Why I started building a travel booking tool

Before COVID threw a wrench in offline activities, our user group was organizing CloudBrew, a 2-day conference with speakers from across the world (mostly Europe).

Every year, I was complaining on Twitter around the time travel for those speakers needed to be booked. Booking flights for a speaker would mean several e-mails back-and-forth about the ideal schedule, checking travel budgets, and then sending the travel confirmation. And because our user group is a legal entity, we’d need invoices for our accountant, which meant contacting the travel agency and more e-mails.

When we started, we did all of this for 5 speakers, which was doable. Then we grew, and in the end needed to do this for 19 speakers. Madness!

That got me thinking, and almost pleading for someone to come up with a solution:

Startup idea: "Give travel site a bunch of e-mail addresses and budgets. Site lets those people select flights within that budget. I say yes/no and get billed." - Would love this for conference organizing!

— Maarten Balliauw (@maartenballiauw) July 4, 2018

Alas, by the time we had that 19 speaker booking coming up, no such solution came about, and we were once again doing the manual process.

How flight tickets work…

In the back of my mind, the idea stuck. Would it be possible to build a solution to this problem, and make booking travel for speakers at our conference an easier task?

Of course, building the app itself would be possible. It’s what we all do for a living! But what about the heart of this solution… You know, actually booking a flight ticket in an automated way?

After researching and reading a lot, it seems that booking a flight ticket always consists of 4 steps:

You search an inventory of available seats for a flight combination;
For that flight combination, a price is requested;
For that flight combination, a booking is created;
For that booking, tickets are issued.

Book flights via any website, and you’ll go through these steps. There’s a reason for this:

The flight inventory is really a big database with all seats on all (or at least, many) airlines. As far as I could find, airlines populate this database a coule of times a year. It does not contain prices, just seats and conditions to book seats.
Pricing checks a given seat with the airline (or other party in between). Requesting a price means the airline can give an actual price for a seat. They can also track interest in a specific seat/group of seats, and price accordingly.
Booking reserves the seat, and removes that seat from the big flight inventory database. Ideally, booking has to happen soon after pricing. If no tickets have been issued after a couple of hours, the seat is made available again.
Issuing tickets confirms the seat, and gives you the actual ticket that can be used to board a plane. Having these two steps separate means that in between, a booking website can ask you for payment, and only when that is confirmed, issue tickets.

So in short, I needed something that could perform all of these steps somehow. More research!

Global Distribution Service (GDS)

One of the first services that popped up were different Global Distribution Services (GDS) for air travel. The world has many of them. You may have heard of Amadeus, Sabre or Travelport, but there are others.

These GDS are an interoperability layer between inventories from airlines, travel agents, and more. They have software in place to handle interactions between all parties involved (airlines, travel agents, hotels, …), and until a few years ago, were always involved in booking flights. Nowadays, airlines often sell their inventory directly, without these middle-men involved.

I explored various GDS’, and quickly found that this was not the way to go. First, they expect certain volumes of sales. I contacted one of them, and essentially got laughed at when I said I wanted to book around 20 flights a year. Second, from a technical point of view, a lot of them had documentation available that talked about XML-over-SOAP, WS-* standards, and all that. Been there, done that, but prefer the more lightweight integrations of recent years.

Flight search affiliate programs

There are a number of affiliate programs out there that provide an API that you can use to search flights (including an approximate price), and give you a link to the booking site. Examples are Travelpayouts and SkyScanner.

The conditions for using these APIs were somewhat restrictive for my use case, but e-mailing one of them confirmed this use case was something that could fit.

Let the speaker search and request a flight, and then the organizer would click through and make the booking. This would still mean entering credit card details and invoice address a number of times, but it could work.

Online Travel Agencies (OTA)

Somewhere in-between GDS and affiliate programs, there are the Online Travel Agencies (OTA) and the likes. These companies are travel agents, and have their contracts with zero, one or several GDS, airlines, and more.

Searching this space, I found a couple of them that had APIs available for the above 4 steps - which seemed promising as it would give full control over the booking process (including automation of sending the correct invoice details when purchasing a ticket):

After contacting them all, some responded only after a couple of weeks, others had requirements in terms of number of tickets sold (volume), and this got me disillusioned.

A travel agent from Sweden

Having talked with a couple of folks about this idea and finding an API, a friend suggested I contact a travel agent they knew well as they could be able to help.

We had a long call about the idea, and they were very helpful in providing some additional insights into the world of flight booking. They were using the TravelPort GDS themselves, and were building their own API on top of that to power their own websites. Unfortunately, they weren’t sure it would ever get completed, so this wasn’t a viable solution.

Nevertheless, lesson learned: it never hurts to talk, even if it’s just for sharing insights and learnings.

AllMyles

Some weeks after my disillusion with OTAs, I was searching the Internet once more and found another service: AllMyles.com.

I decided to get in touch with some questions about my use case and low volumes. With zero expectations: I considered this my last attempt before shelving the entire idea.

Responding in 3 days would have been a record, but these folks responded in 30 seconds (!). A good 10 minutes later I was on Skype with their founder. We chatted about the service I wanted to build, and he even gave some thoughts on how to implement certain parts and workflows.

On their website, a 30-day trial of their staging environment was promoted, and their founder confirmed this was flexible if needed. So I decided to go with this and experiment with the API to see what was possible and what was not, and maybe finally start building this application!

The business side…

With the AllMyles API docs in hand, I set out to writing some code and experimenting with their staging environment. All seemed to work well for my use case.

There was one thing in the way still… To get production access, a one-time certification fee of 3000 EUR would have to be paid. Definitely better than the volume requirements of other solutions, but still quite steep for booking 20 flights a year.

What if this tool would be something that can be used by any conference out there, and I charge a small fee per passenger to cover this certification fee and other costs?

Time to put on the business hat.

CENTS

A couple of years ago, a friend recommended reading The Millionaire Fastlane by MJ DeMarco. It’s a good book with ideas on getting out of the rat race that controls many of us, and very opinionated. You may or may not like this book. There’s one idea from the book that stuck in my head though: CENTS.

CENTS is an acronym for the five aspects on which any idea can be vetted for viability as a business. It’s not a startup canvas or anything, just a simple way of checking if there is some viability:

C ontrol - Do you control as many elements of the business as possible, or would something like a price or policy change with a vendor mess with your business?
E ntry - How hard is entering this market? Can anyone do it in 10 minutes, or would they need a lot of time, money, and other resources?
N eed - Does anyone actually need this thing you are thinking about?
T ime - Will you be converting time into money, or can you decouple the two and also earn while you’re asleep?
S cale - Can you see this scale, are there pivots that would work, …

Before diving into the deep and coughing up that certification fee (and building the tool), I wanted to check these…

For flight booking, C ontrol is never going to be the case. Someone is flying the airplane, someone handles booking. There are parties in between you and that flight, and there’s no way around that. From my research, I knew if really needed I could find another OTA or GDS, and go with that, so I felt there was just enough control to give this aspect a green checkmark.

E ntry was steep enough: that certification fee, research, building the app. Something everyone could overcome, but definitely not something everyone would do. As an added bonus, I had to figure out some tricks to find the same flight twice: once by the speaker making the search, once by the conference organizer to confirm booking. Pricing and booking have to be close together (as in, 20-30 minutes), but for SpeakerTravel there could even be a few days between both parties doing this. In any case, it requires some proper magic to get this right and fine the same (or a very comparable) seat. So E ntry? Check!

The N eed aspect was easy. There are lots of conferences out there that are probably going through the same pain with booking flights. Check!

Same with T ime. This would be a software-as-a-service, that would allow folks to do self-service booking and payments, even when I’m not around. Check!

Finally, S cale. This solution could work for IT conferences, medical conferences, pretty much anything where a third party would pay for someone else’s flights. Business travel could be a pivot, where employees could book and employers would pay. Another pivot could be handling travel for music festivals, etc. So definitely not a hurdle in the long run!

In short: it made ~~sense~~ CENTS!

Legal requirements

Building a tool for our own conference is one thing, building it for third-party use is another. Could I sell flight tickets from my Belgian company?

Instead of trying to figure this out myself, I asked for advise here from a lawyer. The response came in (together with an invoice for their time researching), and for my Belgian company there were a few things to know about:

Flights-only is fine. You’re never selling flights, you are facilitating a transaction between the traveler and the airline.
If you combine flights and hotels, flights and rental cars, etc., you’re selling travel packages. Travel packages have stricter requirements.

Great! So I could go ahead with flights (and only flights), and start building the app!

Payments

While building the app (more on that later), I also was thinking about how to handle flight ticket payments… I’d have a fee per traveler (fixed), and the flight fare itself (variable, and one I’d have to pay directly to AllMyles).

The two-step ticket issuing seemed like a perfect place to shove in a payment gateway, for example Stripe, and collect payment before making the actual booking through the API.

Unfortunately, none of the payment gateways I found let you do “risky business”. All of them have different lists of business types that are not allowed, and travel is always on those lists. One payment gateway from The Netherlands confirmed they could support my scenario, but after requesting written confirmation that stance changed. In other words: credit cards were not an option.

For now, I decided to go with an upfront deposit, to ensure flight fares can be paid when someone confirms their booking.

Building SpeakerTravel

With a good idea in mind, and a blank canvas in front of me, it was time for the excitement of creating a new project in the IDE!

The most important question: Which project template to start with?

Attempt to a single-page application…

Since I’d already built some API integration with AllMyles in C#, at least part of the application would probably be ASP.NET Core. With close to no experience with single page applications at the time, I thought this would be a good learning experience!

So I went with an ASP.NET Core backend, IdentityServer, and React.

About an hour of cursing on a simple “Hello, World” later, React was replaced with Vue.js which seemed easier to get started with. I did have to replicate the ASP.NET Core SPA development experience (blog post) to support Vue.js, but that was fun to do and write about.

What wasn’t fun though, was the slow-going. New to Vue.js, a lot of things went very slow while building. After 2 weeks of spending evenings on just a login that worked smoothly, I started wondering…

“Am I doing this to solve a problem, or to learn new tech?”

Building this thing over the weekend and in the evening hours, I reconsidered the tech stack and started anew.

…replaced with boring technology

This time, I started with an ASP.NET MVC Core project. Individual user accounts using ASP.NET Core Identity, Entity Framework, and SQL Server. A familiar stack for me, and a stack in which I was immediately productive.

A few hours into development, I had the login/register/manage account pages customized. The layout page was converted to load a Bootswatch UI theme (on top of Bootstrap), and was starting to get into building the flows of inviting speakers, searching flights (with 100% made up data), approving and rejecting flights, and all that. This was finished in a week or 6 and then another few weeks to properly integrate with AllMyles’ staging environment.

While developing the app, a lot of new ideas and improvements popped up. I tried to be ruthless in asking myself “do I really need this for version 1?”, and log anything else in the issue tracker and pick it up in the future. This definitely helped with productivity.

Some fun was had implementing tag helpers to show/hide HTML elements (blog post), which realy works well to make certain parts of the UI available based on user permissions and roles.

The first version was ready near the end of August 2019, including a basic product website that is powered by a very simple Markdown-to-HTML script that seems to work well.

The application itself was built and deployed on the following stack:

ASP.NET Core MVC + Razor pages for the scaffolded identity area, on .NET Core 2.1
Bootstrap and Bootswatch for UI
A sprinkle of jQuery
Hangfire for background jobs (the actual bookingm, sending e-mails, anything that’s async/retryable)
SQL Server LocalDb for development, Azure SQL Database for production
Azure Web Apps for the app and product website
Private GitHub repository
Azure DevOps to build and deploy
SendGrid for sending e-mails

Overall, this was and is a very familiar stack for me, and as a result a stack in which I was immediately productive. Server-side rendering is fine 😉 And .NET is truly great!

When you have an idea you want to build out, I can highly recommend going with what you know - unless of course the goal is exploring another tech.

The domain model

When I asked folks on Twitter for what they wanted to see in this post, Cezary Piatek wanted to know about the domain model.

From a high-level, the domain model of this application is simple. There’s an Event that has Travelers, and at some point, they will have a Booking.

For every traveler, the system keeps a TravelerStatus history, which represents state transitions. From invited, to accepted, to bookingrequested, to confirmed/rejected/canceled, to ticketsissued, and potentially back to the start where a rejected traveler goes to accepted again so they can make a new search.

The TravelerStatus history is evaluated for every traveler, and the system takes these into account. In fact, they are somewhat visible in the application UI as well (though some of these state transitions are combined for UX purposes).

When a Traveler requests a booking, some PII is stored. Passenger name, birth date, and whatever the airline requires to book a given seat. This data is stored as a JSON blob - the fields are dynamic and may differ depending on the airline. This data is always destroyed after tickets are issues, the booking request was rejected, or when the booking was still waiting for approval but the event has concluded 10 days ago.

For flight search and booking, the domain model is a 1:1 copy of what AllMyles has in their API. Looking at other APIs, it’s roughly the standard model in the world of flights. A Search returns one or more SearchResults. Each of those has one or more Combinations, typically flights that have the same conditions and price, but different times. E.g. a shuttle flight from Brussels to Frankfurt may return 3 combinations here - same price, and conditions, just 3 different times during the day. A Combination can also have upgrade and baggage option. The booking itself is essentially makign a call that passes a given Combination identifier (and what options are selected on top).

Ready for take-off!

The app was deployed (targeting AllMyles staging), and I requested certification (coughing up the initial fee - no turning back now!). This process took a couple of days, but at some point I was given production access and SpeakerTravel was live!

This was right on time for our CloudBrew conference in 2019, and it was really exciting to see folks request flights, booking them via the API, and seeing actual flight tickets sent out by airlines. Not to mention, much easier in terms of workload and back-and-forth compared to the manual process that triggered this entire endeavour! And speakers themselves also enjoyed this workflow:

Massive props to @CloudBrewConf - their travel booking system for the speakers has really raised the bar!

— Paul Stack (@stack72) August 14, 2019

Thanks, Paul 🤗

Very quickly, a couple of organizer-friends jumped aboard as well. And for a conference I was attending myself, I used it to book a flight in my own name. Pretty cool!

First time taking a flight booked through my own @SpeakerTravel_ - pretty cool to fly on a ticket you issued yourself 😎

— Maarten Balliauw (@maartenballiauw) January 15, 2020

A couple of conferences later, some bugs were ironed out, some feature requests were handled, and the certification fee was covered. Business-wise, and conveniently brushing aside time spent building this thing, SpeakerTravel was break-even!

COVID-19 💊 and working on the backlog

And then, half a year after release, a pandemic hit the world. Conferences all went online, travel virtually halted, and no new conferences onboarded SpeakerTravel for a long time.

This was a bummer, but a good time to work on that backlog of features I wanted to add. Some technical debt got fixed, and thanks to fast release cadences in both the front-end and .NET world, I’ve been upgrading a lot of things, many times.

Today’s tech stack:

ASP.NET Core MVC + Razor pages for the scaffolded identity area, on .NET 6.0 RC2
Bootstrap and Bootswatch
A sprinkle of jQuery (that I want to replace with HTMX)
Hangfire for background jobs (the actual bookingm, sending e-mails, anything that’s async/retryable)
SQL Server LocalDb for development, Azure SQL Database for production
Product website and application are Docker images now, deployed to Azure Web Apps for Containers
JetBrains Space for Git, CI/CD, and container registry
Mailjet for sending out e-mails. Smaller company, better support.

Note: If you’re interested in seeing CI/CD with Space, check this Twitter thread.

What’s next?

Good question! I think this question can be split as well…

What’s next on the technical side?

Let’s start with this one. As in the past months, working on some items from the backlog and just keeping things up to date. Very high on my wishlist is ripping out jQuery and replacing the few bits that require client-side interactivity with HTMX.

One of the things I do want to try at some point is seeing if I can run the entire stack on Kubernetes, but that’s purely out of personal interest.

Any other nerd snipes are welcome in the comments!

What’s next on the business side?

What’s immediately next, is definitely uncertainty. We’re still in a pandemic, and while parts of the world seem to be evolving into the right direction for SpeakerTravel, it’s unsure when in-person conferences will pick up again.

Apart from infrastructure, there’s no real cost to running the application, so I can be patient on that side and keep pitching it to anyone I meet, and provide good support for those who do sign up in the meanwhile.

Speaking of which, I’m super happy that since September 2021, a few conferences have been using the product for in-person travel!

Why not pivot?

A question I got recently was: Why not pivot to business travel? - great idea!

Earlier in this post, I described the model where employees could search and pick travel options, and the company can approve and pay. This would indeed be a great pivot, but there are a couple of things holding me back on this:

It’s a very crowded market (with some big players like American Express). This is not a big issue though, it validates there is a market, but it would need quite some effort to get traction.
I’d have to expand from flights into flights + hotels + cars. While possible in terms of APIs, it does require fulfilling some extra regulations.

Both of these would mean going bigger than what I currently want to handle.

Conclusion and Takeaways

Sometimes, you have a story in you that you just want to write down. This was one of those.

Instead of sharing the event of having SpeakerTravel online, I wanted to share the story about the process that brought it about. Maybe we all focus on the event too much, and not enough on the process towards the event.

Social media consists of short bits, while blogs, articles and tutorials about the process have so much value. Leave breadcrumbs for those on a similar path like you in the future.

Speaking of that: if there’s anything in this blog post you would like to see a follow-up on with more details, let me know via the comments.

Take care!

DEV Community: Maarten Balliauw

Discriminated Unions in C#

Running Large Language Models locally – Your own ChatGPT-like AI in C#

Towards open-source models and execution – A little bit of history…

Getting started with SciSharp/LLamaSharp

1. Download a model

2. Create a console application and install LLamaSharp

3. Initializing the LLaMA model and creating a chat session

Wrapping up

Getting rid of warnings with nullable reference types and JSON object models in C#

Option 1: Make the property nullable

Option 2: Add a default! (please don’t)

Option 3: Add a primary constructor

Option 4: Annotations and default values

Option 5: Use a required property

Mastodon on your own domain without hosting a server

How to discover Mastodon account via custom domain

Bonus: Discovering folks from Twitter

Rate limiting in web applications - Concepts and approaches

Introduction to rate limiting

Why rate limiting?

What is rate limiting?

What should I rate limit?

Naive rate limiting

Rate limiting algorithms

Quantized buckets / Fixed window limit

Token buckets

Deciding rate limits

Which resources to rate limit?

What are sensible limits?

Will you allow bursts or not?

What will be the partition/identifier?

Decide on exceptions

Responding to limits

Where to store rate limit data and counters?

Where to apply rate limiting?

Monitoring and circuit breakers

Wrapping up

ASP.NET Core rate limiting middleware in .NET 7

What is rate limiting?

Rate limiting in ASP.NET Core

Configure what happens when being rate limited

Types of rate limiters

Queue requests instead of rejecting them: QueueLimit

Create custom rate limiting policies

Policies for rate limiting groups of endpoints

Rate limiting policies with ASP.NET Core Minimal API

Rate limiting policies with ASP.NET Core MVC

ASP.NET Core rate limiting with YARP proxy

Wrapping up

How to test ASP.NET Core Minimal APIs

Prerequisites

Create a test project

Update the test project

“Hello World” and the ASP.NET Core test server

Update the Minimal API project

Test different payloads and HTTP methods

Customizing the service collection

Conclusion

Techniques and tools to update your C# project - Migrating to nullable reference types - Part 4

Pick your approach: there is no silver bullet

Start at the edges and work inwards

Add annotations or redesign your code

Don’t be afraid of null

Nullable warning suppressions should be temporary

Where is this value coming from? Where is it being used?

JetBrains Annotations to C# nullable annotations

Determine nullability based on null checks

What about third-party libraries and external code?

Libraries with C# annotations

Libraries with JetBrains Annotations

Libraries without annotations

Pessimistic analysis

Deserializing JSON

Make the property nullable - Bad!

Add a default value and suppress the warning - Bad!

Add a primary constructor (Newtonsoft.Json) - Good!

Annotations and default values - Good!

Be careful with Entity Framework

Conclusion

Option 2: Add a `default!` (please don’t)

Option 5: Use a `required` property

Queue requests instead of rejecting them: `QueueLimit`

Don’t be afraid of `null`

Determine nullability based on `null` checks

`NullableContextAttribute` and `NullableAttribute`

`disable` as the default

`enable` as the default

`warnings` as the default

`annotations` as the default