DEV Community: nausaf

Use .env files for .NET local development configuration with VS Code

nausaf — Mon, 14 Jul 2025 21:33:51 +0000

The Problem with .NET Development Configuration

Most ecosystems I use - Node.js, Docker/Compose, Terraform - expect configuration to come from environment variables.

In production, and other non-local environments, the hosting service loads data from configuration stores (e.g. Azure App Configuration and AWS AppConfig) and secret managers (like Azure Key Vault and AWS Secrets Manager) and sets these as environment variables before launching code.

For local development, config lives in .env files and is loaded automatically into environment variables by the framework or the launcher. Such .env files also typically contain secrets; this is mitigated by excluding them from the repo in .gitignore.

Either way - and in any environment - the app code only ever looks for its configuration in environment variables, which is in keeping with the philosophy of 12-factor apps.

Except in .NET!

In .NET, production configuration would still get loaded from environment variables that are injected by the hosting platform (e.g. from Azure App Configuration references or Key Vault references).

However, in local development, a .NET project loads configuration from an assortment of sources. This makes local .NET config stick out like a sore thumb when working in polyglot workspaces, where everything else just uses .env files.

For example, in a workspace in which I have a .NET Minimal API alongside a Next.js frontend and a Docker Compose compose.yaml, both Next.js and Docker Compose can take in config data in .env files. Here, I would like to use an .env file such as the following to configure the .NET API also:

ASPNETCORE_ENVIRONMENT=Development
ASPNETCORE_URLS=http://localhost:3022
ALLOWED_CORS_ORIGINS=http://localhost:3020
ConnectionStrings__EShop=<redacted>

Yet, instead of an .env file, we have the following sources:

appSettings.*.json configuration files. There are usually several of these, generated as part of boilerplate code.
.NET User Secrets Manager for secrets. This tool has its own quirks!

To complicate matters further, .NET projects also contain a scaffolded Properties/launchSettings.json. While originally meant for use in Visual Studio (the older product that predates VS Code), it still gets loaded and used when you launch your app through a Debug/Launch configuration in VS Code or on the command line using dotnet run. This too can provide config key/value pairs, sometimes without you realising it.

All these sources of development-time configuration obscure the effective source of a config value (there is a precedence order among the sources).

They are also discordant with how everything else is configured: via a single .env file.

For these reasons, I configure .NET projects in my polyglot workspaces to use .env files for local development, rather than the default sources above.

In this post, I'll show you how.

Note:

The solution given is primarily for VS Code. You may be able to adapt it for other IDEs such as JetBrains Rider or Visual Studio but those are outside the scope of this post.
To use a .env file with a .NET project, an alternative approach is to use dotenv-net. I explain at the end of this post why I don't like using it.

Step 1: Create `.env` files for VS Code launch configurations

Store all configuration, secret and non-secret, for a .NET project in a VS Code launch configuration in an .env file. I keep this next to launch.json in the .vscode folder:

I provide the path of such an .env file in envFile attribute in the launch configuration. For example, consider the compound launch configuration in launch.json given below. Note the envFile attributes:

{
  "version": "0.2.0",
  "compounds": [
    {
      "name": "Frontend/Backend",
      "configurations": [
        "Next.js: debug in full stack",
        ".NET: debug in full stack"
      ],
      "stopAll": true
    }
  ],
  "configurations": [
    {
      "name": ".NET: debug in full stack",
      "type": "coreclr",
      "request": "launch",
      "preLaunchTask": "build-backend",
      "program": "${workspaceFolder}/flowmazonbackend/flowmazonapi/bin/Debug/net9.0/flowmazonapi.dll",
      "cwd": "${workspaceFolder}/flowmazonbackend/flowmazonapi",
      "stopAtEntry": false,
      "envFile": "${workspaceFolder}/.vscode/flowmazonapi.env",
      "sourceFileMap": {
        "/Views": "${workspaceFolder}/Views"
      },
      "requireExactSource": false
    },
    {
      "name": "Next.js: debug in full stack",
      "type": "node",
      "request": "launch",
      "cwd": "${workspaceFolder}/flowmazonfrontend",
      "program": "${workspaceFolder}/flowmazonfrontend/node_modules/next/dist/bin/next",
      "args": ["dev", "--port", "3020"],
      "console": "integratedTerminal",
      "envFile": "${workspaceFolder}/.vscode/flowmazonfrontend.env",
      "serverReadyAction": {
        "pattern": "- Local:.+(https?://.+)",
        "uriFormat": "%s",
        "action": "debugWithChrome",
        "webRoot": "${workspaceFolder}/flowmazonfrontend"
      }
    }
  ]
}

flowmazonapi.env configures the .NET minimal API and looks like this:

Logging__LogLevel__Default=Information
Logging__LogLevel__Microsoft.AspNetCore=Warning
ASPNETCORE_ENVIRONMENT=Development
ASPNETCORE_URLS=http://localhost:3022
ALLOWED_CORS_ORIGINS=http://localhost:3020
ConnectionStrings__FlowmazonDB=<redacted>
OTEL_EXPORTER_OTLP_ENDPOINT=https://otlp-gateway-prod-eu-west-2.grafana.net/otlp
OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
OTEL_RESOURCE_ATTRIBUTES=deployment.environment.name=vscode_launch
OTEL_EXPORTER_OTLP_HEADERS=<redacted>

VS Code takes all the key value pairs in the specified .env file and sets them as environment variables. In ASP.NET Core, these are read by the environment variable config source in the Generic Host. This is the last and therefore the highest priority config source in ASP.NET Core host's default sequence, so it overrides any other source providing the same keys.

So having an .env file to configure a .NET project in a VS Code launch configuration means it replaces, for local development, the canonical combination of appSettings.*.json files and .NET User Secrets Manager. For example, the ConnectionStrings__FlowmazonDB=<redacted> line in the .env file shown above equates to the following in appSettings.json:

{
  "ConnectionStrings": {
    "FlowmazonDB": "<connection string redacted>"
  }
}

and these two lines in the .env file above:

Logging__LogLevel__Default=Information
Logging__LogLevel__Microsoft.AspNetCore=Warning

emulate the default log levels in the boilerplate appSettings.json:

{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft.AspNetCore": "Warning"
    }
  }
}

This means that once you have created a .env file like above, you can delete any appSettings* files in the project's folder, which we will do shortly.

SECURITY NOTE: Please make sure that you have a .gitignore file in the workspace root and that it contains the following line:

*.env

This would stop .env files, which may contain secrets such as connection strings for databases you use for local development, from getting checked in. Note that this, together with any secret scanning you might have in your online repo (such as GitHub Secret Scanning), guards against your local development secrets from leaking into your repo. It is important to be aware that, as with .NET User Secrets Manager, any secrets are still stored locally in plaintext.

All of your local development-time secrets and other configuration - for all projects and not just .NET if you have multiple ecosystems - are now in the .env files located in a single folder: in .vscode.

Moreover, all these colocated .env files are explicitly referenced in a single place: in launch configurations in launch.json in the .vscode folder.

If you have multiple launch configurations that need to be configured differently, you can define separate .env files for each of them.

Step 2: Provide an `.env` file to `dotnet` CLI commands

Even when you have one or more VS Code launch configuration to run a .NET project, you would likely still need to run commands on the project in its on the command line, e.g. to generate or apply EF Core migrations from the DbContext in an API project, or just to build run dotnet build on the .NET project or solution to check that it builds.

My solution for that is to use direnv to load the .env file for the API from .vscode folder, as environment variables specifically when you are in the .NET project's folder on the command line. Therefore when I run any dotnet commands in this folder, those environment variables are available as config key/value pairs for the project.

You can set up direnv for this purpose like this:

Install direnv using your operating system's package manager using the installation instructions given here.
On Windows you can do this by installing WinGet if you don't have it already. Then open a shell with Admininistrator permissions, one way of doing which is to find PowerShell, then right click it to select Run as administrator:

Then run command winget install direnv in this shell.
Hook direnv into your shell, as described here.
To do this for Bash on Windows, I did this by adding line eval "$(direnv hook bash)" at the end of the file ~/.bashrc where ~ is my user profile folder C:\Users\<my windows login name>.
Create a file named .envrc in the .NET project's folder with the following content:
```
dotenv <relative path to folder containing env file>/<env file name>.env
```
Normally you would have statements like export MY_VAR=<value> in a .envrc file to create environment variables. However, in this instance, the dotenv command in the .envrc file above is telling direnv to go and load the contents of the specified .env file as environment variables.
On the shell in your project's folder, run:
```
direnv allow
```
In general, an .envrc file for direnv should not be checked in. This is because it can directly contain environment variables and their values, some of which may be secrets. Even though this is not the case with the .envrc above, as a matter of good practice, I would add the following line to the .gitignore file in workspace root:
```
.envrc
```

Now whenever you cd into your .NET project folder in your shell, direnv would run automatically, run the .envrc file in the folder, and load your .env file as environment variables.

This means that while you are in that folder (or a subfolder within it) in your shell, you can run commands like dotnet run or dotnet ef migrations add or dotnet ef database update which require these configuration settings to run.

Also, when you move out of the folder (e.g. cd ..) or close your terminal instance, the environment variables get unloaded by direnv.

Additional Tips

The *.env.template files shown in the screenshot above are pre-filled versions of the respective .env files with secrets redacted (much like the snippets shown above). I do check these in and, together with setup instructions in the project's wiki, they make it easy to recreate the configuration. The *.env files on the other hand obviously DO NOT get checked in.
If you want to use this technique, please make sure to delete the <UserSecretsId>a-guid-here</UserSecretsId> element that would be present in your csproj if you have previously used .NET User Secrets Manager with the project. While .env would be the highest-precedence source anyway, deleting <UserSecretsId> would avoid surprises down the line such as when you realise that you had forgotten to set certain secrets in .env and they had been coming from User Secret Manager all along.
By the same token, it is worth deleting appSettings.json and any other appSettings.*.json files (such as appSettings.Development.json) in the .NET project folder (BUT make sure you have backed them up somewhere before deleting them).

Why I don't use `dotenv-net`

dotenv-net Nuget package can be used to load .env files into environment variables in code. For example:

using dotenv.net;

DotEnv.Load();

//then rest of Program.cs
var builder = WebApplication.CreateBuilder(args);
//...

DotEnv.Load() loads contents of .env file present in the directory of the running process as environment variables (you can also provide an alternate path as argument).

Next, when WebApplicaton.CreateBuilder(args) is called, the .NET configuration system loads config data from the default (or other configured) sources. This is when environment variables configuration provider loads as configuration the environment variables created by DotEnv.Load() earlier.

You can also wrap DotEnv into a configuration provider as this post shows.

The problem with this approach is essentially the mechanics: in ecosystems that are mainly configured using environment variables, .env files in local development are loaded by the launchers/orchestrators:

When Docker Compose launches apps/services for local testing, it reads key/value pairs from a .env file, then sets it as environment variables for every Docker container it launches. This is how your .NET app's container would receive its configuration.
When VS Code launches processes in a launch configuration - e.g. a .NET minimal API and a Next.js frontend - you can specify a separate .env file for each process. Thus each launch configuration can have its own set of .env files.
When the app is launched in a non-local environment, the host - Azure App Service, Kubernetes, ECS etc. - load configuration data from a configuration store and set it as environment variables before launching the process.

This way the processes launched for debugging or testing locally are completely oblivious to the source of configuration data and who loads/provides it. They don't know and don't care:

which .env files were used
whether .env files or some other configuration store (e.g. in Production) was used to retrieve configuration data
who loaded the config data

They just get the right set of environment variables to read at run time.

This makes processes really easy to configure, hence the popularity of this aspect of the 12-factor approach.

Hardcoding sources of configuration such as specific files - whether appsettings.*.json or .env - adds unnecessary complexity in my view.

It also again adds an exception for .NET behaviour in development - that the code is looking for specific files at startup if app.Environment.IsDevelopment() - when every other type of project in the polyglot workspace is probably only looking for environment variables when it runs.

All of this is not to say that there aren't situations where dotenv-net is the right solution. It's just that in general I prefer not to use it, for reasons given.

Why I moved from AutoFixture to Bogus for test data generation in C#/xUnit tests

nausaf — Sat, 05 Jul 2025 16:03:39 +0000

AutoFixture and Bogus are both well-known libraries for generating test data in C# tests. AutoFixture is dated, whereas Bogus is state-of-the-art.

I moved to Bogus from AutoFixture because:

AutoFixture is dated and no longer under active development. Releases are infrequent (last one was 8 months before the date of this writing). Documentation was updated in 2021 and many of the links mentioned in it contain very old posts.
AutoFixture is too basic. I was quite surprised to discover that despite how long it's been around, there seems to be no out of the box way of generating a number in a specified range.

This makes it particularly difficult to use with Price for example which is bounded by zero below and would typically have an upper limit also.

Bogus is a .NET port of Faker, a JavaScript test data generation library, and it does not have the problems above:

it is actively developed
has brilliant documentation
has a flexible and powerful API which allows you to generate (semi-)meaningful test data within specified constraints really easily. Also, the code you write to do so would be a pleasure to read.

Given a Product class that looks like this:

public class Product
{
    public int Id { get; set; }
    public required string Name { get; set; }
    public required string Description { get; set; }
    public required string ImageUrl { get; set; }
    public required decimal Price { get; set; }

}

this is what a Faker for the class looks like:

internal class ProductFaker : Faker<Product>
{
    public ProductFaker()
    {
        RuleFor(p => p.Id, f => f.Random.Int(1, int.MaxValue));
        RuleFor(p => p.Name, f => f.Commerce.ProductName());
        RuleFor(p => p.Description, f => f.Lorem.Paragraph());
        RuleFor(p => p.ImageUrl, f => f.Image.PicsumUrl());
        RuleFor(p => p.Price, f => f.Finance.Amount(0, 5000));

    }
}

Using ProductFaker, you can generate endless amounts of random, yet semi-meaningful Product objects to use in your tests.

And that Image.PicsumUrl() call generates URLs to actual images on https://picsum.photos that you can navigate to in your Playwright or Selenium tests!

What's really great is that GitHub Copilot generated (almost all of) the ProductFaker for me, as soon as I typed : Faker<Product>. On the other hand when I was wrestling with AutoFixture, it was quiet. This may have something to do with how much Bogus-based (and possibly Faker-based; Bogus is a port of Faker.js) test code there is out there that LLMs have been trained on.

Nick Chapsas has an excellent video, Generating realistic fake data in .NET using Bogus, demonstrating the use of Bogus.

Intercepting HTTP Traffic from the Console with Fiddler

nausaf — Tue, 01 Jul 2025 00:48:13 +0000

Fiddler doesn't work out of the box with HTTP traffic originating from commands run in the terminal (like curl). Luckily, it can be configured to do so.

The configuration given below would allow you to both intercept such traffic and decrypt SSL requests and responses.

Configure Fiddler and your terminal

I tested these steps on Fiddler Classic, which is quite old (but free!) but they should work on the newer, flashier Fiddler Everywhere also:

Click Tools | Options menu to bring up the Options dialog. Go to Connections tab:

Make sure the two checkboxes highlighted in red are checked.

Take note of the port on which Fiddler listens (8888 in my case).
Go to your terminal and run the following commands (I am using export keyword for Bash, you might need to use set for Windows shells):
```
export http_proxy=127.0.0.1:8888
export https_proxy=127.0.0.1:8888
```
This needs to be done every time you open a terminal from which you want HTTP traffic to be intercepted.
Run a command on the terminal that executes an HTTP request e.g.:
```
curl --ssl-no-revoke -H "Accept: text/plain" https://icanhazdadjoke.com/ 
```
With curl you have to use the --ssl-no-revoke option, for reasons explained shortly.
You should now see traffic to the site accessed by curl, but it would be encrypted.
To descrypt SSL traffic, you need can click the yellow button shown in the snapshot above. This would bring up Tools | Options dialog again, this time at the HTTPS tab.

Configure it as follows (you can play around with the settings):

Try out the setup

Try the curl command given above again. You should now be able to see the decrypted SSL traffic of further requests:

Why we need to use `--ssl-no-revoke` with cURL

Fiddler is a man-in-the middle. This is what allows it to sniff HTTP traffic and show it to us. To decrypt SSL/TLS traffic Fiddler, issues and uses its own SSL certificate to interface with the client (e.g. the curl command in the example above).

This works fine with many terminal commands that make HTTP calls, for example terraform apply which makes API calls. However, curl throws an error, which I believe is because it actually checks if the certificate that Fiddler is using to establish the TLS tunnel is legit (which it is not, in the sense that it has not been issued by a well-known certificate authority such as VeriSign).

Using the --ssl-no-revoke command line parameter with curl gets around the error.

An attempt to return meaningful Problem Details responses for model binding errors in an ASP.NET Core Minimal API

nausaf — Tue, 19 Nov 2024 18:04:55 +0000

Introduction

Model Binding is the process of binding parameters of the route handler to values provided in the request (to route segments, query string parameters, cookies, request headers or request body) or to services in the DI container.

The automatically added EndpointMiddleware, which is the last middleware in the request pipeline, performs model binding, then invokes the handler for the requested route with the parameter values extracted during model binding.

If an error occurs during model binding, then in Production environment, EndpointMiddleware returns a 400 or a 500 response with an empty response body whereas in Development environment it throws a BadHttpRequestException.

I like to return error conditions from my minimal API handlers as responses that are compliant with the IETF Problem Details specification. I do this because it is a well-crafted yet really lightweight standard for returning errors from REST APIs. It is also mature (now in its second iteration) and ASP.NET Core has good support for it.

I wanted to do the same with model-binding errors also: to return helpful and detailed ProblemDetails responses that would help the client fix an error that had occurred during model binding, instead of the EndpointMiddleware default of a 400 response with empty body in Production environment.

In order to do this I needed to understand exactly how EndpointMiddleware returns an error that it encountered during model binding.

How EndpointMiddleware returns Model Binding Errors

I have verified the following behaviour:

If a model binding error occurred but NOT due to an issue with contents of the HTTP request, then the EndpointMiddleware would return a 500 status code.

This would happen in both Production and Development environments i.e. regardless of whether app.Environment.IsProduction() or app.Environment.IsDevelopment() is true in Program.cs at runtime.

An example is when a service that was meant to be resolved from the DI container and passed in as an argument to the handler could not be resolved. No exception would be throw, only 500 would be set as the status code of HttpContext.Request

On the other hand, if the error occurred due to an issue with contents of the request, for example:

a required field or value is missing in the request
a request parameter has a value that is invalid/malformed or of an incorrect data type
there is an issue with JSON request body (e.g. it is malformed)

then the response from EndpointMiddleware varies by environment:

In Development environment (and possibly in any non-Production environment), a BadHttpRequestException is thrown by the middleware.
This has:
- StatusCode property set to 400
- a Message property that is informative such as:
  
  Failed to read parameter "CreateProductArgs createProductArgs" from the request body as JSON.
- InnerException property which, if there was problem deserializing the JSON request body, would be System.Text.Json.JsonException.
  
  Again, this has an informative Message property:
```
JSON deserialization for type 
'problemdetailstestapi.CreateProductArgs' was 
missing required properties, 
including the following: price
```
But in Production Environment, the EndpointMiddleware sets status code 400 in the outgoing response with a blank response body.

However we can enable throwing of BadHttpRequestException in Production environment by adding the following line in Program.cs before builder.Build() is called (from this issue in dotnet/aspnetcpore repo) to get our hands on the information that this exception would contain in Development:
```
builder.Services.Configure<RouteHandlerOptions>(
  options => { options.ThrowOnBadRequest = true; }
);
```
Now, even in Production, if a model binding error occurred due to an issue with contents of the request, then a BadHttpRequestException would be thrown by EndpointMiddleware. This is useful as the exception can be quite informative.

Clearly, the messages above in BadHttpRequestException are useful but cannot be sent back to the client verbatim as doing so would reveal internal execution and implementation details.

I also do not want to parse them to extract information as for the same exception, the structure of the error message can be different in different sitautions. Also, exception messages may change in the future.

However, the messages are very useful for logging, and that alone is a good reason to turn on throwing of BadHttpRequestException in Production environment (i.e. to catch and log the exception).

I have verified that an error does not get logged at level Information or above (using .NET logging) from within EndpointMiddleware if there is a model binding exception when binding the request. Now, even if it does log at level Debug or below, I don't want to turn on such verbose logging in Production for any part of the request pipeline on an permanent basis.

So we need to log this exception ourselves. This can be done in one of at least two ways:

via request logging at the reverse proxy (e.g. in a Web App in Azure App Service) to log all requests that resulted in a 400 response being returned. Here, bear mind that even though the EndpointMiddleware would throw a BadHttpRequestException up the the middleware chain, what gets returned to the client - what exits the pipeline as response - is a plain 400 with empty body.
by writing a middleware that will catch and log the BadHttpRequestException exception thrown by EndpointMiddleware once throwing of this exception has been turned on in Production environment.

Returning model binding errors as Problem Details responses

Based on the above behaviour, I thought I could create and return informative Problem Details responses in the event of a model binding error by creating a middleware that would also log these exceptions using .NET logging, as follows:

Enable throwing HttpBadRequestException in Production environment.
Add a middleware just before RoutingMiddleware. This would catch catch a BadHttpRequestException.
If a BadHttpRequestException exception is caught in our middleware which was thrown by the EndpointMiddleware rather than by an endpoint filter or by the invoked router handler then we have one of these two error situations:

A. If InnerException is System.Text.Json.JsonException then the request body is invalid in the specific sense that either a provided value is of an incorrect type, or a required value is missing. Report this with a ProblemDetails response.

B. Otherwise the issue is with some other part of the request. Examples include the following: the JSON body is missing altogether, a required route segment is missing or has an invalid value. Report this with a ProblemDetails response.
If a BadHttpRequestException was NOT caught in the middleware that was thrown by the EndpointMiddleware, i.e. no exception was caught, or an exception was caught but it was not BadHtpRequestException or a BadHttpRequestException was caught but it hadn’t been thrown by the EndpointMiddleware (i.e. had ben thrown by an endpoint filter or from somewhere in the invoked route handler), then we do not have the information to create a meaningful ProblemDetails response.

So we just let it - the response or the exception - propagate up the request pipeline. Eventually it would be caught and, in Production environment, returned as a 400 response with blank body.

Was it worth it?

I did create the middleware to do the above (given at the end of this post) but the results were underwhelming.

It is at points 3A and 3B above that we detect model binding errors and where we could formulate and return meaningful Problem Details.

However at these points it is not possible to reliably parse the BadHttpRequestException thrown by EndpointMiddleware to extract details about where in the request the model binding errors occurred or why.

The issue is that the BadHttpRequestException thrown by EndpointMiddleware on model binding errors is not very machine readable. Therefore I could not formulate and return meaningful Problem Details responses that could pinpoint the error and provide detail to help the client fix it.

Instead, all I can return are two very broad errors, one at 3A and another at 3B (shown in middleware code below). These cover, between them, pretty much everything that could go wrong with a request (headers, route segments, query string, request body, cookies). This is to say that the semantics of a 400 (Bad Request) status code are almost the same as these two errors (returned as Problem Details responses) taken together.

This begs the question: Is there any value in returning two very generic Problem Details responses, over returning a plain 400 with an empty response body?

The Problem Details specification offers an answer. To quote my favourite part of the spec:

truly generic problems -- i.e., conditions that might apply to any resource on the Web -- are usually better expressed as plain status codes. For example, a "write access disallowed" problem is probably unnecessary, since a 403 Forbidden status code in response to a PUT request is self-explanatory.

Conclusion

I don’t see what value distinguishing between these two broad errors would add over just sending back a plain 400 with an empty response body.

And in Production environment, a plain 400 with an empty response body is exactly what gets returned on a model binding error. So there's no need to do anything or change anything.

In particular, I will not implement the solution above.

In order to get more specific errors out of the opaque model-binding process, I would need to basically reverse-engineer (at least partially) EndpointMiddleware's model-binding logic in the custom middleware.

I am not particularly inclined to do this either because I cannot see how the value of this to the consumers of my current API project exceeds the investment in reverse-engineering the model-binding logic, and keeping it in sync with future ASP.NET Core releases.

Isn't the whole point of model-binding in ASP.NET Core that it is free compared to something like Express/Node.js?

However I would always want to have logging of model-binding errors so I could inspect them later. This might lead me to precise model-binding errors that are worth reporting from a middleware as ProblemDetails responses in the future.

To get this logging there are two broad options:

turn on request logging in the reverse proxy to log all requests that led to 400 responses with empty bodies (these would include those that were returned by the EndpointMiddleware on model binding errors.
My Preferred Option: Use a custom middleware to log BadHttpRequestException thrown from EndpointMiddleware.
You can adapt the middleware shown below to do this and place it - by calling the UseXXX method to register it - as follows:
- If you do not have app.UseRouting() or app.UseEndpoints() in your Program.cs then place it at the very end of middleware registrations, before the first app.MapXXX call. This is because in minimal API, since at least .NET 9+, app.Run() automatically add the Routing and Endpoints middlewares at the end if they haven't been registered explicitly.
- If you do have either app.UseRouting or app.UseEndpoints() in your Program.cs then add the middleware just before both of those.

The middleware was as follows.

It doesn't include the check for the stack trace to see if the BadHttpRequestException was thrown by the EndpointMiddleware or not; I give a sketch of how to do this further down.

You can try it out in your own ASP.NET Core minimal API app by calling ProblemDetailsForBadRequestMiddlewareExtensions.UseProblemDetailsForBadRequest in your Program.cs. I did this by adding line app.UseProblemDetailsForBadRequest() after line var app = builder.Build(); in the boilderplate code but before app.UseRouting();. If you do not have app.UseRouting(); of app.UseEndpoints(); (these are not necessary in .NET minimal APIs any more) then simply add it as the last middleware before the first app.MapXXX() method call in your Program.cs.

The process of creating and using a custom middleware is described here in ASP.NET Core documentation.

using System.Text;
using System.Text.Json;

namespace problemdetailsmiddleware.Middleware;

public class ProblemDetailsForBadRequestMiddleware
{
    private readonly RequestDelegate _next;
    private readonly ILogger<ProblemDetailsForBadRequestMiddleware> _logger;

    public ProblemDetailsForBadRequestMiddleware(RequestDelegate next, ILogger<ProblemDetailsForBadRequestMiddleware> logger)
    {
        _next = next;
        _logger = logger;
    }

    public async Task InvokeAsync(HttpContext context)
    {
        try
        {
            await _next(context);
        }
        catch (BadHttpRequestException ex)
        {
            context.Response.StatusCode = 200;
            await context.Response.WriteAsync(ex.StackTrace ?? "");
            return;

            context.Response.StatusCode = ex.StatusCode;
            _logger.LogError(ex, "BadRequestException occurred while processing HTTP request");
            if (ex.InnerException is JsonException)
            {
                //this would only happen if the 
                // var validationProblem = TypedResults.ValidationProblem(new Dictionary<string, string[]> {
                //     {"request body json", new string[] {@"An error occurred when parsing the provided request body. One of three things is likely to be wrong:

                //     1. The provided request body is not well-formed JSON
                //     2. A required key is missing in the request body JSON
                //     3. A value of an incorrect type is provided for a key in the request body JSON"}}
                // });

                var problem = TypedResults.Problem(
                    statusCode: StatusCodes.Status400BadRequest,
                    type: "http://example.com/problems/invalid-request-body-json",
                    title: "\"An error occurred while parsing request body JSON\","
                    detail: "Request body was provided but an error occurred when parsing it. One of three things is likely to be wrong: 1. The provided request body is not well-formed JSON 2. A required property is missing in the request body JSON 3. A value of an incorrect or incompatible type was provided for a property in the request body JSON"
                    );

                await problem.ExecuteAsync(context);
            }
            else
            {
                var problem = TypedResults.Problem(
                    statusCode: StatusCodes.Status400BadRequest,
                    type: "http://example.com/problems/missing-body-or-invalid-request-parameter-values",
                    title: "\"Request body is missing or a request parameter value is missing or invalid\","
                    detail: "Either the request body is required but missing, or the value of a request parameter - in request headers, query string, route segments or cookies - is either missing ( in case of a required parameter) or invalid (e.g. of an incorrect type). Check your request against the OpenAPI description of the operation."
                );
                await problem.ExecuteAsync(context);
            }
        }
    }

}


public static class ProblemDetailsForBadRequestMiddlewareExtensions
{
    public static IApplicationBuilder UseProblemDetailsForBadRequest(this IApplicationBuilder builder)
    {
        return builder.UseMiddleware<ProblemDetailsForBadRequestMiddleware>();
    }
}

Distinguishing between whether the BadHtpRequestException was thrown from EndpointMiddleware of from further down the request processing pipeline:

If every frame in the stack (each is a new line) begins with at Microsoft.AspNetCore.Http.RequestDelegateFactory until potentially a --- End of stack trace from previous location --- line is ensountered, then the exception was thrown from EndpointMiddleware (I believe RequestDelegateFactory create a RequestDelegate out of every middleware in the pipeline that is to be invoked). For example,

   at Microsoft.AspNetCore.Http.RequestDelegateFactory.Log.InvalidJsonRequestBody(HttpContext httpContext, String parameterTypeName, String parameterName, Exception exception, Boolean shouldThrow)
   at Microsoft.AspNetCore.Http.RequestDelegateFactory.<HandleRequestBodyAndCompileRequestDelegateForJson>g__TryReadBodyAsync|102_0(HttpContext httpContext, Type bodyType, String parameterTypeName, String parameterName, Boolean allowEmptyRequestBody, Boolean throwOnBadRequest, JsonTypeInfo jsonTypeInfo)
   at Microsoft.AspNetCore.Http.RequestDelegateFactory.<>c__DisplayClass102_2.<<HandleRequestBodyAndCompileRequestDelegateForJson>b__2>d.MoveNext()
--- End of stack trace from previous location ---
   at problemdetailsmiddleware.Middleware.ProblemDetailsForBadRequestMiddleware.InvokeAsync(HttpContext context) in C:\MyWork\problemdetails\problemdetailsmiddleware\Middleware\ProblemDetailsForBadRequestMiddleware.cs:line 20

If there are other frames, theses would come from an endpoint filter or from somewhere in the invoked route handler. In this case, not all frames, until the line --- End of stack trace from previous location --- is encountered, would start with at Microsoft.AspNetCore.Http.RequestDelegateFactory, as in this example from a Release build of a minimal API (for some reason the Release build also contained a .pdb, hence the topmost frame even tells youthat the error occurred at line 80):

   at Program.<>c.<<Main>$>b__0_3(IList`1 products, LinkGenerator linkGen, CreateProductArgs createProductArgs) in C:\MyWork\problemdetails\problemdetailsmiddleware\Program.cs:line 80
   at lambda_method4(Closure, EndpointFilterInvocationContext)
   at FluentValidation.AspNetCore.Http.FluentValidationEndpointFilter.InvokeAsync(EndpointFilterInvocationContext context, EndpointFilterDelegate next)
   at Microsoft.AspNetCore.Http.RequestDelegateFactory.<ExecuteValueTaskOfObject>g__ExecuteAwaited|129_0(ValueTask`1 valueTask, HttpContext httpContext, JsonTypeInfo`1 jsonTypeInfo)
   at Microsoft.AspNetCore.Http.RequestDelegateFactory.<>c__DisplayClass102_2.<<HandleRequestBodyAndCompileRequestDelegateForJson>b__2>d.MoveNext()
--- End of stack trace from previous location ---
   at problemdetailsmiddleware.Middleware.ProblemDetailsForBadRequestMiddleware.InvokeAsync(HttpContext context) in C:\MyWork\problemdetails\problemdetailsmiddleware\Middleware\ProblemDetailsForBadRequestMiddleware.cs:line 20

Patterns for Routing in ASP.NET Core minimal APIs

nausaf — Sun, 27 Oct 2024 12:59:32 +0000

Introduction to Routing in ASP.NET Core

In ASP.NET Core, Routing is the process of mapping a requested URL to a handler.

First, during startup in Program.cs, you have to register a handler against a route template by calling MapXXX method on app or on a RouteGroupBuilder:

routeBuilder.MapGet("/{id}",
 HandleGetProduct).WithName(HandlerNames.GetProduct)

Note that in my apps, I do not call MapXXX directly on app. Instead I always call these on a RouteGroupBuilder object obtained from app for a certain URL prefix e.g. the routeBuilder used above was obtained from app by saying app.MapGroup("/product"). Therefore the route template that was mapped to the handler function HandleCreateProduct is /product/{id}. It would be obtained by concatenating segment with which the called RouteGroupBuilder was initialised with the segment that passed to MapXXX method when registering the handler.

All registered route templates are stored in a dictionary, each against its registered handler.

Next, when an HTTP request for a URL is received, routing is done by the RoutingMiddleware: it first maps the incoming URL to one of the route templates stored as keys in the dictionary, then it maps that route template to its registered handler stored in the dictionary. It sets this handler in HttpContext so that it is available to every subsequent middleware until the request reaches the EndpointMiddleware.

EndpointMiddleware would invoke the handler function, passing to it any parameters using its model binding logic, sourcing them from request and from the DI container. There is one route parameter in the example route template, {id}. If, given the example route template /product/{id}, the URL requested in an HTP request was /product/12, then id parameter would be assigned value 12 during model binding. This would be passed value of an int id parameter to the handler whose signature could be something like this:

public static async Task<Results<Ok<Product>> HandleGetProduct(int id)
{
    //code to retrieve product from database and return it
    //in JSON body of response
}

In addition to routing, i.e. mapping an incoming URL to a handler, the Routing system in ASP.NET Core allows us to perform the reverse process: it allows generation of URL for a given handler.

In a handler, we can call LinkGenerator.GetPathByName (a instance of LinkGenerator - this class is part of the Routing system - is injected from DI if declared as a parameter in a handler) and provide a handler's name to it:

return TypedResults.Created(
  linkGen.GetPathByName(HandlerNames.GetProduct, new { id = result })
);

The name would have have been declared for a handler by chaining .WithName to the MapXXX call when registering the handle with the Routing system. For example in the snippet shown above for registering the function HandleGetProduct as handler, we chained .WithName(HandlerNames.GetProduct) where HandlerNames.GetProduct is a const:

private static class HandlerNames
{
    public const string GetProduct = "get-product";
    public const string CreateProduct = "create-product";

}

The declared name for a handler should be unique among all registered handlers in the app. The naming convention I use - <operation>-<microservice> ensures that.

Referencing a handler by name rather than by name rather than by passing in a delegate to the handler function makes sense: the handler delegate may be private and may not be available throughout the app where it need to be referenced when a URL to it needs to be generated.

If the route template to which the referenced handler was mapped includes route parameters, you can provide values for these in a dictionary passed as second parameter of LinkGenerator.GetPathByName as shown above.

LinkGenerator.GetPathByName generates a absolute URL to the handler but without a host name. LinkGenerator has other methods such as GetUriByName which prefixes includes the hostname/domain name at the beginning also. I avoid this as this can lead to security vulnerabilities such as to a host name spoofing attack if the HostFilteringMiddleware is not properly configured (it is added to the middleware pipeline by default but in a disabled state).

For further information, see MS Docs page Routing in ASP.NET Core.

Patterns and Guidelines I Use to Implement Routing

These are the patterns and guidelines that I use in my minimal API projects to implement Routing:

1. Mapping Route Templates to Handlers

Create a class which collects together all handlers for a cohesive area of business logic, say for a (synchronous) microservice, and registers these in a static MapRoutes method. This method is called from Program.cs.

The advantage of this pattern is that:

a microservice is responsible for registering all of its operation handlers with the names and metadata that it sees fit to declare. In particular names, route details and metadata for all of the operations in a cohesive group of handlers or a microservice do not pollute Program.cs.
At the same time, Program.cs can choose a route prefix and metadata for the whole group, which can depend on any other groups that Program.cs chooses to register handlers for.

To implement this pattern:

Create Handlers class

In Handlers folder, create a class named <service name>Handlers.
In this class, create each handler as a private static method named Handle<operation>.
Create a nested class private static class HandlerNames to keep string constants for handler names.
Create method public static RouteGroupBuilder MapRoutes(RouteGroupBuilder routeBuilder) registers each handler with its name from HandlerNames by calling routeBuilde.MapXXX method for the appropriate HTTP verb, then returns the passed in RouteGroupBuilder.

Example:

using flowmazonapi.Domain;
using flowmazonapi.BusinessLogic;
using flowmazonapi.BusinessLogic.ProductService;

using Microsoft.AspNetCore.Http.HttpResults;

public class ProductHandlers
{

    private static class HandlerNames
    {
        public const string GetProduct = "get-product";
        public const string CreateProduct = "create-product";

    }

    public static async Task<Results<Ok<Product>, ValidationProblem>> HandleGetProduct(int id)
    {
        //throw new NotImplementedException();
    }

    private static async Task<Results<Created, ValidationProblem>> HandleCreateProduct(CreateProductArgs p, IProductService productService, LinkGenerator linkGen, HttpContext httpContext)
    {
        //throw new NotImplementedException();
    }

    public static RouteGroupBuilder MapRoutes(RouteGroupBuilder routeBuilder)
    {
        routeBuilder.MapPost("/", HandleCreateProduct).WithName(HandlerNames.CreateProduct);

        routeBuilder.MapGet("/{id}", HandleGetProduct).WithName(HandlerNames.GetProduct);

        );


        return routeBuilder;

    }
}

Call `MapRoutes` on a Handlers class from `Program.cs`

In Program.cs, once all middleware have been added (just before app.Run() is called, call MapRoutes on the handlers class, passing in a RouteGroupBuilder taht has been initialised with the prefix that would be prefixed to route templates that will be registered.

ProductHandlers.MapRoutes(app.MapGroup("/product")).WithTags("product Operations");

app.Run();

Here WithTags is an estension method provided by OpenApiRouteHandlerBuilderExtensions and attaches the tag product Operations to the whole group of routes that was registered by the called ProductHandlers.MapRoutes method. Being able to do this using a fluent syntax shows the value of returning the same RouteGroupBuidler that was passed in to the MapRoutes method.

The nullability checks in modern C# make it very difficult for ProductHandlers.MapRoutes not to return the RouteGroupBuilder that was passed in to it. Therefore I am happy with this convention to achieve a fluent interface.

2. Mapping Handlers to URLs

In a handler that needs to return a URL, e.g. to a resource that was created in the handler:

Declare LinkGenerator linkGen parameter in the handler (this would be resolved from DI container when the handler is called by EndpointMiddleware).
In the handler, call linkGen.GetPathByName(HandlersName.<const for handler name>, <any route parameters>) to generate a URL to the handler.
Always use LinkGenerator.GetPathByName, never use LinkGenerator.GetUriByName to avoid security issues such as host name spoofing mentioned in the intro section above.

Example:

private static async Task<Results<Created, ValidationProblem>> HandleCreateProduct(CreateProductArgs p, IProductService productService, LinkGenerator linkGen, HttpContext httpContext)
{

    var result = await productService.CreateProduct(p);

    return TypedResults.Created(linkGen.GetPathByName(HandlerNames.GetProduct, new { id = result }));

}

3. Go easy on route constraints

Route templates should be as simple as possible. While ASP.NET Core provides a fairly rich set of constraints that may be placed on route parameters (such as {id} in code shown above), these should be avoided as much as possible.

In particular, as the documentation advises, route constraints should not be used for validation of route parameter values.

4. Use `**` for catchall parameter rather than `*`

When using catchall parameters, I only use ** rather than single *.

Both have the same behaviour in mapping URL to a route template and to parameter value during model binding, but in the reverse process, when we use LinkGeneratorclass to generate a URL from a handler's name and route parameter values,*output/as a/in the generated URL whereas the singel asterisk () outputs%2F` which I almost never want.

Creating an NPM package that runs on command line

nausaf — Wed, 02 Oct 2024 16:59:10 +0000

Every now and then I need to create an NPM package that runs on the command line, as a CLI (Command Line Interace).

For example, npx eslint runs and lints code in a directory and can accept command line arguments also. Similarly npx create-next-app --eslint --tailwind would scaffold a Next.js app that has eslint and tailwind configured.

There are three things you need to do to turn an NPM package into one that can be run from the command line:

Create a file that contains the code that would run when the package is run using npx from the command line, e.g. cli.js.
In package.json, place a "bin" field:
```
"bin": "./cli.js",
"type": "module",
```
"bin": "./cli.js" tells npx that when the package is run from the command line using npx <package name>, then ./cli.js should be run.

ASIDE: "type": "module" tells NPM utilities that the type of modules which would be imported in the code files in this package would be ES6 (using import statement). Otherwise you would only be able to import Common JS modules (using require) which, this being the tail-end of 2024, you probably don't want to do.

On top of the file declared in "bin" in package.json, place the line #!/usr/bin/env node. This allows the cli.js to run using the Node.js executable when it is launched by npx.

For example, my cli.js in package root would look like this:

#!/usr/bin/env node
import { createRequire } from "module";
const require = createRequire(import.meta.url);
const packageJson = require("./package.json");

console.log("Hello World!");
console.log(`Version number of the package is ${packageJson.version}`);

If I publish the package to NPM by running npm publish on the terminal in the project folder, then go to a completely different folder on the terminal and execute npx show-version-number (where show-version-number is the name of the package in package.json and therefore in NPM registry), it would still run:

I checked that npx downloaded and stored the package in C:\Users\{My User Name}\AppData\Local\npm-cache\_npx\ on my Windows machine.

Code is in this GitHub repo. If you want to publish it to NPM, please change "name" in package.json to a different value as I have already published a package with the name "show-version-number". I use this tool to check if a package name is available in NPM registry.

You can also create named commands by creating named values in "bin" in package.json like this:

"bin": {
  "showver": "./cli.js",
}

Now when you publish the package to NPM, and run it as npx show-version-number on the command line, cli.js would still run as before. I believe npx <package name> picks up the first entry in "bin" (in this case the key "showver") and runs the code file defined there (in this case ./cli.js).

However, you can also install the package on your machine:

npm install --global show-version-number

The npm install --global command would register a command named showversion with the operating system (as a cmd file on Windows and as a symlink on Unix-based system such as Linux, as described here) that aps to cli.js file.

Now, you can say the following on the terminal in any folder on your machine:

showver

This would run cli.js with Node executable and you would have the same output as when you ran npx show-version-number.

Of course, a single package may provide multiple named commands.

ASIDE: I had to import package.json using the following lines in cli.js:

import { createRequire } from "module";
const require = createRequire(import.meta.url);
const packageJson = require("./package.json");

instead of import packageJson from "package.json" because in my version of Node (v20+), this latter import throws a ERR_IMPORT_ASSERTION_TYPE_MISSING error when I try to run the package using npx ..

To fix the error I had to either rewrite the import as:

import packageJson from "./package.json" with { type: "json" };

or using the assert keyword as:

import packageJson from "./package.json" assert { type: "json" };

In either case, I got the following warning when I ran the code:

However, the three lines I use to import package.json instead, clunky as they are, get rid of the warning. See this StackOverflow thread for more details.

Environments in GitHub (with example of deploying a Next.js app to Vercel)

nausaf — Mon, 30 Sep 2024 00:09:32 +0000

An Environment in GitHub is basically a set of three things:

Secrets
Variables
Protection rules

When you define an environment, you provide a name and can configure any of the above.

For example, when defining an environment named UAT (in Environments tab in repo Settings), the environment definition page would look like this:

What is interesting is that you can reference an environment in a job in a GitHub Actions workflow using an environment block like this:

jobs:
  deploy-to-vercel-pr-preview-env:
    runs-on: ubuntu-24.04
    environment:
      name: Preview
      url: ${{ steps.deploy-artifacts.outputs.previewUrl }}

In addition to the fact that such a job is allowed to access Variables and Secrets defined within the environment (as opposed to at the repo level) and Protection Rules such as delayed execution, manual approval, and deployment only from branches meeting specified criteria (e.g. with matching names and/or with branch protection rules) apply, when a job references an environment, this enables a number of other interesting behaviours:

There is a really nice sticky comment on the PR (so it updates with every push to the source branch if that triggers the CI workflow) which shows the value of the url property for the environment once the job has completed:
You can use check Require deployments to succeed before merging in a branch protection rule on the base branch (say main). This would ensure that any head branch (source branch) in a PR has successfully deployed to the the specified environment.

Deployment of the source branch to an environment is indicated by there being a job in a workflow that runs on the source branch which references the specified and provides a deployment URL. An example can be seen in the snippet above in which job deploy-to-vercel-pr-preview-env has an envionment object that specifies name: Preview and also sets the value of the url property which would be the URL to which the job deployed.

I am not sure of the value of this check in simple branching workflows as you would already have a check in branch protections rules on main (or other base branch) to say that a certain job ran successfully. One such job would be the one that deploys to a UAT or other pre-release environment successfully (e.g. deploy-to-vercel-pr-preview-env in the snippet above).
The url property of environment can be static but it can also reference a step output parameter of the same job. In the latter case, the url property of the environment block in the job will be evaluated after the step which outputs that parameter value has executed.

In the job snippet shown above, the step that produces the referenced step output parameter is:
```
id: deploy-artifacts
    run: |
      previewUrl=$(vercel deploy --prebuilt --token=${{ secrets.VERCEL_TOKEN }})
      echo "previewUrl=$previewUrl" >> "$GITHUB_OUTPUT"
```
Once the job has completed (or perhaps when the step above has completed), value of step output parameter previewUrl would be available. This would be assigned to url proeprty of the environment. This is what gets displayed on the sticky comment for the environment as the deployment URL
If a job referencing an environment runs on multiple source branches (in multiple PRs), they don't seem to wait for each other i.e. they seem to run in parallel (I might be wrong on this).

In this case each would post a different URL in its sticky comment as long as the underlying deployment logic generated a different URL on every deployment. For example, every deployment of Next.js project to Preview environment in your Vercel project would generate a new URL.

In fact, since Vercel has such a generous allowance of deployment to an environment that would keep active, all deployments of the same branch, within the same pull request are active and accessible. If your deployment job references a GitHub environment (e.g. "Preview" in above example) then all of these deployment URLs are accessible on the pull request, not just the latest one:

All deployments to an environment, with their respective URLs if provided, can also be seen on repo main page in a section on right hand side named Deployments:

You can click an environment name and see a list of all deployments to it, including link to each. The link to the most recent deployment is shown right at the top:

To use GitHub Environments in my Next.js project which I deploy to Vercel, I do the following:

Create two GitHub Environments, named Production and Preview.

A Vercel project to which the repo is deployed also has evironments with the same names (these are Vercel environments though, not GitHub environments). So its good to have matching names in both GitHub repo and Vercel project.

My CI workflow deploys a pull request's source branch to Vercel's Preview environment and references GitHub Environment named "Preview":

name: CI
concurrency:
  group: ci-${{ github.ref }}
  cancel-in-progress: true
permissions:
  checks: write
  pull-requests: write
on:
  pull_request:
    branches:
      - main
env:
  VERCEL_ORG_ID: ${{ secrets.VERCEL_ORG_ID }}
  VERCEL_PROJECT_ID: ${{ secrets.VERCEL_PROJECT_ID }}

jobs:
  deploy-to-vercel-preview-env:
    name: Deploy to Vercel Preview environment
    runs-on: ubuntu-24.04
    environment:
      name: Preview
      url: ${{ steps.deploy-artifacts.outputs.previewUrl }}
    steps:
      - uses: actions/checkout@v2
      - name: Install Vercel CLI
        run: npm install --global vercel@latest
      - name: Pull Vercel Environment Information
        run: vercel pull --yes --environment=preview --token=${{ secrets.VERCEL_TOKEN }}
      - name: Build Project Artifacts
        run: vercel build --token=${{ secrets.VERCEL_TOKEN }}
      - name: Deploy Project Artifacts to Vercel
        id: deploy-artifacts
        run: |
          previewUrl=$(vercel deploy --prebuilt --token=${{ secrets.VERCEL_TOKEN }})
          echo "previewUrl=$previewUrl" >> "$GITHUB_OUTPUT"

Release/CD workflow that deploys main to Vercel Production environment references the GitHub Environment named "Production":

name: Release to Production
concurrency: release-to-prod-pipeline
on:
  push:
    branches:
      - main

jobs:
  deploy:
    name: Deploy to Vercel
    runs-on: ubuntu-24.04
    environment:
      name: Production
      url: ${{ steps.deploy-artifacts.outputs.previewUrl }}
    env:
      VERCEL_ORG_ID: ${{ secrets.VERCEL_ORG_ID }}
      VERCEL_PROJECT_ID: ${{ secrets.VERCEL_PROJECT_ID }}
    steps:
      - uses: actions/checkout@v2
      - name: Install Vercel CLI
        run: npm install --global vercel@latest
      - name: Pull Vercel Environment Information
        run: vercel pull --yes --environment=production --token=${{ secrets.VERCEL_TOKEN }}
      - name: Build Project Artifacts
        run: vercel build --prod --token=${{ secrets.VERCEL_TOKEN }}
      - name: Deploy Project Artifacts to Vercel
        id: deploy-artifacts
        run: |
          previewUrl=$(vercel deploy --prebuilt --prod --token=${{ secrets.VERCEL_TOKEN }})
          echo "previewUrl=$previewUrl" >> "$GITHUB_OUTPUT"

Set up linting and formatting for code and (S)CSS in Next.js with ESLint, Stylelint, Prettier and lint-staged

nausaf — Wed, 26 Apr 2023 21:20:21 +0000

UPDATE: Please find an up to date version of this article on FreeCodeCamp.

Replacement for retired VS Code Jest codelens

nausaf — Fri, 30 Dec 2022 23:32:54 +0000

The Codelens provided by VS Code Jest extension on tests in the editor has been retired (hence this post on Jest CodeLens I wrote a while back is also outdated).

The fix is really simple:

In your User Settings file (setttings.json), put this line in:

"testing.defaultGutterClickAction": "contextMenu"

You should now be able to access Debug test etc. from the Play button in the gutter:

VS Code Jest extension: tackling Debug codelens disappearing act

nausaf — Tue, 18 Oct 2022 16:24:03 +0000

This is the section of my VS Code User Preferences settings.json (to access, Ctrl +P then select Preferences: Open User Settings) for Jest VS Code extension:

{
    ...
    "jest.coverageFormatter": "GutterFormatter",  
    "jest.debugCodeLens.showWhenTestStateIn": [
        "fail",
        "unknown",
        "pass",
        "skip"
    ]
}

I found that the Jest extension's Debug CodeLens wouldn't always appear on tests. A lot of the time I had to go to Testing window in VS Code sidebar to start debugging on a test because the Debug codelens wasn't available on the test in situ in the code file.

Turns out this was because jest.debugCodeLens.showWhenTestStateIn has default value of ["fail", "unknown"].

I added the other two possible statuses to ensure that the Debug codelens is always available.

The other setting here is jest.coverageFormatter. I find seeing coverage in the gutter on the left of a file unobtrusive compared to the extension's default behaviour of colour-coding regions of code in the file.

You can of course set the above on a per project basis by putting the snippet above in .vscode\settings.json in your workspace (this file can be created/navigated to using the command Ctrl + P then Preferences: Open Workspace Settings).

Configure different Jest timeouts for unit and integration tests in the same project

nausaf — Mon, 17 Oct 2022 21:54:21 +0000

The Problem

You want to configure different test execution timeouts for Jest tests in different folders in the same project e.g. 1 second for tests in ./tests/unitTests/ and 60 seconds for tests in ./tests/integrationTests.
During debugging, the timeout for tests in any folder should be quite large, say 10 minutes.

Otherwise Jest could throw timeout errors like thrown: "Exceeded timeout of 5000 ms for a test. even if a test completes successfully. This happens when debugging a test takes longer than the (default or explicitly configured) timeout for it.

Solution

Given the folder structure:

.
│   jest.config.js
│   package-lock.json
│   package.json
│
└───tests
    ├───integrationTests
    │       integrationTestSuite.test.js
    │
    └───unitTests
            unitTestSuite.test.js

a solution is as follows:

npm install --save-dev debugger-is-attached
Create a jestSetup.js file in each of the test folders and set the desired timeout via a call to jest.setTimeout() method.
So we have a tests/unitTests/jestSetup.js file:
```
jest.setTimeout(1000); //timeout of 1 second
```
and a tests/integrationTests/jestSetup.js file:
```
jest.setTimeout(60000); //timeout of 1 minute
```

Create jest.config.js in project root as follows

const { debuggerIsAttached } = require("debugger-is-attached");
const path = require("path");

module.exports = async () => {
  const isDebuggerAttached = await debuggerIsAttached();

  const unitTestFolder = "<rootDir>/tests/unitTests";
  const integrationTestFolder = "<rootDir>/tests/integrationTests";
  const getSetupFiles = (folder) =>
    isDebuggerAttached ? [] : [path.join(folder, "jestSetup.js")];

  const baseProjectConfig = {
    //Here put any properties that are the same for
    //all folders and can be specified at level
    //of the project object (all such properties
    //are declared in type ProjectConfig in
    //Config.ts in Jest repo)
  };

  let config = {
    //any config key/values in configuration (except those
    //that are to specified in ProjectConfig)
    //e.g. 
    //collectCoverage: true,

    //project config
    projects: [
      {
        ...baseProjectConfig,
        displayName: "UnitTests",
        testMatch: [path.join(unitTestFolder, "**/*.test.js")],
        slowTestThreshold: 1000, //1 second
        setupFilesAfterEnv: getSetupFiles(unitTestFolder),
      },
      {
        ...baseProjectConfig,
        displayName: "IntegrationTests",
        testMatch: [path.join(integrationTestFolder, "**/*.test.js")],
        slowTestThreshold: 60000, //1 minute
        setupFilesAfterEnv: getSetupFiles(integrationTestFolder),
      },
    ],
    //any other Jest config goes here
    //(these are any properties declared in type
    //InitialConfig but not in type ProjectConfig
    //in Config.ts in Jest repo)
  };

  if (isDebuggerAttached) config["testTimeout"] = 600000; //ten minutes

  return config;
};

In User Settings (settings.json which appears when from the Ctrl + P command pallette you select Preferences: Open User Setting (JSON)), set "jest.monitorLongRun" to a value that is equal to or greater than the largest of the folder-specific timeouts declared in jest.config.js above. For example,
```
"jest.monitorLongRun": 60000, //1 minute
```

Explanation

Configuring different timeouts for different test folders

testTimeout property can be set in jest.config.js in project root to set a timeout other than the default of 5s:

module.exports = {
    ...
    testTimeout: 60000; //60 seconds
};

How to specify testTimeout separately for different subfolders?

The canonical way to assign different configurations to tests in different subfolders is to use Jest's monorepo configuration. We pretend, as far as Jest is concerned, that folders tests/integrationTests and tests/unitTests are two separate projects in a monorepo (a collection of related projects stored in a single repository).

Using this approach we can configure separate timeouts for our two subfolders as follows:

Create a jest.config.js in each subfolder. In this set testTimeout property as shown in the snippet above.

Declare the different folder-specific config files as projects in the top-level jest.config.js:

module.exports = {
  projects: [
    "<rootDir>/tests/unitTests/jest.config.js",
    "<rootDir>/tests/integrationTests/jest.config.js",
  ],
};

At this point the folder structure would be as follows:

.
│   jest.config.js
│   package-lock.json
│   package.json
│
└───tests
    ├───integrationTests
    │       integrationTestSuite.test.js
    │       jest.config.js
    │
    └───unitTests
            jest.config.js
            unitTestSuite.test.js

The problem is that if we configure testTimeout property in a folder-specific config file it will not have any effect.

This is because testTimeout is not defined in type ProjectConfig in Config.ts which specifies the config schema of project (which for use are the two subfolders of tests).

Even though the top-level as well as folder-specific config files are all called jest.config.js, the properties that are allowed in folder level config are not all the same as the properties allowed in top level config.

Folder-specific config files need to have be those defined in type ProjectConfig whereas top-level jest.config.js specifies properties that are probably declared in type InitialConfig in Config.ts.

Many keys are defined in both types but not testTimeout: it is only contained in InitialConfig and therefore only has effect if declared in the top-level config file.

Therefore testTimeout property cannot be use to override test timeouts in jest.config.js files in subfolders.

Instead, to set timeout at subfolder level:

We can call jest.setTimeout(TIMEOUT_IN_MS) in a .js file in the subfolder, conventionally named jestSetup.js.
Declare jestSetup.js in the folder-level jest.config.js so that it would be run by Jest before any tests in that folder are executed.

For example create tests/integrationTests/jestSetup.js as follows:

jest.setTimeout(60000); //timeout of 1 minute

and a tests/integrationTests/jest.config.js to go with it:

module.exports = {
  displayName: "IntegrationTests",
  setupFilesAfterEnv: [`<rootDir>/jestSetup.js`],
};

For unit tests, ./tests/unitTests/jest.config.js would be the same as the config file for integration tests folder shown above (because <rootDir> always resolves to the containing folder). However ./tests/integrationTests/jestSetup.js would specify a different timeout:

jest.setTimeout(1000); //timeout of 1 second

The folder structure of this working solution looks like this:

.
│   jest.config.js
│   package-lock.json
│   package.json
│
└───tests
    ├───integrationTests
    │       integrationTestSuite.test.js
    │       jest.config.js
    |       jestSetup.js
    │
    └───unitTests                 
            jest.config.js
            jestSetup.js
            unitTestSuite.test.js

We can eliminate folder-specific jest.config.js files by pulling the info in the top level config so the ./jest.config.js in the root now looks like this:

module.exports = {
  projects: [
    {
      displayName: "UnitTests",      
      testMatch: ["<rootDir>/tests/unitTests/**/*.test.js"],
      setupFilesAfterEnv: ["<rootDir>/tests/unitTests/jestSetup.js"],
    },
    {
      displayName: "IntegrationTests",
      testMatch: ["<rootDir>/tests/integrationTests/**/*.test.js"],
      setupFilesAfterEnv: ["<rootDir>/tests/integrationTests/jestSetup.js"],
    },
  ],
};

The folder structure of this more compact solution looks like this:

.
│   jest.config.js
│   package-lock.json
│   package.json
│
└───tests
    ├───integrationTests
    │       integrationTestSuite.test.js
    │       jestSetup.js
    │
    └───unitTests
            jestSetup.js
            unitTestSuite.test.js

I'd like to point out three improvements that can be made:

It is worth adding a slowTestThreshold property in every project object and set it equal to or somewhat greater than the timeout declared in the corresponding jestSetup.js.

A longer timeout prevents Jest from throwing an error on longer running tests. But it would still show the execution time of such a tests with a red background:

Adding slowTestThreshold: 60000 to the folder's config in jest.config.js tells Jest that this is how long you expect tests in that folder to take so it doesn't show execution times in warning colour.
If a property is exported both ProjectConfig and InitialConfig types then, if you configure it in top level jest.config.js but not in folder-level config, it would be overridden by project config to its default value which would probably be null or undefined.

For example given a jest.config.js that looks like this:
```
module.exports = {
    setupFiles = ['./topLevelSetupFile.js'],
    ...

    projects: [
        {
            displayName: "IntegrationTests",
            testMatch: ["<rootDir>/tests/integrationTests/**/*.test.js"],
            setupFilesAfterEnv: ["<rootDir>/tests/integrationTests/jestSetup.js"],
        }
    ];
};
```
In the effective folder-specific configuration for folder ./tests/integrationTests/, setupFiles property would be set to nothing (null I think).

A nice solution (from Orlando Bayo's post) is to set the shared config - properties that are shared across all folders - in a baseProjectconfig object and spread this into every project object.

Incorporating both of these improvements into our solution, we have the following jest.config.js:

const baseProjectConfig = {
    //Here put any properties that are the same for
    //all folders and can be specified at level
    //of the project object (all such properties
    //are declared in type ProjectConfig in
    //Config.ts in Jest repo)
  };

  let config = {
    //any config key/values in configuration (except those
    //that are specified in ProjectConfig)
    //e.g. 
    //collectCoverage: true,

    //project config
    projects: [
      {
        ...baseProjectConfig,
        displayName: "UnitTests",
        testMatch: [path.join(unitTestFolder, "**/*.test.js")],
        slowTestThreshold: 1000, //1 second
        setupFilesAfterEnv: getSetupFiles(unitTestFolder),
      },
      {
        ...baseProjectConfig,
        displayName: "IntegrationTests",
        testMatch: [path.join(integrationTestFolder, "**/*.test.js")],
        slowTestThreshold: 60000, //1 minute
        setupFilesAfterEnv: getSetupFiles(integrationTestFolder),
      },
    ],        
    //any other Jest config goes here
    //(these are any properties declared in type
    //InitialConfig but not in type ProjectConfig
    //in Config.ts in Jest repo)
  };

module.exports = config

Jest VS Code extension still shows a popup if a test takes too long to execute (although, because of the configuration above, the test won't fail on a timeout):

To prevent this warning window from popping up, set "jest.monitorLongRun" in User Setting file (to edit it select Preferences: Open User Settings from Ctrl + P command pallette) to the value of the longest of your folder-specific timeouts e.g.:
```
"jest.monitorLongRun": 60000, //1 minute
```

Setting a long timeout when debugging

When debugging, if you take longer to step through a test than the (default or explicitly configured) timeout, Jest would throw a timeout error at the end even if the test completed successfully. I find that for a test to fail like this is confusing when you're debugging a test that you expect to pass.

To address this issue we can use the debugger-is-attached package. This exports an async function that returns true if debugger is attached and false otherwise.

It can be integrated into jest.config.js by exporting an async function that returns the config object instead of directly returning the object in question.

const { debuggerIsAttached } = require("debugger-is-attached");

module.exports = async () => {
    //do async things
    ...
    return {
        //the config object
    }
}

Inside the function, if the debugger is not attached then everything should be the same as before but if it is attached, then we make two changes to the returned object:

not configure jestSetup.js, the file that declares the timeout for tests in its containing folder, to run.
set testTimeout property to 10 minutes (600000 ms) to give us plenty of time to debug a test.

After these changes, the final top-level jest.config.js is as shown in TL;DR at the top.

Thanks for reading. Any comments or suggestions for improvement would be greatly appreciated.