DEV Community: NotoriousWRLD

How a dog could clean your orphan infrastructures after a test integration process crash

NotoriousWRLD — Wed, 22 Apr 2026 15:16:28 +0000

ℹ English is not my native language and i'm not using any AI to write this article. Links to code will be added so you can follow the whole process through the articles, do not hesitate to click links

Never have you ever debugged an integration test, found that it has failed, and unfortunately pressed the red squared button to kill the debugging, skipping the whole teardown logic?

Tell me you've never seen an integration test CI starting your test and for some reason, be stopped unexpectedly ? (Timeout, OOM, Job canceled).

That's why TestContainers has made Ryuk, a docker container alongside others that clean them after your tests.
It works absolutely wonderfully because it does not rely on the test process to clean your environment.

But i had one problem.
Docker was not available for the infrastructure i wanted to set up.
And my dev server was not always cleaned by the test.

A lot of orphan test message bus subscriptions were left alone.
Making the dev server a complete mess of useless queues with large random guid names. Databases suffered from the same problem.

That's why i have been working on a watchdog that handles not only docker containers, but any infrastructure that is set up in your tests.
And it is integrated in NotoriousTest, an integration test framework made to simplify the making of clean, reusable integration tests.

DoggyDog has been made to work alongside NotoriousTest, all behaviour described here will be supported natively by NT without you doing anything.

But since we are here to talk about the watchdog, lets see how it works :

Doggydog, the nice and gentle dog that keeps tracks of your infrastructures.

DoggyDog is a self-contained executable, made with .NET 10.
It has the responsibility to clean all your orphan infrastructures after the test process quits without success.

A well trained dog that waits for your test to crash

🐶 DoggyDog is well trained, he will sit gently before doing anything to your infrastructures. Don't worry, he will move only after the right signal is emitted.

You will need to provide a PID to the Watchdog.
He will wait until the process ends.

After the process ended, he will look for a signal, more specifically, a temporary signal file.
This file name is formatted with an "EnvironmentId" (nt-{EnvironmentId}.signal).
If he finds this file, it means that the process has exited correctly and had the time to clean the environment.

ℹ Why does DoggyDog not use exit code to ensure the test process exits correctly or not ?
On Windows, the exit code is available for everyone as a public information.
On Linux, however, this information is available only for the parent process. Which is not the case here since DoggyDog is finding the process with the PID.
For compatibility reasons, i needed to find an alternative.

🐶 Then, the DoggyDog will leave and return to his Doghouse until you need him for another guard duty.

But what if he doesn't find his signal file ?
If the file is not present, DoggyDog will start the clean process.

How DoggyDog takes care of the mess after dinner

Loading the test assembly as a plugin

While tests were running, DoggyDog loaded the test assembly inside his default load context.

To do so, he takes a path to the test assembly and its runtimes to load them inside his default load context.

You need to think of the test assembly as a plugin of DoggyDog.
You define classes with predefined interfaces and attributes, and DoggyDog will find them later to do his job.
DoggyDog takes runtimes as well to resolve ASP.NET Core assemblies and/or .NET assemblies that could be referenced inside the test projects. (e.g. Microsoft.* or System.* packages).

ℹ Why not using an Assembly Load Context ?
DoggyDog must be seen as a process "inside" the test process.
Later, we will see that DoggyDog uses a sqlite registry to retrieve infrastructure types, and loads them from the test assembly.
He will need to make comparisons between referenced types in DoggyDog and types in the test project.
Comparison between 2 instances of the same version but with different calling assemblies will not result in equality. I didn't want to have to design DoggyDog around full reflection.

Runtimes paths are needed for two reasons :

Because DoggyDog is self contained. You don't need to have .NET 10 installed on your machine to run DoggyDog (NotoriousTest is trying to be as compatible as possible with .netstandard2.1, and when it's not possible, .NET 8). Therefore, he will not resolve framework types from the .net assembly.
Because DoggyDog is a .NET executable, and does not reference ASP.NET Core. And i didn't want to make a direct reference to ASP.NET to handle assembly resolution.

Retrieving orphan infrastructures and their cleaners

The test process will populate a sqlite registry when it initializes an infrastructure with InfrastructureRegistryEntry:

EnvironmentId : An ID that identifies a subset of infrastructures associated with a test campaign.
InfrastructureId
InfrastructureType
Metadata and MetadataType : Necessary information to clean the infrastructure (such as connection strings, container name/id, subscription name, etc...)

For every infrastructure that has not been deleted from the test process, DoggyDog will :

Load the infrastructure type.
Retrieve the CleanerAttribute associated with the infrastructure class.
Retrieve the IInfrastructureCleaner in the CleanerType property of the attribute.
Instantiate the Cleaner as an IInfrastructureCleaner.
Call the method CleanAfterCrash with the metadata object loaded from the registry, serialized as the MetadataType.
Remove the infrastructure from the registry.

By adding on your infrastructure classes the cleaner attribute, and implementing an IInfrastructureCleaner, you can clean any infrastructure that you want, even if it's a docker container, a real database, a file.

Infrastructure events

DoggyDog is always watching the registry.
When the test project creates an infrastructure in the registry, adds a reset date, or deletes an infrastructure, DoggyDog will see it and log into the console what happened.

The End

In conclusion, DoggyDog is a watchdog that waits for your test process to stop, looks in a sqlite registry for all infrastructures that have been created, and calls their associated cleaner, while notifying what happened to them.

It was really fun to get through all this reflection stuff.
I had never had the chance to dive into it and making everything work makes me genuinely proud.

Do not hesitate to share your thoughts about it, go through the repository , the docs and samples.
And to star the repo if you like it !

Have a great day !

🐶 Woof !

How to write reusable and maintainable integration tests in .NET 6

NotoriousWRLD — Tue, 11 Feb 2025 12:00:00 +0000

Introduction

Integration tests play a crucial role in maintaining the reliability and stability of applications. However, their implementation presents major challenges.

Situated just below end-to-end tests in the testing pyramid, they provide essential validation before deploying new features. Nevertheless, even minor communication issues between components can compromise their effectiveness.

The setup and maintenance of integration tests can be complex and overwhelming for teams. Managing the provisioning, resetting, and teardown of infrastructures—such as databases, FTP servers, and message brokers—requires meticulous orchestration.

Best Practices for Maintainable and Reusable Integration Tests

1. Isolation

Each test must operate independently to prevent unintended interference.

For example:

If a test verifies the deletion of data while another test, running beforehand, checks for data retrieval, the latter may fail—not due to faulty design but because it relies on a state modified by the earlier test.

Isolation can be enforced through various means:

Architecting integration tests to allocate distinct databases per test.
Implementing strategies such as dedicated schemas for each execution (e.g., multi-tenancy).

2. Coverage

Comprehensive integration tests should span all system layers, from entry points to critical infrastructures such as databases, FTP servers, and message brokers.

To ensure realism, excessive mocking should be avoided. An effective integration test must:

✅ Persist data in an actual database
✅ Transmit real messages via a message bus
✅ Utilize FTP or blob storage for file management
✅ etc...

Although some external service calls may require mocking, all application-specific infrastructures should remain real.

3. Automated and frictionless setup

Integration tests should be easy to set up and require minimal manual intervention from developers.

The installation of specific servers, certificates, or a dedicated testing environment should be fully automated within the test code itself, rather than requiring prior manual configuration on a developer’s machine.

This ensures better portability, reduces human errors, and enables seamless test execution across different machines and CI/CD pipelines.

4. Parallel Execution

Integration tests should be designed for parallel execution, reinforcing their isolation:

Each test should run independently without interfering with others.
Concurrent access to shared infrastructures (e.g., databases, queues) must be properly managed.

Failure to support parallel execution can drastically slow down CI/CD pipelines.

5. Production-like Environment

Integration tests should closely mirror production conditions.

This includes:

Utilizing the same versions of servers, databases, and applications as those in production.
Replicating network configurations and production architectures as faithfully as possible.

By aligning integration tests with production environments, unexpected failures during deployment can be minimized.

Available Solutions

Several NuGet packages and tools simplify the implementation of integration tests:

TestContainers – Enables on-the-fly creation of Docker containers.
Respawn – Facilitates database resets.
WebApplicationFactory : Supports streamlined API testing.
DistributedApplicationFactory – A newly introduced feature in .NET Aspire.

However, none of these solutions fully orchestrate the setup and interaction between multiple components. This is where NotoriousTests excels.

Each of the previously mentioned packages integrates seamlessly within this solution, allowing you to focus on what truly matters: your tests.

Creating a SUT (System Under Test) with NotoriousTests

Prerequisites

NotoriousTests is fully built on xUnit. Therefore, you need to create a test project compatible with this framework before proceeding.

It comes in the form of a NuGet package, which can be installed via Visual Studio, NuGet CLI, or your preferred dependency manager (such as Paket or others).

From package manager console :

PM> Install-Package NotoriousTest

Or from the .NET CLI as:

dotnet add package NotoriousTest

Key Concepts

Let's first understand the two key concepts of this framework:

Infrastructure. An infrastructure component is a hardware or software dependency necessary for the application to function, such as an SQL database, message bus, blob storage, HTTP server, or API.
Environment. An environment is a collection of infrastructures that manages their lifecycle, including creation, reset, and teardown.

Creating an Infrastructure for an SQL Server with TestContainers and Respawn

❗ Before diving into the example, it's important to mention that NotoriousTest 2.3.0 introduces a dedicated package (NotoriousTest.SqlServer) for automatically managing SQL Server databases in tests. This package makes it much easier to create, reset, and destroy a database using TestContainers and Respawn. However, for educational purposes and to better understand what happens under the hood, I will still demonstrate how to do it manually. More info on the ReadMe - Advanced functionnalities - SqlServer

Let's start by preparing our infrastructure.
This infrastructure must generate a connection string as output, start/stop, and clear the database.

NotoriousTest automatically manages configuration setup.
To integrate your infrastructure, simply inherit from:

AsyncConfiguredInfrastructure: Mark this class as an infrastructure component that utilizes configuration.

    public class DatabaseInfrastructure : AsyncConfiguredInfrastructure
    {
        public override Task Destroy()
        {
            throw new NotImplementedException();
        }

        public override Task Initialize()
        {
            throw new NotImplementedException();
        }

        public override Task Reset()
        {
            throw new NotImplementedException();
        }
    }

1️⃣ Creating the SQL Server with TestContainers

Using TestContainers, we can easily start and stop a Docker container running an SQL Server instance.

public class DatabaseInfrastructure : AsyncConfiguredInfrastructure
{
    private MsSqlContainer _container = new MsSqlBuilder().Build();

    public override async Task Destroy()
    {
        await _container.StopAsync();
    }

    public override async Task Initialize()
    {
        await _container.StartAsync();
    }

    public override Task Reset()
    {
        throw new NotImplementedException();
    }
}

2️⃣ Creating the Database and Adding Tables

We now need to add the required tables to the server.
This depends on your database management approach. In this example, we use a SQL file, but you can also use an EF Core DbContext.

public class DatabaseInfrastructure : AsyncConfiguredInfrastructure
{
    private MsSqlContainer _container = new MsSqlBuilder().Build();
    // In an environment context, ContextId is a GUID identical to the EnvironmentId value.
    private string DbName => $"TestDB_{ContextId}";

    public override async Task Destroy()
    {
        await _container.StopAsync();
    }

    public override async Task Initialize()
    {
        await _container.StartAsync();
        using (var connection = GetConnection())
        {
            await connection.OpenAsync();
            await CreateDatabase(connection);

            await connection.ChangeDatabaseAsync(DbName);
            await CreateTables(connection);
        }
    }

    public override Task Reset()
    {
        throw new NotImplementedException();
    }

    private async Task CreateDatabase(SqlConnection sqlConnection)
    {
        using (SqlCommand command = sqlConnection.CreateCommand())
        {
            command.CommandText = $"CREATE DATABASE [{DbName}]";
            await command.ExecuteNonQueryAsync();
        }
    }

    private async Task CreateTables(SqlConnection connection)
    {
        // Here you could use a migration tool like DBUp or EF Core to create the tables
        var sql = File.ReadAllText("Tables.sql");

        using (SqlCommand command = connection.CreateCommand())
        {
            command.CommandText = sql;
            await command.ExecuteNonQueryAsync();
        }
    }

    private SqlConnection GetConnection() => new SqlConnection(_container.GetConnectionString());

}

Now that the container is available, the connection string can be added to the configuration produced by the infrastructure !

public class DatabaseInfrastructure : AsyncConfiguredInfrastructure
{
    private MsSqlContainer _container = new MsSqlBuilder().Build();
    private string DbName => $"TestDB_{ContextId}";

    public override async Task Destroy()
    {
        await _container.StopAsync();
    }

    public override async Task Initialize()
    {
        await _container.StartAsync();
        using (var connection = GetConnection())
        {
            await connection.OpenAsync();
            await CreateDatabase(connection);

            await connection.ChangeDatabaseAsync(DbName);
            await CreateTables(connection);

        }

        // This configuration will be added to the inmemory collection of the WebApplication !
        Configuration.Add("ConnectionStrings:SqlServer", BuildConnectionString());
    }

    public override Task Reset()
    {
        throw new NotImplementedException();
    }

    private async Task CreateDatabase(SqlConnection sqlConnection)
    {
        using (SqlCommand command = sqlConnection.CreateCommand())
        {
            command.CommandText = $"CREATE DATABASE [{DbName}]";
            await command.ExecuteNonQueryAsync();
        }
    }

    private async Task CreateTables(SqlConnection connection)
    {
        // Here you could use a migration tool like DBUp or EF Core to create the tables
        var sql = File.ReadAllText("Tables.sql");

        using (SqlCommand command = connection.CreateCommand())
        {
            command.CommandText = sql;
            await command.ExecuteNonQueryAsync();
        }
    }

    private SqlConnection GetConnection() => new SqlConnection(_container.GetConnectionString());

    private string BuildConnectionString()
    {
        SqlConnectionStringBuilder connectionString = new SqlConnectionStringBuilder(_container.GetConnectionString());
        connectionString.InitialCatalog = DbName;
        return connectionString.ToString();
    }
}

3️⃣ Resetting the Database with Respawn

To ensure a clean state between tests, we implement the Resetmethod using Respawn.

        public override async Task Reset()
        {
            await using (var connection = GetSqlConnection())
            {
                await connection.OpenAsync();
                await connection.ChangeDatabaseAsync(DbName);
                Respawner dbRespawner = await Respawner.CreateAsync(connection);
                await dbRespawner.ResetAsync(connection);
            }
        }

💡 You can exclude tables by passing a RespawnerOptionsinstance to the CreateAsync method. This can be useful when using DBUp or EF Core, which keep track of migrations within the database.

Respawner dbRespawner = await Respawner.CreateAsync(connection, new RespawnerOptions()
{
    TablesToIgnore = new Table[] { "MyTableToIgnore" }
});

The infrastructure is fully configured to be reset between each integration test!

4️⃣ Exposing a Method to Retrieve the Database Connection

We are going to create a public method to retrieve the SQL connection, which we will later use within our tests to interact with the database!

        public SqlConnection GetSqlConnection()
        {
            SqlConnectionStringBuilder connectionString = new SqlConnectionStringBuilder(_container.GetConnectionString());
            connectionString.InitialCatalog = DbName;

            return new SqlConnection(connectionString.ToString());
        }

5️⃣ Creating a Web Application

NotoriousTests fully integrates with .NET’s WebApplicationFactory. You can create a web application like this:

using NotoriousTest.Web.Applications;
...
    public class MyWebApplication : WebApplication<Program>
    {

    }

A WebApplicationis already configured to retrieve infrastructure settings from an InMemory collection. All WebApplicationFactorymethods can be overridden within the WebApplication.

Ensure that you add a partial class definition in Program.cs to support testing:

public partial class Program { }

6️⃣ Defining the Test Environment

Next, we create an environment that includes the infrastructure:

    public class TestEnvironment : AsyncWebEnvironment<Program>
    {
        public override async Task ConfigureEnvironmentAsync()
        {
            await AddInfrastructure(new DatabaseInfrastructure());

            await AddWebApplication(new TestWebApplication());
        }
    }

Since this is a web-based environment, we use AsyncWebEnvironment.
Other options include AsyncEnvironmentand AsyncConfiguredEnvironment.

Here's the difference:

AsyncEnvironment: Provides essential environment features, including infrastructure lifecycle management.
AsyncConfiguredEnvironment: Includes everything from AsyncEnvironment, but also enables configuration management with AsyncConfiguredInfrastructure.
AsyncWebEnvironment: Extends AsyncConfiguredEnvironment by adding methods to handle WebApplicationFactory and its automatic configuration.

7️⃣ Running Tests with the Environment

public class SampleTests : AsyncIntegrationTest<TestEnvironment>
    {
        public SampleTests(TestEnvironment environment) : base(environment)
        {
        }

        [Fact]
        public async Task Test1()
        {
            DatabaseInfrastructure sqlInfrastructure = await CurrentEnvironment.GetInfrastructureAsync<DatabaseInfrastructure>();
            await using(SqlConnection sql = sqlInfrastructure.GetSqlConnection())
            {

                HttpClient client = (await CurrentEnvironment.GetWebApplication()).HttpClient;
                HttpResponseMessage response = await client.GetAsync("WeatherForecast");
                Assert.True(response.IsSuccessStatusCode);
            }

        }
    }

CurrentEnvironment: This property provides access to the current test environment, which manages the lifecycle of infrastructures and web applications.
GetInfrastructure<T>(): This method allows retrieving a specific infrastructure instance, such as a database or message bus, ensuring it is properly configured and accessible for testing.
GetWebApplication(): This method retrieves the configured web application instance, which provides access to the HTTP client, enabling API testing within an isolated environment.

Your tests are now fully functional and completely isolated! 🚀

🎁 Enhanced Testing Framework: Structuring Acts and Asserts

One of the key strengths of this architecture is its reliance on the Environment class as the central component. This example, derived from a nutrition project built with ASP.NET Core and EF Core, demonstrates how this structure enables streamlined and efficient testing.

Thanks to this, we can design a structured framework class to encapsulate both test actions (Act) and validation logic (Assert), as well as setup actions (Arrange), which are not represented here:

Here you can see the framework :

    public class NutritionFramework
    {
        private readonly OMONEnvironment _environment;
        private readonly SqlServerDBInfrastructure _sql;
        private readonly NutritionDbContext _dbContext;
        private readonly HttpClient _client;

        public readonly NutritionAsserts Then;
        public readonly NutritionActs When;

        public NutritionFramework(OMONEnvironment environment)
        {
            _environment = environment ?? throw new ArgumentNullException(nameof(environment));
            _sql = _environment.GetInfrastructure<SqlServerDBInfrastructure>();
            _client = _environment.GetWebApplication().HttpClient;
            _dbContext = _sql.DbContext;

            Then = new NutritionAsserts(_dbContext);
            When = new NutritionActs(_client);
        }
    }

To streamline API interactions, all actions are encapsulated within a dedicated class that leverages HttpClient. This ensures test actions remain consistent and modular. Additionally, I utilize my own package, NotoriousClient, to simplify and enhance HTTP request handling.


    public class NutritionActs
    {
        private readonly HttpClient _client;

        private Endpoint ADD_INGREDIENT_ENDPOINT = new Endpoint("/nutrition/ingredient", Method.Post);

        public NutritionActs(HttpClient client)
        {
            _client = client;
        }


        public async Task<NutritionActs> AddIngredient(Ingredient ingredient)
        {
            HttpRequestMessage request = new RequestBuilder(_client.BaseAddress.ToString(), ADD_INGREDIENT_ENDPOINT)
                .WithJsonBody(AddIngredientFeatureParameters.From(ingredient))
                .Build();

            HttpResponseMessage message = await _client.SendAsync(request);

            Assert.True(message.IsSuccessStatusCode);

            return this;
        }

    }

The NutritionAssertsclass is responsible for validating test outcomes by querying the database to ensure expected entities exist with the correct attributes.

    public class NutritionAsserts
    {
        private readonly NutritionDbContext _context;

        public NutritionAsserts(NutritionDbContext context)
        {
            _context = context;
        }

        public async Task Exist(Ingredient ingredient)
        {
            Ingredient? actualIngredient = await _context.Ingredients.FirstOrDefaultAsync(i =>
                i.Fibers == ingredient.Fibers &&
                i.Fats == ingredient.Fats &&
                i.Carbohydrates == ingredient.Carbohydrates &&
                i.Proteins == ingredient.Carbohydrates &&
                i.Name == ingredient.Name
            );

            Assert.NotNull(actualIngredient);
        }

        public async Task Exist(int id)
        {
            Ingredient? actualIngredient = await _context.Ingredients.FirstOrDefaultAsync(i => i.Id == id);

            Assert.NotNull(actualIngredient);
        }
    }

Finally, writing an integration test that is both concise and well-structured for clarity:

        [Fact]
        public async Task Given_NoIngredient_When_AddingAnIngredient_Then_IngredientIsCreated()
        {
            var expected = new Ingredient()
            {
                Fats = 100,
                Carbohydrates = 100,
                Fibers = 100,
                Name = "Patate",
                Proteins = 100,
                Calories = 100,
                Category = Category.GrainsAndStarches
            };

            await Framework.When.AddIngredient(expected);
            await Framework.Then.Exist(expected);
        }

End Word

I hope you enjoyed this brief journey into the world of integration testing and that NotoriousTests proved to be a valuable tool for you! Don't forget to leave a like and to share this article !

For more information, feel free to:

Read the README on GitHub
Ask questions, provide feedback in the GitHub issues or GitHub discussions
Give a ⭐ to the repository!

Have a great day!