DEV Community: Dennis

Olen lopettanut mocking kirjaston käyttämisen. Tässä on syy

Dennis — Sat, 10 Oct 2026 10:56:35 +0000

ℹ️ Opin suomea
Tämä on mulle harjoitus oppimaan suomea. Voit myös lukea englanniksi.

Se tuntuu ihan itsestään selvältä: Kun luot uuden unit-testing projektin, asennat Moq tai NSubtitute myös. Miten muuten vaihdat riipuvuuden test-double-ksi? Olet hölmö kun kirjoitat omat test-double-t, eikö niin? Tässä artikkelissa, haluaisin vakuuttaa sinut että se ei ole niin hölmö kuin ajattelet.

Testien luettavuus on parannettu

On helpommin näytetty esimerkissä. Tässä on esimerkki NSubstituten kanssa:

[Fact]
public void ShouldGetUserNameById()
{
    // given
    var userRepository = Substitute.For<IUserRepository>();
    userRepository.GetById(1234).Returns(new User(userName: "JohnDoe"));
    var userService = new UserService(userRepository);

    // when
    var result = userService.GetUserNameById(1234);

    // then
    Assert.Equal("JohnDoe", result);
}

Se ei ole huono. Vertaillaan sitä ja testiä joka käyttää mukautettua test-double:

[Fact]
public void ShouldGetUserNameById()
{
    // given
    var userRepository = new FakeUserRepository()
        .WithUser(id: 1234, userName: "JohnDoe");
    var userService = new UserService(userRepository);

    // when
    var result = userService.GetUserNameById(1234);

    // then
    Assert.Equal("JohnDoe", result);
}

Testillä joka käyttää mukautettua test-double on vähempi sanoja. Tämä näyttää merkityksettömältä, mutta ero tulee huomattavaksi kun käytät monta test-double.

Testit tulee itsenäisempi toteutuksesta

Esimerkissä näet että versio joka käyttää NSubstitutea määrittää vain yhteen funktioon tuloksen. Versiolla joka käyttää mukautettua test-double ei ole mitään viiteitä funktioihin. Tuo tarkoittaa että on vain yhteys test-double-n kanssa, mutta ei ole suoraa yhteystä oikean funktion kanssa. Siksi oikea funktio on helpompi muuttaa. Eli, ei mun tarvitse muuttaa kaikkea testeja kun muutan funktion.

Lisäksi, mulla on lisää vapaus toteuttaa ominaisuuden. Versio joka käyttää NSubstitutea on riipuvainen toteutuksen yksityiskohtasta: funktiopuhelu jota käytän toteutukseksi. Joskus on monta tapaa saavuttaa tavoitteen, mutta mocking kirjasto vaatii valita ratkaisun ennen toteuttamisen. Tämä voisi olla hankala.

Mulla on nyt kätevät työkalut

Kun totut kirjoittamiseen omia test-double-ia, alat kerätä käteviä työkaluja joita kiihdyttävät testikirjoitamisen. Mulla on työkalut:

Asetuksille IOptionsMonitor<...>n avulla
API pyyntille ja vastauksille HttpClientin avulla
Umbracon IPublishedContentille, modelsbuilder mallille, sivu navigointille ja URL luomisille
TelemetryClientille Application Insightsista

Tästä syystä olen myös löytänyt olemassa olevia työkaluja päivämäärille ja kirjauksille. Näiden työkalujen avulla testeistä tulee helpompia lukea ja kirjoittaa.

TDD tuntuu luonnollisempi

Tämä on henkilökohtainen syy kuinka pidän testien kirjoittamisesta. Kuvittelen miten testi voisi näyttää ja ensin kirjoitan sitä. Yleensä testit eivät käännä, koska mä keksin funktiot, tyypit ja ominaisuudet jotka eivät vielä ole olemassa. Tämä on erityisen tehokas tekoälyavustajan kanssa, koska se on ihan taitava saamaan koodit valmiiksi. Tämä on vaikeampi mocking kirjaston avulla, koska mun pitää ensin ajatella kirjaston käytämisestä ja unohdan ajatella miten testi pitäisi näyttää. Ajattelen toteutuksesta ennen kuin olen ajattelut suunnittelusta.

Johtopäätös

Oman test-double-n tekeminen ei ole niin hölmö kuin ajattelet. Mocking kirjastot ovät voimakkaat työkalut, mutta sun ei pitäisi aliarvioida miten paljon voit tehdä oman yksinkertaisen test-double-n avulla. Ne parantavat testien luettavuutta ja voisivat tehdä testit itsenäisempi toteutuksesta.

Kiitos lukemisesta ja nähdään ensi blogissa! 😊

I stopped using mocking libraries, here's why

Dennis — Sat, 10 Oct 2026 10:56:01 +0000

It seems like a no-brainer: When you create a new unit testing project, you install Moq or NSubstitute as well. How else are you going to fake your dependencies? You'd be crazy to write your own implementations, right? In this article, I hope to convince you that it's not as crazy as you might think.

Readability of my tests improved

This is easiest demonstrated with an example. Here is a traditional example using NSubstitute:

[Fact]
public void ShouldGetUserNameById()
{
    // given
    var userRepository = Substitute.For<IUserRepository>();
    userRepository.GetById(1234).Returns(new User(userName: "JohnDoe"));
    var userService = new UserService(userRepository);

    // when
    var result = userService.GetUserNameById(1234);

    // then
    Assert.Equal("JohnDoe", result);
}

It's decent, but let's compare it with the same test using a custom fake:

[Fact]
public void ShouldGetUserNameById()
{
    // given
    var userRepository = new FakeUserRepository()
        .WithUser(id: 1234, userName: "JohnDoe");
    var userService = new UserService(userRepository);

    // when
    var result = userService.GetUserNameById(1234);

    // then
    Assert.Equal("JohnDoe", result);
}

The test with the custom fake is much less "wordy". This may seem insignificant in a test like this, but once you get to your third test-double, the difference becomes noticeable.

Tests are more decoupled from implementations

You can see in the previous example that the version with with NSubstitute specifies the result for a single specific method. The example with custom test-double does not reference any specific method. This means that the test is only coupled to the signature of the test-double and not to the signature of the dependency itself. This makes the dependency more easy to change. To be more specific: When the signature of the dependency changes, I don't need to update all the tests that use it.

Additionally, I have more freedom in how to make my test pass. The version with NSubstitute relies on an implementation detail: the method call which I use to implement the requirement. In some cases, there may be multiple ways to achieve the same result and the mocking library forces you to choose which option to take beforehand. This may or may not be desirable.

I have a bunch of useful tools now

When you get used to writing your own test-doubles, you end up writing reusable tools that make testing easier and faster. I now have my own custom testing tools for:

Configuration using IOptionsMonitor<...>
Api requests and responses using HttpClient
IPublishedContent from Umbraco including modelsbuilder models, content navigation and URLs
TelemetryClient from Azure Application Insights

This has also led me to discover existing reusable testing tools for dates and times and logging. All of these tools have made my tests easier to read and easier to write.

TDD feels more natural

This is a more personal point about how I like to write my tests. I simply like to imagine how my test could look and just write that out. Often, my tests can't compile at first, because I made up function calls and types and properties that don't exist yet. This is particularly effective with AI assistants nowadays, which are very good at filling in the gaps. This is more difficult when I use mocking libraries, because I start thinking about how to write the test-double and forget to think about how the test reads. My mind shifts towards implementation before I'm done thinking about design.

Conclusion

Writing your own custom test-doubles is not as crazy as you may have thought. While mocking libraries are powerful tools, it should not be underestimated how much you can do with simple custom test-doubles. They improve the readability of tests and can decouple tests more from implementation.

Thank you for reading and I'll see you in my next blog! 😊

Improve code quality in dotnet using AI and code analysis

Dennis — Fri, 02 Oct 2026 06:53:11 +0000

The amount of code that is written has significantly increased with the rise of LLMs. Along with it, grows uncertainty about the quality of our work. How do we know that the code written by AI is good? How can we trust that a non-deterministic process generates high-quality code without bugs? In the practice of Harness-engineering, this uncertainty is addressed with sensors: deterministic tools that validate the output of LLMs (or people) using consistent rules.

In this blog I propose one possible answer to the question: how do we build such sensors in dotnet applications? I will tell about my experience with building code analyzers and give some general tips for when you want to dive into the topic for yourself.

Roslyn analyzers

One option is to write roslyn analyzers. Roslyn is the name for the C# compiler that the dotnet CLI and visual studio use to build your source code. If you've enabled nullable reference types or you've installed StyleCop before, then you're already familiar with what roslyn analyzers do: They identify problematic code patterns and give you warnings and errors at build-time.

As it turns out, LLMs are very good at writing roslyn analyzers. It's also very easy to scaffold a code analyzer project using Microsoft's instructions. I got my first simple working analyzer up in about 3 hours. Custom roslyn analyzers have these benefits and drawbacks:

✅ Easy to make with LLMs: With a good description of what you want, LLMs are excellent at writing custom analyzers.
✅ Quick to get started with: Setting up a project with code analyzers is easy to do.
✅ Low risk: The code for analyzers does not run in production, only on your machine during development. Code quality for analyzers is therefore not as important as otherwise. If it doesn't do what you need, you can throw it away without losing anything.
✅ Also benefits development without LLMs: Recognizing bad code patterns is good no matter how you develop. Being able to enforce good practices is a super power no matter what.
✅ Exposes blindspots: Code analyzers are deterministic and will therefore always work the same. If you failed to enforce your rule anywhere in your code, then an analyzer will find it, guaranteed.
❌ Cannot co-exist with source code: Code analyzers are either installed as NuGet package or as Visual Studio plugin. You cannot add analyzers as a class library alongside your source code. You can therefore also not evolve it together with your source code. You must separately publish a NuGet package or Visual Studio plugin.
❌ May significantly increase warnings in code: If a code analyzer rule is invasive, it may introduce many new warnings. If you have warnings-as-errors enabled, then a code analyzer that introduces many new warnings may not be productive. A long list of warnings may hide other important issues.
❌ All warnings must be fixed: You can only enforce the rules that you make by failing the build. If you don't do this, the LLM may choose to ignore your warnings.

When are Roslyn analyzers a good choice?

Roslyn analyzers work on the syntax tree of your code. While they are good at identifying patterns in syntax, they are not great at inferring intent. In other words: It's good at seeing what you do, but not good at understanding why you do it.

An example of a good use-case is as follows:
A reference to a disposable object should not escape the scope of the using block in which the object is created.

using (var myObject = new SomeDisposableType())
{
    return new ResultType(myObject); // WARNING: disposable object should not escape the scope of it's corresponding using-block.
}

This is a concrete syntax pattern with demonstrable negative effects: consumer code may throw ObjectDisposedException. A Roslyn analyzer can recognize this syntax pattern and generate warnings for it.

On the other hand, a Roslyn analyzer is not good at inferring intent. For example:
Two or more mutually exclusive properties on a model should be replaced with a single discriminated union.

public class MyPageModel
{
    // WARNING: two mutually exclusive properties should be replaced with a single discriminated union
    public HeaderSmallVariant? HeaderSmall { get; init; }
    public HeaderLargeVariant? HeaderLarge { get; init; }
}

This case is ambiguous: is it possible to have both a small and a large header at the same time? Is it possible to have neither? We can only infer this by looking at how the type is used. We need to use context clues to know what the writer of the code meant.

Tips to get started

Here are some broad steps and findings that will help you get up-and-running with analyzers.

Follow the Microsoft instructions for setting up an analyzer project. The scaffolded project works out-of-the-box and comes with an example.
Do not immediately switch the test-framework. You should update all packages first to the latest versions. You'll find that several packages need to be replaced. NuGet recommends alternatives and those alternatives are test-framework agnostic. After replacing the packages, you'll need to fix some build errors. The fixes are straightforward.
Ask the LLM to replace MSTest with your test-framework of choice. The LLM can do this very well.
Do not upgrade Microsoft.CodeAnalysis.CSharp and Microsoft.CodeAnalysis.CSharp.Workspaces to the latest version. If you're still developing applications on dotnet 8, version 4.3.0 works well. If you're on dotnet 10, you could also use 5.0.0, but 4.3.0 works for both. Upgrading to the latest version will cause additional warnings when you install your analyzer package in a project.
If you have an example of bad code in your source repository, ask the LLM to export the sample into a document. Also find examples of better code. Make the LLM write an explanation of why the bad code is bad and the good code is good. Ask the LLM to fully qualify types in the document. When you think the document is complete, move over to your analyzer solution and feed it to the LLM. Ask it to plan an analyzer to find warnings for the bad code. This gets you the best results.
Ask the LLM for examples in the form of unit tests. The unit tests for code analyzers are easy to read and illustrate how the analyzer works.

At the moment I still have one problem: I am unable to debug the analyzer and manually test it with the visual studio plugin. I don't know why. It hasn't been a big problem for me, because the unit tests already helped me solve all the problems that I had thus far.

Final thoughts

It was surprisingly easy to get started with code analyzers using LLMs. This experiment has made me think deeper about what code quality means and if it's possible to make it more measurable somehow. Although I think code analyzers have somewhat limited applicability, they do seem to be very helpful in some cases. Now that the bar has been significantly lowered to create code analyzers, I believe they are a viable option for everyday development tasks, with or without LLM support.

Thank you for reading and I'll see you in my next blog! 😊

I attended Umbraco Codegarden 2026!

Dennis — Mon, 15 Jun 2026 07:00:00 +0000

Codegarden was last week in Copenhagen! I was there this time and here are my thoughts about the event. Previous time I attended was in 2023, so I will draw some comparisons to that year. It was a great experience and I left feeling generally excited and inspired.

Organization

Just like previous time, the event was very well-organized. At Codegarden, you're always well taken care of with lunch and dinner, great instructions, a little kick-off at the start of every day and a lot of spatial diversity. You always know what to expect.

I was very happy to see that the quiet area was easy to find and it was mentioned at the beginning of the event. The previous time I didn't know that such a quiet area existed and I spoke out about that. It gave me comfort that I knew where to retreat to when I needed to. Additionally, there were many places to stand or sit outside and inside. The event was also a little shorter, which made it more manageable for me.

The location was absolutely ideal for the amount of people that attended. There were many attendees (about 700 they said, if I remember correctly), but at no point did the venue feel crowded. I could always easily get to where I wanted to go, there were no long queues and there was always enough seats for all the talks at every stage.

Talks

I'll admit that, on the previous Codegarden, I had the wrong expectations for the talks. Talks are obviously aimed at specific target audiences and I don't belong to all of them. Some talks aim to inform, while others aim to inspire. Stories are not tutorials. Also, sometimes a talk doesn't teach you something new, but simply validates what you were already doing.

So it was this time too. My focus is currently shifting slightly away from coding and more towards collaboration and I am looking for inspiration on how to deliver better software faster. I specifically wanted to attend Debugging Teams: What High-Performing Developer Teams Do Differently, because it seemed relevant to me. Although I was hoping for a deep analysis in development process and teamwork strategies, I ended up learning about the importance of trust and of sharing the "what", "how" and "why" of what we do. It surely gave me something to think about.

I listened to Building Resilient Umbraco Integrations. A more tutorial-style demonstration with insightful information about what you need to think about when integrating with external services. The presentation included concrete code examples that gave concrete solutions to the presented problems. I personally already knew all of it, but it was nice to see anyway, because it confirms that I'm already doing it right.

A special mention goes to Semira Allen and Jason Wodicka with their talks Drowning in Details: Turning TechTalk Into Everyday Language and How (not) to mess up everything. Their talks were incredibly engaging and they were a joy to listen to. I had a good laugh and they were each inspiring in their own ways.

Some additional thoughts

Copenhagen: The fact that it was in Copenhagen was what made me consider it again. I am not big on traveling abroad and the trip to Odense and back was seriously daunting. Being able to get there with just the airplane made the whole trip so much better!
AI: I haven't mentioned AI in this blog. That's because I find the ethics of the application of AI at the moment very questionable. I am still working on forming a well-informed opinion. Umbraco and the community have showcased some very impressive work with AI.
Umbraco awards: In case you didn't know it yet: we were nominated for an Umbraco award for "Accessibility and Security"! Even though we didn't win the award, I believe that our submission was still the best. The winning solution had a more visible approach to accessibility, but we can show off a real certification of accessibility with end-user testimonies to back it up. The nomination is a nice validation of my team's hard work.
Umbraco in the bigger picture? Right now I am learning from people like Dave Farley, Emily Bache and Derek Comartin about better software development practices. Some of it is more about architecture and some of it is about collaboration and process. I feel like these topics deserve more attention in the Umbraco community. I need a talk named How I upgraded my mid-size web agency from feature branch and PR drag to real CI/CD. Maybe I get to give this talk myself one day, who knows.

Conclusion

It was a great event, and I'm very happy that I could be there. If it's like this again next year, then I will be eager to attend. Thank you Umbraco for everything!

I'm learning python with AI and TDD

Dennis — Sun, 15 Mar 2026 08:57:27 +0000

I decided to make my next hobby project in python. I used this project to learn python, but also to become familiar with AI-assisted coding. In this article I document my experiences with AI as a pair programming buddy, doing Test-Driven Development together.

About TDD, the project and me

Test-Driven Development is about writing tests before writing your code. This is often done in the Red, Green, Refactor cycle: Write a failing test, specifying the behaviour that the code should exhibit, make all tests pass with as few changes as possible and finally make your code (and your tests) nice. It's a design tool that helps make your code easier to understand and maintain.

The project is a self-hosted information database that connects Animal Crossing Amiibos to information on Nookipedia. An API exposes the data so that an app can scan an amiibo and direct the user straight to the character's page on the Nookipedia. The project, besides providing desired functionality, is also an excuse for me to try out some enterprise software patterns that I wish to learn more about.

D-Inventor / my-animal-crossing-app

This is a hobby project. It is a combination of an app for android and a python-based server API. The server supplies information about Animal Crossing villagers. The app lets me scan Amiibo cards and then I can see basic information about the villager.

This is a hobby project. I wanted to get easy access to basic villager information from my Amiibo card collection.

This project combines an android app with a python-based API so that I can scan the NFC chip of an amiibo card and then fetch the information. That's all.

View on GitHub

I am a dotnet developer with very little python experience. Python has a special place in my heart, because it's the first language that I earned some money with. That was more than 10 years ago and I haven't touched python that much since then.

AI as a pair programming buddy

Most of the code in this project has been written by AI. Overall, the results were mostly good, sometimes very frustrating, but sometimes also very impressive.

Scaffolding a project baseline

Yes, AI got me started very quickly and effectively. I knew what I wanted: multiple python projects in a monorepo with some shared modules. I know what python code looks like, but actually setting up a project like this is quite a lot. Setting up testing with pytest, multiple independent modules with a shared pyproject.toml. Module imports are very confusing in python and if you don't know what you're doing, they're difficult to get right sometimes. I asked the AI to do it for me with working examples of unit tests and shared modules. It did all of it, the code ran, the tests worked and the code looked acceptable. Big win 💪

So many comments, so much "documentation"

It was really my own fault and it was quickly fixed. The AI was writing excessive comments. Every generated function filled with comments and "documentation", detailing exactly what the code does. That's not what I need comments for. I can read the code to see what it does. The thing here is that I let the AI generate its own instruction file and the instruction file literally instructed the AI to do this. I simply updated the instruction file and instructed the AI to never write comments.

Stop! Wait! Not so fast! Not so much!

Some people may find it wonderful: just tell the AI what you want to make and it'll happily build the whole thing for you. Or try to at least... Although it's magnificent, I think it's also incredibly irresponsible to let AI generate large amounts of code at once. Especially in a professional environment, I would be held personally responsible for bugs and mistakes and saying "it's the fault of the AI" simply will not fly. It's essential that I can review the code that the AI generates and I can only do this if I develop in small steps.

Test-Driven Development has been very helpful to put a leash on the AI. AI is very eager to write code and will simply sprint to the finish line if you let it. By strictly following the Red, Green, Refactor cycle, I was able to stay more in control of what the AI does and how it does it. It took a bit of trial-and-error, but the following instructions seemed to be very helpful:

Follow strict TDD practice with Red, Green and Refactor phases
Always run the test in the Red phase to verify that the test fails in the way that we expect
Prefer runtime failures over compile-time failures at all times
Prefer failures on assertions over failures in code under test whenever this is reasonable
Make sure in the green phase to write as little code as possible to make the test pass
Always ask for verification from the user after each phase before proceeding to the next

Additionally, it seemed helpful when I start each session with "We are pair programming buddies and we are going to do Test Driven Development". Nevertheless, the AI would still occasionally do more than I asked for, especially the green phase was difficult at times.

Please give me an example in the form of an automated test

In this project I needed a handful of slightly more advanced concepts, including:

Messaging
HTTP API calls
Database querying

Though the concepts are familiar, I had never done them in python specifically. I know approximately what good integrations look like, but I simply needed some examples in python specifically. AI has been an excellent tool for learning and it helped me to understand python frameworks, tools and syntax. By asking about it in the shape of an automated test, I could also immediately apply the concepts I learned with Test Driven Development. This was a massive win! I would like to share a particular experience that really blew my mind.

I needed to build an integration with an external HTTP API. Coincidentally, I recently also had to do this for work in dotnet, so I had a good idea what a good integration looks like and also how a good unit test looks for such an integration. It had taken me about 5 hours to build in dotnet. I asked the AI to generate me an example of a unit test that demonstrates an http request and response in python. the unit test was quite bloated and unreadable, but it did work. I told the AI as much and I gave it an example of what I expected the test to look like, based on a test that I wrote in dotnet. I asked to refactor the test to make it as close as possible to the example that I gave. In just a few minutes it generated exactly the testing utility that I had spent several hours on earlier! This was shocking and incredibly impressive.

Thoughts overall

In general, I find the workflow decent. Although I did get much further with AI in a shorter amount of time than I would have without, I don't believe (yet) that AI is a massive time-saver overall. That sounds like a contradiction, but you must see that it was my lack of understanding of the python ecosystem that held me back and not the speed by which I type code. Although the AI can create code much faster than me, the code is often not up to standards and at times I have had to throw away work of the past 10 minutes, because prompting for corrections was only making things worse.

An important thing, in my opinion, is that AI may replace your typing, but shouldn't replace your brain. The skills that I've developed as a professional developer are still required to judge the quality of the output and are still essential to prioritize long-term success over short-term gains.

So yeah, if you haven't tried it yet, perhaps you could also turn your AI into your TDD pair programming buddy. Thank you for reading and maybe I'll see you in my next blog! 😊

Umbraco testing examples now also for Umbraco 17

Dennis — Wed, 25 Feb 2026 17:48:29 +0000

I updated my testing examples repository to the latest Umbraco version.

D-Inventor / automated-testing-in-umbraco

A working example of integration- and unittests with Umbraco. A demonstration of various concepts for testing your Umbraco website

Umbraco 17 automated testing setup

This project is a fully functioning setup for automated testing with Umbraco 17. You can use this project as a reference or starting point to get started with testing on your Umbraco website. The tests are set up with Test Driven Development (TDD) in mind.

Tools

The most important tools that are used in the automated tests are as follows:

Name	Description
xUnit v3	The testing framework. You can use any testing framework that you like though
NSubstitute	Library for mocking. Any mocking library will work. This project doesn't do extensive mocking, but for example `IPublishedValueFallback` is a mandatory parameter for any published content item, even if you don't actually use it. It's just convenient to insert a mock.
Test Containers	Automatically creates docker containers while running tests. It is used to create an empty SQL Server database that is automatically cleaned up after testing.

…

View on GitHub

Though not much has changed, here are the most notable differences compared to the Umbraco 16 version

Database initialization

The SqlServerDatabase resource no longer creates a second database inside the SqlServer test container. While Umbraco 16 would happily create a new database on boot if it didn't exist yet, Umbraco 17 does not and will throw connection errors.

For your regular use of Umbraco, this change makes no significant difference. If you also use EF Core, you may need to pay some extra attention here!

A base URL for integration tests

The WebsiteFixture will now set a fixed base URL when creating a backoffice httpclient. If you need to use a different domain, this base URL can be adapted, but what is important is that the URL uses https protocol. Compared to Umbraco 16, Umbraco 17 requires https to backoffice endpoints by default and requests using http are rejected by default. That's at least what the error would indicate.

If you want to see all the changes, you can check out this commit:
https://github.com/D-Inventor/automated-testing-in-umbraco/commit/573ef96a155c390ba3d4383bf555f9ff5723b3b4

An additional note

I have given some attention to the example website under test in this repository. The homepage at least has a slightly more fancy template, but it's all static content. I haven't yet taken the time to really tie it all together, and I wanted to do so using Test Driven Development, which is the main focus of this example repository.

I noticed that the version of playwright for C# doesn't actually let you compare screenshots!? This was a mild surprise and disappointment, because that means I can't actually use snapshot testing to ensure my template doesn't accidentally change when I implement the logic. The Javascript version of playwright does seem to support it, so I guess for better end-to-end testing, we'd need to create an equivalent management API client and scenario builder. Oof!

That's all

that's all I wanted to share! Hope you check out the repository, give it a star and let me know if the testing examples have been helpful to you or not! Thank you for reading 😊

Learning TDD by doing: Acceptance testing with Umbraco's management API

Dennis — Mon, 27 Oct 2025 10:00:00 +0000

Remember: 🔴 Red, 🟢 Green, ♻️ Refactor

In my previous post I speculated about end-to-end testing with Umbraco's management API. I have tried it out and I can confirm that it works! In this post I will dive into what I wanted to achieve, how I did it and what challenges I faced along the way.

The goal

My goal was to automate content scenarios in order to provide a consistent initial state for acceptance tests. The result should be a suite of tests that interact with the website like an end user would.

If you want to skip straight to the end result: check out my github repository with testing examples. I shared all my code there.

D-Inventor / automated-testing-in-umbraco

A working example of integration- and unittests with Umbraco. A demonstration of various concepts for testing your Umbraco website

Umbraco 17 automated testing setup

Tools

The most important tools that are used in the automated tests are as follows:

Name	Description
xUnit v3	The testing framework. You can use any testing framework that you like though
NSubstitute	Library for mocking. Any mocking library will work. This project doesn't do extensive mocking, but for example `IPublishedValueFallback` is a mandatory parameter for any published content item, even if you don't actually use it. It's just convenient to insert a mock.
Test Containers	Automatically creates docker containers while running tests. It is used to create an empty SQL Server database that is automatically cleaned up after testing.

…

View on GitHub

The journey

There were two steps to my plan: First, I created a client that could communicate with Umbraco's management API. Then, I built an interface on top of the client, so that I could describe the different kinds of content in easy-to-use models. I also created a small website to create a realistic test-case to use these tests on.

Step 1: Management API Client

Umbraco's management API is the interface that the Umbraco backoffice communicates with when you edit content. It is built with external integrations in mind.

To create the API client, I used NSwagStudio. With this tool, I can consume the OpenAPI spec that Umbraco exposes and automatically generate a C# client. Although it didn't go perfectly, the generated client was very usable and Umbraco appears to have designed very nice and clear models and endpoints. The generated C# code couldn't compile right away, because for some reason it depended on a type FileResponse, which doesn't exist. Fortunately, creating an empty class was enough to make it work.

NSwagStudio does not generate the authentication code, so I had to extend the generated client to include authentication. This was easy enough with Umbraco's documentation on external access. After this, I was able to write a crude script that could access the backoffice and produce some basic content items.

Step 2: A layer of convenience

The client works, but preparing a scenario using the Management API Client alone takes a lot of code. My goal was to be able to describe a scenario in code the same way as I would describe it to a coworker. So before I got to work, I wrote down some example sentences that I would use to describe a test scenario:

Given a content page with a tag, when I go to the page, I see related content with the same tag.
Given a content page with the content from our figma design, when I go to the page, I see the page just like in our design
Given a news overview with many news articles, when I load more articles, I see the next N articles

I designed a test that would describe a simple scenario:

[Fact]
public void ShouldDisplayTitleFromContent()
{
    // given
    var contentPage = Scenario
        .WithContentPage()
        .WithHeader(title: "My content page");

    // when
    User.Visits(contentPage);

    // then
    Assert.Equal("My content page", CurrentPage.Title);
}

The focus here was to get the 'given' part working. I used test-driven development to design and build this scenario builder. The tests that I wrote are included in the github repository, so you can have a look at those if it interests you. The fun thing about test-driven development is that a test makes me think about how I'm going to use the code that I'm about to write.

In the end, I didn't get exactly the design that I wanted, simply because "I want content with a header" is too vague, but I did get close. The test above ended up looking like this:

[Fact]
public async Task ShouldDisplayTitleFromContent()
{
    // given
    Scenario.Homepage()
        .HasContent(Variation.Invariant, content => content.WithHeader(title: "Welcome to the website"));
    await Scenario.BuildAsync(TestContext.Current.CancellationToken);
    var homepage = new HomePageObject(await Browser.NewPageAsync(), Scenario.Website());

    // when
    await homepage.GoToAsync();

    // then
    await Expect(homepage.Title).ToHaveTextAsync("Welcome to the website");
}

Instead of saying "The homepage has a header", it now says "The invariant content of the homepage has a header". It's a bit longer, but also more explicit. It also allows you to specify cultures and variants. Very useful when you want to test multilingual sites or if you use segments to personalize content. Additionally, I couldn't get around an explicit call to BuildAsync, because I need a trigger to actually make the changes.

Challenges / difficulties

Here are some of the things that surprised me, some things that I found difficult and some roadblocks that I came across while developing this.

Visual regression testing? Yes, but not in dotnet

The biggest challenge and also my largest surprise, was the fact that there don't seem to be any visual regression testing tools available for dotnet. Playwright for example supports comparing screenshots in the version for javascript / typescript, but does not support this for its dotnet version. I had no plans to create my own visual regression testing tool, so I had to give up. It might have been better to create my scenario builder in javascript / typescript, in retrospect.

This was somewhat disappointing for me. In my team at work, it is very important that our websites look like the designs that we make upfront. One of our success criteria is that we can recreate the design using content in Umbraco. Being able to match a browser screenshot with an export from our design software would have been very valuable.

Domains in Umbraco didn't quite work

Just before writing this blog, I submitted a bug report to Umbraco, because it appears that Umbraco has a stale caching layer on top of their domain service. When I ran the tests more than once or on multiple scenario's, I would run into bad status codes because the domain was reserved for the content of the previous scenario, even though that content was deleted.

UPDATE November 8 2025:

Andy Butland suggested on the bug report to call the cache refresh endpoint before setting up a new scenario. I tried this out and it reliably solved this issue. It's still a work-around, but in some way it makes sense to empty all caches before setting up a test scenario. I updated the testing example repository to call the cache refresh endpoint and removed my old work-around.

Content is inherently complex

My example on GitHub and the example code above is simple on purpose. String values are easy to model. But how do you model related media? Images with crops? What about a blocklist? These types of content are inherently complex and there is no single best answer. Blocklists have quite complex rules, take this article in the Umbraco documentation about block-level-variance for example. Media has its own section in the backoffice and introduces an additional external dependency: the filesystem. Do I need to recreate all media items on every test as well? Or maybe I can simply reserve a set of media items in the backoffice and reuse them? It really depends on your use-case and I haven't quite figured this out for myself yet.

Layers and abstractions

I found it difficult to assign clear responsibilities between the ScenarioBuilder and the DocumentClient. I had a gut-feeling that somewhere here is supposed to be a "repository". Intuitively, the ScenarioBuilder seemed to play the role of repository, because it translated domain models into DTOs for the management api, but after building everything, the IDocumentClient ended up looking more like a repository than the ScenarioBuilder. In the end, I couldn't really make a proper case for either, so neither of these types turned into a repository. This once again shows how test-driven development helps you make better design choices. I didn't need a repository, so I didn't make one.

final thoughts

Most of all, I'm very excited that I managed to make it work. I have to admit that without visual regression testing, I'm not exactly sure how I'm going to use this technology. That doesn't mean that it's useless though. I feel like there is a lot of potential and I am very happy with how easy everything turned out to be.

Please check out my repository in GitHub with testing examples. It includes everything that I talked about in this post. Let me know your thoughts: do you have extensive test suites for your Umbraco sites? What kind of tests do you find most valuable?

Thank you for reading and I'll see you in my next blog! 😊

Learning TDD by doing: Is Umbraco's management API the answer to acceptance testing?

Dennis — Mon, 15 Sep 2025 08:00:00 +0000

Remember: 🔴 Red, 🟢 Green, ♻️ Refactor

For a while I've been thinking about how to approach acceptance testing in an Umbraco application. It's a topic that I've been stuck on for a while, because I couldn't figure out how to set up test scenario's in Umbraco on a real website for testing. In this post I will explain what makes this so difficult and how I think the management API might help. This post is more speculative and I will share more once I've given this an honest try.

It's been a while since I've written about TDD. The thing is: I have now been writing unit tests for a while and unit testing has now become somewhat repetitive and not necessarily engaging to write about. This is a good thing: it means that I have gotten better since I started doing this. Yay for me 😅. However, now I need to find new topics to explore and specifically acceptance testing has been on my todo-list for a while. I just couldn't figure out where to start with it... until now.

About acceptance testing

The idea of acceptance testing is to engage with your system from the perspective of a user. If your user visits your website with a webbrowser, you'll want to test your system with a real browser, when your system is deployed to a real environment. An acceptance test verifies that a user can do what you want them to do by simulating different scenarios and user interactions. This is the difficult part:

What makes it so difficult?

The most important part of a good test is that it starts with a predictable state. Without control over your entire system, your tests might become flaky. For example: A test that verifies that your "related content block" shows a specific set of related content, requires a predictable set of related content pages. A test that runs before or parallel to this test, may modify the content of your site and thereby influence the result. This is highly undesirable.

In an Umbraco website, the state is your content, your media, configured domains and languages and sometimes even members. If you want to do acceptance testing, you need to be able to create a consistent content tree.

There are different ways to approach this challenge:

Create a brand new database and website for each test(set). This is great for parallel testing, provided you have the ability to create new websites and databases on-demand and you can clean these up afterwards.
Use a single website and database and reset the content tree before every test(set). This option requires each test(set) to run in series, but the hosting setup is less complex. You also don't need to worry about cleanup afterwards, because cleanup is part of the setup for each test(set). You may experience flakiness if you don't clean up properly.

Neither of these are easy to set up. For me it's simply not possible to create websites and databases on-demand, so that option is off the table. So, how do you set up different content scenario's in a single Umbraco website then?

Umbraco management API for setting up scenarios

Setting up content programmatically on a real environment requires two things:

Secure machine-to-machine authentication
An HTTP API to influence the backoffice

In Umbraco 13, you would have had to build both of these yourself. I would go as far as to say that it is not worth the effort. In Umbraco 16 however, this is not only possible, but quite easy.

If you visit the backoffice and go to the Users section, you'll find that, in addition to regular users, you can create API Users. An API User follows the same permission model as regular users, but allows you to create machine credentials. That takes care of the first point.

The machine credentials allow you to connect with the backoffice. Specifically to Umbraco's management API. The management API is Umbraco's interface for managing the state of the backoffice. You can create and modify content, media and everything else, just like a regular user would. By leveraging the management API, you should be able to create a consistent content tree.

Since Umbraco implements OpenAPI, you might leverage that to generate your own API client. I'm interested to see how this performs with NSwagStudio: my preferred C# client generator. With a management API client, it should be possible to create reliable acceptance tests.

What a test might look like

In an acceptance test, you use the domain language to specify your requirements without technical terms. You may use a tool for that, but you can also just do this in C# (or any conventional programming language for that matter). Here is how I imagine a test might look:

[Fact]
public async Task ShouldUseSEOTitleAsPageTitle()
{
    // given
    await Scenario
        .HasBaseContent()
        .Has<DetailPage>().WithSEO(title: "Title overwritten by SEO")
        .BuildAsync();

    // when
    await User.VisitAsync(Scenario.DetailPages[0]);

    // then
    Assert.Equal("Title overwritten by SEO", Page.Title);
}

The idea here is that the Scenario is a builder that defines the starting condition. The .BuildAsync() call would communicate with Umbraco through the management API and set up the content. The User could be a selenium webdriver and the test finishes with a classic xUnit assertion.

Obviously, this is only the tip of the iceberg: How do I set up this "base content"? How exactly does the scenario builder work? How is the user implemented? There are a lot of open ends, but I believe it can all be worked out.

In conclusion

Although all of this is speculative, the Umbraco management API seems like a suitable tool for acceptance testing inside a real environment. In combination with API users, it provides an integration point through which testing scenarios can be configured.

I am very interested to explore this further and will ofcourse share an update when I get to it. Exciting!

The ultimate Umbraco automated testing reference?

Dennis — Mon, 04 Aug 2025 05:49:34 +0000

I published my testing setup with Umbraco to GitHub! For a while, I've been writing about testing and test-driven development in Umbraco. I've shared several examples to demonstrate how I test certain things and now I'm bundling everything together into a single working repository:

D-Inventor / automated-testing-in-umbraco

A working example of integration- and unittests with Umbraco. A demonstration of various concepts for testing your Umbraco website

Umbraco 17 automated testing setup

Tools

The most important tools that are used in the automated tests are as follows:

Name	Description
xUnit v3	The testing framework. You can use any testing framework that you like though
NSubstitute	Library for mocking. Any mocking library will work. This project doesn't do extensive mocking, but for example `IPublishedValueFallback` is a mandatory parameter for any published content item, even if you don't actually use it. It's just convenient to insert a mock.
Test Containers	Automatically creates docker containers while running tests. It is used to create an empty SQL Server database that is automatically cleaned up after testing.

…

View on GitHub

This repository is meant to be used as a reference to get started with automated testing in an Umbraco project. Using practical examples, this repository demonstrates how you can cover various parts of your Umbraco application with automated tests.

What can I find in the repository?

The repository contains three projects:

A small example website
A unit testing project
An integration testing project

In the unit testing project, you'll find examples that demonstrate how to write tests for code with IPublishedContent. In the integration testing project, you'll find examples that demonstrate how to run your whole Umbraco website in-memory using a real SQL Server database. Each testing project comes with a README that explains how the various kinds of tests work and how to use the custom testing utilities.

"It doesn't cover my specific use-case!"

I intend to cover as much of Umbraco as possible, but I'm definitely not there yet! If there is a use-case that you wish to see covered, you can start a discussion! It would be amazing if we could turn this repository into the ultimate Umbraco testing reference! If you have any questions about automated testing in Umbraco, you may ask them in the discussions as well, but don't forget that there is a big Umbraco forum where many more people are able to help out! I occasionally look there as well.

"I have an idea how to test X in Umbraco!"

I welcome additions to this repository! Any examples on how to test with specific Umbraco features are welcome! Please start by making an "issue" in the issues section to suggest your addition. That allows me to understand what you're proposing and gives others a chance to share ideas as well. It also gives me a chance to point out if it has already been implemented before and I might spare you some time.

Can you publish it as a package?

I'm not going to make a package out of this project. I don't have time to maintain packages. Additionally, I find it important that everybody learns how testing in Umbraco really works. I think if this project turns into a package, its contents turn into "magic" and I don't stand for that. Furthermore, you don't have to wait until I update to the latest Umbraco versions, just to continue testing.

That being said: I don't mind accommodating a bit if somebody else wants to wrap this into a package for their own team / company / personal use. So if there is something I can do to make the project more "packable", don't hesitate to make a suggestion!

I'm still working on a few things

For example: I posted about testing your backoffice controllers in an integration test before, but it isn't included in the repo yet! I believe that there is a better way to test backoffice controllers nowadays and I'm still figuring that out.

That's all

I find great importance in automated testing and I see it as the best way to make great software. I hope that this project inspires all of you to get started as well if you haven't yet. Thank you for reading and I'll see you in my next blog!

Seamlessly integrate strongly-typed primitives into your Umbraco apis

Dennis — Mon, 14 Jul 2025 09:00:00 +0000

Strongly-typed primitives are an essential part of Domain Driven Design. By giving descriptive types to primitives, you can effectively communicate the meaning of primitive values. Additionally, strongly-typed primitives give you compile-time guards against misuse of primitive types.

A strongly-typed primitive might look like this:

public readonly record struct UserID(Guid Value);

This clearly communicates that the Guid value is the unique identifier of a user. Other example use-cases for strongly-typed primitives are:

Emailaddresses
Stock Keeping Units
Version numbers
URLs

Strongly-typed primitives come with some unique challenges: The dotnet ecosystem cannot tell that your custom type is actually a primitive. You will need to configure your solution so that it knows how to work with your custom types. Doing this wrong can turn strongly-typed primitives into a big chore. In this blog I will demonstrate how I universally integrated strongly-typed primitives into Umbraco.

Example: messageboard api

For illustration, I'm going to use a simple messageboard api. This api allows one to post a message and then fetch it with a user-friendly URL or by ID. The complete sample project is posted on github:

D-Inventor / umbraco-strongly-typed-primitive-example

An example Umbraco 16 website that demonstrates how to work with strongly typed primitives

The focus is on the MessageEntity, which contains a strongly-typed ID, a title, a user-friendly slug and a strongly-typed reference to the author. It looks like this:

MessageEntity.cs

public class MessageEntity
{
    private MessageEntity()
    {
    }

    public MessageID Id { get; set; }

    public required Slug Slug { get; init; }

    public required string Title { get; init; }

    public PersonID AuthorId { get; init; }

    public static MessageEntity Create(string title, Slug slug, PersonID authorId)
        => new() { Title = title, Slug = slug, AuthorId = authorId };
}

public readonly record struct MessageID(Guid Value);
public readonly record struct PersonID(Guid Value);
public readonly record struct Slug
{
    public Slug(string value)
    {
        ArgumentException.ThrowIfNullOrWhiteSpace(value);
        if (value.Any(char.IsUpper)) throw new ArgumentException("A slug cannot contain uppercase characters", nameof(value));
        if (value.StartsWith('-') || value.EndsWith('-')) throw new ArgumentException("A slug cannot start or end with a dash", nameof(value));

        Value = value;
    }

    public string Value { get; }
}

The strongly-typed primitives are easy to recognize as they wrap around a single primitive-type, like Guid and string or int.

The api controller looks like this:

MessageApiController.cs

[Route("/api/[controller]")]
[ApiController]
public class MessageApiController(MessageRepository messageRepository) : ControllerBase
{
    private readonly MessageRepository _messageRepository = messageRepository;

    [HttpPost]
    [ProducesResponseType<CreateMessageResponse>(StatusCodes.Status201Created)]
    public IActionResult Create([FromBody] CreateMessageRequest request)
    {
        var message = MessageEntity.Create(request.Title, request.Slug, request.AuthorId);
        _messageRepository.Save(message);

        var apiUrl = new Uri(Url.ActionLink(nameof(GetById), values: new { messageId = message.Id })!);
        var friendlyUrl = new Uri(Url.ActionLink(nameof(GetBySlug), values: new { slug = message.Slug })!);
        return Created(apiUrl, new CreateMessageResponse(message.Id, friendlyUrl));
    }

    [HttpGet("{messageId:guid}")]
    [Produces<GetMessageResponse>]
    [ProducesResponseType(StatusCodes.Status404NotFound)]
    public IActionResult GetById([FromRoute] MessageID messageId)
    {
        var message = _messageRepository.Find(messageId);

        return message is not null ? Ok(new GetMessageResponse(message.Id, message.Title, message.AuthorId))
            : NotFound();
    }

    [HttpGet("{slug}")]
    [Produces<GetMessageResponse>]
    [ProducesResponseType(StatusCodes.Status404NotFound)]
    public IActionResult GetBySlug([FromRoute] Slug slug)
    {
        var message = _messageRepository.Find(slug);

        return message is not null ? Ok(new GetMessageResponse(message.Id, message.Title, message.AuthorId))
            : NotFound();
    }
}

These API endpoints are documented with a swagger UI:

Problem #1: Strongly-typed primitives are seen as "complex objects" by default

Since strongly-typed primitives are classes and structs, the dotnet ecosystem treats them just like any other class or struct in the language. Before we can fix anything, we need to address this first. Modern dotnet allows us to specify how a strongly-typed primitive is constructed and deconstructed with an interface:

public interface IStronglyTypedPrimitive<TSelf, TPrimitive>
    where TSelf : IStronglyTypedPrimitive<TSelf, TPrimitive>
{
    static abstract TSelf Wrap(TPrimitive value);
    static abstract TPrimitive Unwrap(TSelf value);
}

Each strongly-typed primitive now needs to implement this interface. Additionally, each strongly-typed primitive should override the ToString() method:

public readonly record struct MessageID(Guid Value) : IStronglyTypedPrimitive<MessageID, Guid>
{
    public static Guid Unwrap(MessageID value) => value.Value;

    public static MessageID Wrap(Guid value) => new(value);

    public override string ToString() => Value.ToString();
}

public readonly record struct PersonID(Guid Value) : IStronglyTypedPrimitive<PersonID, Guid>
{
    public static Guid Unwrap(PersonID value) => value.Value;

    public static PersonID Wrap(Guid value) => new(value);

    public override string ToString() => Value.ToString();
}

public readonly record struct Slug : IStronglyTypedPrimitive<Slug, string>
{
    public Slug(string value)
    {
        ArgumentException.ThrowIfNullOrWhiteSpace(value);
        if (value.Any(char.IsUpper)) throw new ArgumentException("A slug cannot contain uppercase characters", nameof(value));
        if (value.StartsWith('-') || value.EndsWith('-')) throw new ArgumentException("A slug cannot start or end with a dash", nameof(value));

        Value = value;
    }

    public string Value { get; }

    public static string Unwrap(Slug value) => value.Value;

    public static Slug Wrap(string value) => new(value);

    public override string ToString() => Value;
}

Problem #2: Json (de-)serialization

Strongly-typed primitives should serialize and deserialize in json like the primitive type that they wrap. You can see in the HTTP Post body, that this is not the case:

{
  "title": "Hello world",
  "slug": {
    "value": "hello-world"
  },
  "authorId": {
    "value": "3fa85f64-5717-4562-b3fc-2c963f66afa6"
  }
}

The strongly-typed primitives behave like objects, which is ugly and undesirable. We can fix this by implementing JsonSerializer. For example for Guid:

StronglyTypedGuidJsonConverter.cs

// 👇 We only allow strongly-typed primitives that implement our interface
public class StronglyTypedGuidJsonConverter<TValue> : JsonConverter<TValue>
    where TValue : IStronglyTypedPrimitive<TValue, Guid>
{
    // 👇 Because of the interface, we can call the static methods on the type to wrap and unwrap the primitive value and serialize it.
    public override TValue? Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
        => TValue.Wrap(reader.GetGuid());

    public override void Write(Utf8JsonWriter writer, TValue value, JsonSerializerOptions options)
        => writer.WriteStringValue(TValue.Unwrap(value));
}

In order to use this type, we need to create a json serializer factory that can dynamically create instances of the json converter for each concrete implementation:

StronglyTypedPrimitiveJsonConverterFactory.cs

internal class StronglyTypedPrimitiveJsonConverterFactory : JsonConverterFactory
{
    // 👇 These are all the primitive types that we support with their corresponding converters.
    // All implementations can be found in the GitHub repo.
    private readonly Dictionary<Type, Type> _converterTypeMap = new()
    {
        { typeof(int), typeof(StronglyTypedIntJsonConverter<>) },
        { typeof(string), typeof(StronglyTypedStringJsonConverter<>) },
        { typeof(Guid), typeof(StronglyTypedGuidJsonConverter<>) }
    };

    // 👇 This factory can convert any type that implements the `IStronglyTypedPrimitive` interface.
    // Additionally, a converter must exist for the concrete primitive type
    public override bool CanConvert(Type typeToConvert)
    {
        var @interface = GetStronglyTypedPrimitiveInterface(typeToConvert);
        return @interface != null && _converterTypeMap.ContainsKey(GetPrimitiveType(@interface));
    }

    public override JsonConverter? CreateConverter(Type typeToConvert, JsonSerializerOptions options)
    {
        var @interface = GetStronglyTypedPrimitiveInterface(typeToConvert);
        if (@interface is null) return null;

        if (!_converterTypeMap.TryGetValue(GetPrimitiveType(@interface), out var converterType)) return null;

        // 👇 Using reflection, a concrete JsonConverter type is constructed.
        // Based on the example in the documentation: https://learn.microsoft.com/en-us/dotnet/standard/serialization/system-text-json/converters-how-to#sample-factory-pattern-converter
        var completeType = converterType.MakeGenericType(typeToConvert);
        var converter = Activator.CreateInstance(
            completeType,
            BindingFlags.Instance | BindingFlags.Public,
            binder: null,
            args: [],
            culture: null) as JsonConverter;

        return converter;
    }

    private static Type? GetStronglyTypedPrimitiveInterface(Type typeToConvert)
        => typeToConvert
        .GetInterfaces()
        .SingleOrDefault(@interface => @interface.IsGenericType && @interface.GetGenericTypeDefinition() == typeof(IStronglyTypedPrimitive<,>));

    private static Type GetPrimitiveType(Type @interface)
        => @interface.GetGenericArguments()[1];
}

We register this factory with the options pattern and dependency injection:

ConfigureStronglyTypedPrimitiveJsonOptions.cs

public class ConfigureStronglyTypedPrimitiveJsonOptions : IConfigureOptions<JsonOptions>
{
    public void Configure(JsonOptions options)
    {
        options.JsonSerializerOptions.Converters.Insert(0, new StronglyTypedPrimitiveJsonConverterFactory());
    }
}

Program.cs

// Umbraco's ootb configuration
builder.CreateUmbracoBuilder()
    .AddBackOffice()
    .AddWebsite()
    .AddComposers()
    .Build();

// 👇 Register options in DI
builder.Services.ConfigureOptions<ConfigureStronglyTypedPrimitiveJsonOptions>();

Now we can call the create endpoint with this body:

{
  "title": "Hello world",
  "slug": "hello-world",
  "authorId": "3fa85f64-5717-4562-b3fc-2c963f66afa6"
}

Problem #3: Modelbinding

Modelbinding is what you use to bind route parameters and query parameters to arguments on your action method in your controller. In our case, the two GET endpoints use modelbinding to either bind a MessageID or a Slug. If we attempt to call the api-endpoint with a valid slug or id, we get an error:

System.InvalidOperationException: Could not create an instance of type 'MessageID'. Model bound complex types must not be abstract or value types and must have a parameterless constructor. Record types must have a single primary constructor. Alternatively, give the 'messageId' parameter a non-null default value.

In order to fix this, we implement IModelBinderProvider:

StronglyTypedPrimitiveModelBinderProvider.cs

public class StronglyTypedPrimitiveModelBinderProvider : IModelBinderProvider
{
    public IModelBinder? GetBinder(ModelBinderProviderContext context)
    {
        // 👇 We dynamically create modelbinders for each concrete type that implements IStronglyTypedPrimitive.
        // The implementation depends on the primitive type
        ArgumentNullException.ThrowIfNull(context);

        var modelType = context.Metadata.ModelType;
        var @interface = GetStronglyTypedPrimitiveInterface(modelType);
        if (@interface is null) return null;

        var primitiveType = GetPrimitiveType(@interface);
        if (primitiveType == typeof(string)) return CreateStringPrimitiveModelBinder(modelType);
        if (PrimitiveIsParsable(primitiveType)) return CreateParsablePrimitiveModelBinder(modelType, primitiveType);

        return null;
    }

    private static BinderTypeModelBinder CreateParsablePrimitiveModelBinder(Type modelType, Type primitiveType)
    {
        var modelBinderType = typeof(ParsablePrimitiveModelBinder<,>).MakeGenericType(modelType, primitiveType);
        return new BinderTypeModelBinder(modelBinderType);
    }

    private static BinderTypeModelBinder CreateStringPrimitiveModelBinder(Type modelType)
    {
        var modelBinderType = typeof(StringPrimitiveModelBinder<>).MakeGenericType(modelType);
        return new BinderTypeModelBinder(modelBinderType);
    }

    // 👇 Every dotnet primitive type implements the `IParsable<>` interface
    // That interface allows us to universally parse strings into primitive types
    private static bool PrimitiveIsParsable(Type primitiveType)
        => primitiveType
        .GetInterfaces()
        .Any(@interface => @interface.IsGenericType && @interface.GetGenericTypeDefinition() == typeof(IParsable<>));

    private static Type? GetStronglyTypedPrimitiveInterface(Type typeToConvert)
        => typeToConvert
        .GetInterfaces()
        .SingleOrDefault(@interface => @interface.IsGenericType && @interface.GetGenericTypeDefinition() == typeof(IStronglyTypedPrimitive<,>));

    private static Type GetPrimitiveType(Type @interface)
        => @interface.GetGenericArguments()[1];
}

The implementations for StringPrimitiveModelBinder<> and ParsablePrimitiveModelBinder<,> can be found in the repo on GitHub.

The model binder is registered with the options pattern in DI:

ConfigureStronglyTypedPrimitiveModelBinding.cs

public class ConfigureStronglyTypedPrimitiveModelBinding : IConfigureOptions<MvcOptions>
{
    public void Configure(MvcOptions options)
    {
        options.ModelBinderProviders.Insert(0, new StronglyTypedPrimitiveModelBinderProvider());
    }
}

Program.cs

builder.CreateUmbracoBuilder()
    .AddBackOffice()
    .AddWebsite()
    .AddComposers()
    .Build();

builder.Services.ConfigureOptions<ConfigureStronglyTypedPrimitiveJsonOptions>();
// 👇
builder.Services.ConfigureOptions<ConfigureStronglyTypedPrimitiveModelBinding>();

Problem #4: Open API docs

Although the API works as intended, our changes are not automatically picked up by the swagger documentation.

We need to configure the swagger documentation so that it respects our strongly-typed primitives and presents them correctly. We apply the same tricks that we've done before, but now in a ISchemaFilter:

StronglyTypedPrimitiveSchemaFilter.cs

public class StronglyTypedPrimitiveSchemaFilter : ISchemaFilter
{
    private static readonly Dictionary<Type, (string type, string? format)> _primitiveTypeMap = new()
    {
        [typeof(int)] = ("integer", null),
        [typeof(string)] = ("string", null),
        [typeof(Guid)] = ("string", "uuid")
    };

    public void Apply(OpenApiSchema schema, SchemaFilterContext context)
    {
        var @interface = GetStronglyTypedPrimitiveInterface(context.Type);
        if (@interface is null || !_primitiveTypeMap.TryGetValue(GetPrimitiveType(@interface), out var schemaType)) return;

        // 👇 Some of these fields are set by Umbraco's configuration, some of them are just filled by default.
        // Anything that doesn't look like a primitive gets reset to default values
        // Additionally, the type and format are mapped to the primitive types
        schema.Properties.Clear();
        schema.Required.Clear();
        schema.OneOf.Clear();
        schema.AdditionalPropertiesAllowed = true;
        schema.Type = schemaType.type;
        schema.Format = schemaType.format;
    }

    private static Type? GetStronglyTypedPrimitiveInterface(Type type)
        => type
        .GetInterfaces()
        .SingleOrDefault(@interface => @interface.IsGenericType && @interface.GetGenericTypeDefinition() == typeof(IStronglyTypedPrimitive<,>));

    private static Type GetPrimitiveType(Type @interface)
        => @interface.GetGenericArguments()[1];
}

Once again, we register this type with the options pattern in DI:

ConfigureStronglyTypedPrimitiveOptions.cs

public class ConfigureStronglyTypedPrimitiveOptions : IConfigureOptions<SwaggerGenOptions>
{
    public void Configure(SwaggerGenOptions options)
    {
        options.SchemaFilter<StronglyTypedPrimitiveSchemaFilter>();
    }
}

Program.cs

builder.CreateUmbracoBuilder()
    .AddBackOffice()
    .AddWebsite()
    .AddComposers()
    .Build();

builder.Services.ConfigureOptions<ConfigureStronglyTypedPrimitiveJsonOptions>();
builder.Services.ConfigureOptions<ConfigureStronglyTypedPrimitiveModelBinding>();
// 👇
builder.Services.ConfigureOptions<ConfigureStronglyTypedPrimitiveOptions>();

When we return to the swagger UI, we see that it looks clean and correct:

Final thoughts

I like the idea of strongly-typed primitives and I've already seen them work in practice. That being said: It has been quite a challenge to figure out how to properly integrate them into the edges of a software system. By the length of this blog, you can see that it takes a serious amount of effort to begin working with them. Although, now that I've figured it out, it would probably not take thát long anymore to repeat it.

I left a few things out of this blog for the sake of simplicity. One thing that I noticed for example, is that the user-friendly endpoint actually returns HTTP 500 when you don't format your slug properly. I'll leave that as a little exercise for you to fix.

I hope you learned something new today and I'll see you in my next blog! 😊

How to test Umbraco against a real database

Dennis — Thu, 17 Apr 2025 10:26:48 +0000

In my last post about Test Driven Development, I briefly touched on testing with a real database and how I used TestContainers to do it.

Dennis

Apr 7 '25

Learning TDD by doing: Umbraco & EF Core

#webdev #tutorial #testing #umbraco

5 min read

In this post I'm going to explain step-by-step how to run some simple tests against an Umbraco project with a real database:

Things you need before you get started
Setup before writing tests
Integration test with EF Core
Integration test with an Umbraco site running in-memory

Things you need before you get started

The first thing you need is docker. If you're using windows like I do, you can install Docker Desktop for windows. We use docker to create new databases on-demand for our tests.

You also need:

A working Umbraco project
A unit test project

The code examples in this tutorial are based on xUnit, but the concepts apply to any testing framework that you might use.

Setup before writing tests

We start by installing the necessary packages into the test project with the following command:

dotnet add package Testcontainers.MsSql
dotnet add package Testcontainers.Xunit

The first thing we need, before we write any tests, is a utility to easily consume test databases. XUnit makes this very easy. We will start with a class that represents our resource:

SqlServerDatabase.cs

public class SqlServerDatabase : IAsyncLifetime
{
    // 👇 This defines a SqlServer container in docker
    private readonly MsSqlContainer _dbContainer = new MsSqlBuilder()
            .WithImage("mcr.microsoft.com/mssql/server:2022-latest")
            .WithEnvironment("ACCEPT_EULA", "Y")
            .Build();

    private string? _connectionString;

    // 👇 We expose a public connection string property that we can insert into a DbContext or even a whole Umbraco website
    public string ConnectionString => _connectionString ?? throw new InvalidOperationException("Cannot get the connection string, because the database hasn't been initialised yet.");

    public async Task DisposeAsync()
    {
        await _dbContainer.StopAsync();
        await _dbContainer.DisposeAsync();
    }

    public async Task InitializeAsync()
    {
        await _dbContainer.StartAsync();

        // The default connection string from the test container connects to the 'master' database
        // EF Core cannot properly reset the database after each test if we connect to the 'master' database.
        // This connection string connects to a 'different' database on which ef core can do everything that it needs 
        _connectionString = new SqlConnectionStringBuilder(_dbContainer.GetConnectionString())
        {
            InitialCatalog = Guid.NewGuid().ToString("D")
        }.ToString();
    }
}

How you use this class depends on the kind of test that you write. We'll start with a test for Entity Framework Core.

Integration test with EF Core

When writing tests for EF Core, we need the ability to create instances of our DbContext on-demand. The examples here assume the existance of a context class named BlogContext.

In order to create instances of BlogContext, we first need to get an instance of SqlServerDatabase: the type that we created in the previous chapter. We do that with a collection fixture in xUnit:

SqlServerDatabaseFixture.cs

[CollectionDefinition(SqlServerDatabaseFixture.Definition)]
public sealed class SqlServerDatabaseFixture
    : ICollectionFixture<SqlServerDatabase>
{
    public const string Definition = "SQL Server database";
}

In our test, we can request an instance of the sql server database like this:

MyBlogContextTests.cs

[Collection(SqlServerDatabaseFixture.Definition)]
public class MyBlogContextTests(SqlServerDatabase database)
{
    private readonly SqlServerDatabase _database = database;
}

Since we're going to create instances of BlogContext quite frequently, it may be convenient to add some small utility extensions to make our lives a little easier:

SqlServerBlogContextExtensions.cs

public static class SqlServerBlogContextExtensions
{
    public static BlogContext CreateBlogContext(this SqlServerDatabase database)
    {
        var dbOptions = new DbContextOptionsBuilder<BlogContext>()
            .UseSqlServer(database.ConnectionString)
            .Options;
        return new BlogContext(dbOptions);
    }

    public static async Task EnsureCleanStateAsync(this BlogContext context, CancellationToken cancellationToken = default)
    {
        await context.Database.EnsureDeletedAsync(cancellationToken);
        await context.Database.EnsureCreatedAsync(cancellationToken);
    }
}

Now we can create our first test. For example: I should be able to post a comment on a blogpost:

MyBlogContextTests.cs

[Collection(SqlServerDatabaseFixture.Definition)]
public class MyBlogContextTests(SqlServerDatabase database)
    : IAsyncLifetime
{
    private readonly SqlServerDatabase _database = database;

    public async Task InitializeAsync()
    {
        // Before each test, the database must be in a consistent state so that our tests are stable
        using var context = _database.CreateBlogContext();
        await context.EnsureCleanStateAsync();
    }

    public Task DisposeAsync() => Task.CompletedTask;

    [Fact]
    public async Task ShouldPostCommentOnBlog()
    {
        // given
        int blogId;
        using (var context = _database.CreateBlogContext())
        {
            var blog = Blog.CreateNew(title: "Test blog", userId: 2);
            await context.AddAsync(blog);
            await context.SaveChangesAsync();
            blogId = blog.Id;
        }

        // when
        using (var context = _database.CreateBlogContext())
        {
            await new BlogCommentRequestHandler().PostCommentOnBlog(
                context: context,
                blogId: blogId,
                comment: "Test comment",
                userId: 5);
        }

        // then
        using (var context = _database.CreateBlogContext())
        {
            var result = await context.Blogs.Include(blog => blog.Comments).FirstOrDefaultAsync(blog => blog.Id == blogId);
            Assert.Equal(1, result.Comments.Count);
        }
    }
}

If you keep an eye on Docker Desktop while the test is running, you'll see that a new container with SQL Server is created and then deleted after your test completes. It may take several minutes on the first try: the SQL Server container image needs to be downloaded from the internet!

ℹ️ Using multiple contexts in a single test
It's wise to create separate contexts for the Arrange, Act and Assert steps in a test. This is because a DbContext caches models in memory. If you use a single DbContext for the whole test, you might not actually be fetching the data from the database.

With a test in place, we can implement our feature:

BlogCommentRequestHandler.cs

public class BlogCommentRequestHandler
{
    public async Task PostCommentOnBlog(BlogContext context, int blogId, string comment, int userId, CancellationToken cancellationToken = default)
    {
        var blog = await context.blogs.FindAsync([blogId], cancellationToken);
        var user = await context.users.FindAsync([userId], cancellationToken);

        blog.PostComment(comment, user);
        await context.SaveChangesAsync(cancellationToken);
    }
}

Integration test with an Umbraco site running in-memory

The setup for running your Umbraco site with a sql server with test containers is largely the same as what I've written in my first blog about integration testing with an Umbraco site. For this blog, I'm going to skip over the details of running your site in-memory and will only focus on connecting a site to the testcontainers SQL Server database.

Much like the referenced blog, this example will assume that a class MyCustomWebApplicationFactory exists, which extends WebApplicationFactory<Program>. We create a new resource that is a composition of the database and the website:

SqlServerWebsite.cs

public class SqlServerWebsite
    : IAsyncLifetime
{
    public SqlServerWebsite()
    {
        Database = new SqlServerDatabase();
        Website = new MyCustomWebApplicationFactory(_database.ConnectionString);
    }

    public SqlServerDatabase Database { get; }
    public MyCustomWebApplicationFactory Website { get; }

    public async Task InitializeAsync()
    {
        await Database.InitializeAsync();
        await Website.InitializeAsync();
    }

    public async Task DisposeAsync()
    {
        await Website.DisposeAsync();
        await Database.DisposeAsync();
    }
}

We need to make a slight modification to the web application factory so that it can consume our new connection string:

MyCustomWebApplicationFactory.cs

public class MyCustomWebApplicationFactory(string connectionString)
    : WebApplicationFactory<Program>, IAsyncLifetime
{
    protected override void ConfigureWebHost(IWebHostBuilder builder)
    {
        builder.ConfigureAppConfiguration(conf =>
        {
            conf.AddInMemoryCollection([
                new KeyValuePair<string, string?>("ConnectionStrings:umbracoDbDSN", connectionString),
                new KeyValuePair<string, string?>("ConnectionStrings:umbracoDbDSN_ProviderName", "Microsoft.Data.SqlClient")
            ]);
        });
    }
}

We create a collection fixture for this new resource:

SqlServerWebsiteFixture.cs

[CollectionDefinition(SqlServerWebsiteFixture.Definition)]
public sealed class SqlServerWebsiteFixture
    : ICollectionFixture<SqlServerWebsite>
{
    public const string Definition = "SQL Server website";
}

And now we're ready to consume our website as a resource. For example: I should be able to visit the homepage:

HomepageIntegrationTests.cs

[Collection(SqlServerWebsiteFixture.Definition)]
public class HomepageIntegrationTests(SqlServerWebsite website)
{
    private readonly SqlServerWebsite _website;
    private MyCustomWebApplicationFactory Website => _website.Website;

    [Fact]
    public async Task ShouldShowHomepageOnRootUrl()
    {
        var response = await Website.CreateClient().GetAsync("/");
        Assert.Equal(HttpStatusCode.Ok, response.StatusCode);
    }
}

Final thoughts

TestContainers has turned out to be incredibly easy to set up for an Umbraco website and for various testing purposes. It reliably creates and destroys databases. It has really changed the way I look at testing and it has opened up a lot of possibilities for me.

I hope this tutorial has been helpful to you. See you in my next blog! 😊

Learning TDD by doing: Umbraco & EF Core

Dennis — Mon, 07 Apr 2025 09:00:00 +0000

Remember: 🔴 Red, 🟢 Green, ♻️ Refactor

Umbraco has support for EF Core since Umbraco 12. EF Core is a convenient tool to work with custom database tables. I happened to be in need of that, so I took a dive into EF Core and Umbraco to see if I could use them with Test Driven Development.

Exploring EF Core & Umbraco
My approach and why it was a mistake
What would I recommend now?

Exploring EF Core & Umbraco

Umbraco's documentation on entity framework core is a good place to start. It shows you just enough to get some working example code. However, Umbraco doesn't really give any recommendations beyond the bare minimum. I had some questions still, after reading through the documentation:

How does Umbraco expect me to test with DbContext?
Where does Umbraco expect me to create this "scope"? In the outermost layer in a controller for example, or in the innermost layer inside my business logic?
What is Umbraco's expectation from me for passing the DbContext to different parts of my application? Through method parameters or exclusively with dependency injection?

Although I don't want Umbraco to tell me how to use EF Core, I would appreciate some best practices on Umbraco's wrapper types: the IScopeProvider and IScopeAccessor.

My approach and why it was a mistake

I based my inspiration on the example from their documentation:

[HttpGet("all")]
public async Task<IActionResult> All()
{
    using IEfCoreScope<BlogContext> scope = _efCoreScopeProvider.CreateScope();
    IEnumerable<BlogComment> comments = await scope.ExecuteWithContextAsync(async db => db.BlogComments.ToArray());
    scope.Complete();
    return Ok(comments);
}

At the time that I started, I saw two options to use this with Test Driven Development:

Mock IEfCoreScopeProvider and IEfCoreScope
Abstract Umbraco's types along with the DbContext behind a different interface.

I chose to abstract everything behind an interface because the thought of mocking Umbraco's scope provider and scope scared me and an abstraction seemed to be easier to test. Umbraco's scopes felt like they added a lot of overhead over time as well: I didn't really want to create scopes in all places where I needed to use the DbContext. In the end I regret this choice and I'll explain why:

In my domain, I manage "topics". It started easy: I need to fetch a topic by its ID. I do not want to depend on my context directly, so I create an abstraction:

public interface ITopicStore
{
    Task<TopicDTO?> GetAsync(int id);
}

A test could easily implement ITopicStore with an in-memory list and it would pretty much work just like that.

But the trouble starts brewing quickly:

I want to also fetch the topic by a slug

public interface ITopicStore
{
    Task<TopicDTO?> GetAsync(int id);
    Task<TopicDTO?> GetAsync(string slug);
}

I need to show a summary of the topic in a related place

public interface ITopicStore
{
    Task<TopicDTO?> GetAsync(int id);
    Task<TopicDTO?> GetAsync(string slug);
    Task<TopicSummary?> GetSummaryAsync(int id);
}

I need to get a list of topics that is paginated

public interface ITopicStore
{
    Task<TopicDTO?> GetAsync(int id);
    Task<TopicDTO?> GetAsync(string slug);
    Task<TopicSummary?> GetSummaryAsync(int id);
    Task<IEnumerable<TopicSummary>> GetSummaryPaginatedAsync(int skip, int take);
}

And final example: I need to get all topics that the current user is following

public interface ITopicStore
{
    Task<TopicDTO?> GetAsync(int id);
    Task<TopicDTO?> GetAsync(string slug);
    Task<TopicSummary?> GetSummaryAsync(int id);
    Task<IEnumerable<TopicSummary>> GetSummaryPaginatedAsync(int skip, int take);
    Task<IEnumerable<TopicSummary>> GetSummaryWhereUserFollows(int userid);
}

The point is: I need to ask the database many different questions and if I completely abstract the DbContext behind an interface, I end up with a massive interface that is incredibly unreadable (At the time of writing, my interface has 18 different methods!!).

What would I recommend now?

In hindsight, I would actually say that Umbraco's example code is the best place to start and then extract your business logic into a separate object. I'm personally quite fond of pairing up controllers with "request handlers". That is: Controllers are responsible for validating input and sending output, while a complementing request handler executes the logic of a request. In short:

Do not abstract DbContext behind an interface
Do not depend on IEfCoreScopeProvider in your business logic. Push your dependency on Umbraco to the edge of your system, keep it in your controller for example.
Use a real database. TestContainers works very well.

Test driven development becomes quite easy with this setup. It allows you to build and test your code as if Umbraco's wrapping layer doesn't exist. Given a "request handler" class, you might produce a test like this for example:

public class BlogRequestHandlerTests
{
    // ... necessary set up here to create instances of `DbContext`

    [Fact]
    public async Task ShouldFindCommentsByBlogId()
    {
        // given
        var sut = new BlogDetailPageRequestHandler();
        var blogWithComments = CreateBlog().WithComment("Test comment");
        using (var context = new BlogContext(this.ContextOptions))
        {
            await context.Blogs.AddAsync(blogWithComments);
            await context.SaveChangesAsync();
        }

        // when
        IEnumerable<BlogComment>? result = null;
        using (var context = new BlogContext(this.ContextOptions))
        {
            // 👇 Notice that I'm passing the DbContext as a parameter to the method
            result = await sut.GetCommentsByBlogIdAsync(context, blogWithComments.Id);
        }

        // then
        Assert.NotNull(result);
        Assert.Single(result);
    }
}

And the implementation to make the test pass:

public class BlogDetailPageRequestHandler
{
    public Task<IEnumerable<BlogComment>> GetCommentsByBlogIdAsync(BlogContext context, int blogId, CancellationToken cancellationToken = default)
    {
        return context.Blogs
            .AsNoTracking()
            .Where(blog => blog.Id == blogId)
            .SelectMany(blog => blog.Comments)
            .ToListAsync(cancellationToken);
    }
}

These tests are easy to write and still acceptable to read. I also have control over the lifetime of the DbContext throughout my test.

Once your business logic is done, connecting this to a controller is as simple as copy/pasting Umbraco's example code:

public class BlogDetailPageController(
    BlogDetailPageRequestHandler requestHandler,
    IEfCoreScopeProvider<BlogContext> efCoreScopeProvider)
    : RenderController
{
    public async Task<IActionResult> BlogDetailPageAsync(CancellationToken cancellationToken)
    {
        // 👇 For simplicity of the example, I only produce the "comments" in the request handler 
        // In reality, I would produce the entire `BlogDetailPageViewModel` inside the request handler
        using IEfCoreScope<BlogContext> scope = efCoreScopeProvider.CreateScope();
        var comments = await scope.ExecuteWithContextAsync(context => requestHandler.GetCommentsByBlogIdAsync(context, CurrentPage.BlogId, cancellationToken));
        scope.Complete();
        return CurrentTemplate(new BlogDetailPageViewModel(CurrentPage, comments));
    }
}

Final words

It was an interesting experience. I'm sad that I hadn't heard of TestContainers earlier. If I had, I would've approached this way differently as opposed to what I have done now. I have learned a lot from this and I feel like a better developer now, which also counts for something.

I hope you found this interesting to read, I hope you learned something from my mistake and maybe you'll stick around for my next blog! 😊