Follow me on Twitter, happy to take your suggestions on topics or improvements /Chris
Testing is something we need to do to increase the confidence in what we are building. After all, we want to ship working software. We do know that bugs happen so we need to be disciplined and preferably add a test for each bug so we at least know we fixed that one. It's a game of whack a mole and we need to keep up. There are different ways of doing tests though, unit tests, integration tests, E2E tests. In this article, we will focus on unit tests and ensure we adopt some good practices.
TLDR; This is a primer on testing in .Net Core. If you are completely new to testing or testing in .Net Core, then this is for you. I will follow up with a more advanced article on testing discussing object mothers, mocking and other things.
In this article, we will cover
- Creating test projects and running your tests
- Different types of testing, here we will mention different types of testing if you are completely new to testing and need a primer on the different phases and levels.
- Authoring tests, here we will go through some good practices for naming and arranging your tests.
.Net Core supports testing using MsTest, xUnit as well as nUnit so you have quite a lot of choices. For this article, we have selected MsTest. Check the reference section for links to the other frameworks.
References
xUnit testing
This page describes how to use xUnit with .Net CorenUnit testing
This page describes how to use nUnit with .Net Core.dotnet test, terminal command description
This page describes the terminal commanddotnet testand all the different arguments you can call it with.dotnet selective test
This page describes how to do selective testing and how to set up filters and query using filters.
WHY
We mentioned at the beginning of the article that testing is important especially in the context of bugs. I would argue and say that testing is the most important tool in your toolbox as a developer. Learning to check your work will not only build a great reputation with your clients but also with your colleagues. The better you are at testing your code and find various ways to do so, the better for everybody.
Different types of testing
There are different types of testing. They are used at different stages of the software's life cycle. You also write tests at different levels to test details or large scale behavior. We usually say that unit testing is meant to test implementation details where integration testing is used to test if two or more, larger pieces work together. We should test at all levels and life cycles.
Test last
We are at a point where we've written the whole software and the so-called happy path seems to work. With happy path we mean the scenario we think the client will use to carry out a task or whatever else is running our software, it could be other software :). In this scenario, we realize that we might get in trouble if something unexpected happens so we start adding tests at this point and logging to ensure that our software covers some scenarios that we think might happen. Adding tests at this point often feels like a chore and frankly not fun. But we must add them to call ourselves professionals.
Regression testing
Well, I would say that testing is something that should always be with you, a mentality of how can this break. Is it the type of input, is memory consumption, is there a dependency that answers with something we didn't expect?
You can't think of everything of course, even though you try and this deliberate analysis before, during and after you've written the code will make the code better for sure. Because you can't think of everything, you will have bugs. A good practice here is trying to write a test that proves that the bug exists in the first place. Once established you can go on to fix the bug and see your test pass. At least you've cured that symptom/bug.
The above isn't really regression testing but rather a good practice when you find a bug. Regression testing is more when you do a change, to add a feature or fix a bug and you rerun some tests and ensure everything is still working. It's tightly connected and a term you should know.
Refactoring
Of course, there's another case refactoring. Most codebases I've seen turns into an untangled mess over time as more and more features are added to it or the features themselves change. What was once an easy to understand piece of code becomes a ball of spaghetti. At this point, you feel like starting from scratch, most likely. Having tests is a great way to feel confident that you can change code and it still does whats it's supposed to once you've stopped improving it through refactoring.
Test driven development, test first
This is a methodology in which you think out tests and write them before you actually start writing any production code. The idea is that you start with a failing test and then write production code and the test passes. This is called red-green testing. Red for failing test and Green for passing test. Some people like this way of working and others feel constrained by it. On the positive side, you don't write code that you won't need as all the code you write are meant to make a certain test scenario pass.
Integration testing
This is usually some high-level testing where we test if different components, large parts of the system, work together. It's usually a certain layer talking to another layer, e.g an API layer talking to a data layer for example or even a separate component talking to another component that together makes our a large complex system.
WHAT - creating a demo
In this article, we aim to test at a lower level, the unit testing level. We will show how you can easily create a test project, author tests and run them. Hopefully, we will also give some great guidance on authoring and how to improve the tests. We will carry out the following:
- Scaffold a test project
- Author a test
- Run the test
- Improving our test
Scaffold a test project
Essentially we want to create a solution with application code as one project and test code in a separate project. So the overall solution will be this:
solution
app // project containing our implementation
app-test // project containing our tests
Creating the solution
Let's start by creating a directory. This hold our solution. The directory can be called anything you like but we call ours TestExample so we do:
mkdir TestExample
Next we will create a solution by calling:
cd TestExample
dotnet new sln
Create the library project
Thereafter we will create a library project that will hold our production code:
dotnet new classlib -o app
Next we want to add the project reference to the solution, like so:
dotnet sln add app/app.csproj
Creating the test project
Thereafter we want to create a test project like so:
dotnet new mstest -o app-test
It should produce an output looking something like so:
and add a reference to our solution like so:
dotnet sln add app-test/app-test.csproj
Lastly, we want to add a reference to the app project in the app-test project so we can refer to code in the app that we want to test. To do so type the following commands:
cd app-test
dotnet add reference ../app/app.csproj
cd ..
Writing a test
Let's create a file CalculatorTest.cs and give it the following content:
// CalculatorTest.cs
using Microsoft.VisualStudio.TestTools.UnitTesting;
namespace app_test
{
[TestClass]
public class CalculatorTest
{
[TestMethod]
public void Add()
{
Assert.AreEqual(1,1);
}
}
}
The decorator TestClass means our class CalculatorTest should be treated as a test class. TestMethod is a decorator we add to our Add() method to signal that this is a test method.
Inside the method bofy of Add() we add an assertion statement Assert.AreEqual(1,1). The prototype of AreEqual() is AreEqual(expected, actual). This is all we need to create a test.
Running a test
Now, let's run it. We do that by either:
- A terminal command
This command dotnet test comes with a lot of arguments so make sure to check out the documentation page
- Click
Run Testin VS Code
As you can see above we can easily debug a test as well and if we look at the class level there will be a link to debug/run all tests in the class.
We have a thing called filter that we can use to run specific tests that we target. Let's add another test like so:
[TestMethod]
[Priority(1)]
public void Subtract()
{
Assert.AreEqual(1,2);
}
Above we have not only added test method Subtract() but we have also added a decorator Priority(1). We can filter so we only run tests matching this name by typing the following:
dotnet test --filter Priority=1
As you can see above it's only running 1 test Subtract(), it's skipping over our Add() test.
There's another decorator we could be adding, TestCategory. Let's add the following two tests:
[TestMethod]
[TestCategory("DivideAndMultiply")]
public void Divide()
{
Assert.AreEqual(1,1);
}
[TestMethod]
[TestCategory("DivideAndMultiply")]
public void Multiply()
{
Assert.AreEqual(1, 1);
}
The two tests have the same tags DivideAndMultiply. We can filter on that by typing dotnet test --filter TestCategory=DivideAndMultiply. The TestCategory allows us to create more descriptive tags than the Priority one.
A more real scenario
Ok, we have a test class CalculatorTest but let's face it how many times in production are we coding a calculator? Yea I didn't think so. Most likely we are doing something like an e-commerce company. So let's dream up what that could look like. Let's talk about orders and how to evaluate them.
That's easy I got an order and a number of items and then just loop through them and I got a total and then I add some shipping cost and BOOM I'm done :)
I wish it was that easy. When you are starting out maybe, but then someone mentions the word discount and sure having one discount on 20% is easy to calculate but suddenly it grows into a monster. Before you know it you got a discount on different product types, 3 for 2 discounts, holiday discounts, discounts on all items that are tagged in a certain way and so on and so forth. Trust me on this Discounts soon becomes it's own out of control spaghetti monster that takes a team to manage and you can imagine what the code looks like trying to keep up with that.
Starting out
Let's start small though. Imagine you've just started a WebShop and you need to write some code to support calculating the sum of an order. We start by creating the file Order.cs in our app project and give it the following content:
// Order.cs
using System;
using System.Collections.Generic;
namespace OrderSystem {
public enum ProductType
{
CD,
DVD,
Book,
Clothes,
Game
}
public class Product
{
public string Name { get; set; }
public string Description { get; set; }
public ProductType Type { get; set; }
}
public class OrderItem
{
public int Quantity { get; set; }
public double Price { get; set; }
public Product Product { get; set; }
}
public class Order
{
public List<OrderItem> Items { get; set; }
public DateTime Created { get; set; }
}
}
Then we create a file OrderHelper.cs that will help us calculate the sum of an order and related Order logic. For now, we give it the content:
namespace OrderSystem
{
public class OrderHelper
{
public OrderHelper()
{}
public static double Cost(Order order)
{
return 0;
}
}
}
Ok, that doesn't do much, that's the point, cause now we will write a test.
Let's create a file OrderHelperTest.cs in our app-test project and give it the following content:
using System.Collections.Generic;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using OrderSystem;
namespace app_test
{
[TestClass]
public class OrderHelperTest
{
[TestMethod]
public void ShouldSumCostOfOrder()
{
// Arrange
// Act
// Assert
}
}
}
Good practice when authoring
Now let's talk about some good practices to use when authoring a test. Tests should be easy to read. The test name should say something about what we are doing and what we hope of the outcome. Giving it the name ShouldSumCostOfOrder() tells us what's going on. You could be even more specific and say something about the outcome e.g ShouldSumOrderTo30(). Ultimately it's what makes the most sense to you and what you are testing.
Next thing to look at is the three As, Arrange, Act and Assert. In arrange you should set up everything you need. In Act you should call your production code. In the final Assert step, you should verify you got the outcome you expected.
As you can see we have a method ShouldSumCost().
Writing the test
Ok, we understand a bit more on naming and what steps to have in a test so let's flesh out the test a bit. The first thing we need to focus on is setting up our test, the so-called Arrange phase, with the following code:
// Arrange
var productCD = new Product()
{
Type = ProductType.CD,
Name = "Nirvana",
Description = "Album"
};
var productMovie = new Product()
{
Type = ProductType.DVD,
Name = "Gladiator",
Description = "Movie"
};
var order = new Order()
{
Items = new List<OrderItem>() {
new OrderItem(){
Quantity = 1,
Price = 10,
Product = productCD
},
new OrderItem(){
Quantity = 2,
Price = 10,
Product = productMovie
}
}
};
we could easily move these to another class to make the test easier to read. For now though let's keep going and move to the next phase Act.
// Act
var actual = OrderHelper.Cost(order);
Note the naming of the variable actual. It's good practice to use names like actual and expected when writing tests as these are terms other developers know and it will make it easier to read your code.
Finally we have the Assert phase where we check whether the code does what we think it does:
// Assert
Assert.AreEqual(30, actual);
The full file OrderHelperTest.cs should look like this:
// OrderHelperTest.cs
using System.Collections.Generic;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using OrderSystem;
namespace app_test
{
[TestClass]
public class OrderHelperTest
{
[TestMethod]
public void ShouldSumCostOfOrder()
{
// Arrange
var productCD = new Product()
{
Type = ProductType.CD,
Name = "Nirvana",
Description = "Album"
};
var productMovie = new Product()
{
Type = ProductType.DVD,
Name = "Gladiator",
Description = "Movie"
};
var order = new Order()
{
Items = new List<OrderItem>() {
new OrderItem(){
Quantity = 1,
Price = 10,
Product = productCD
},
new OrderItem(){
Quantity = 2,
Price = 10,
Product = productMovie
}
}
};
// Act
var actual = OrderHelper.Cost(order);
// Assert
Assert.AreEqual(30, actual);
}
}
}
At this point our test will fail
Adding code
Ok, so we need to fix the test by adding a real implementation to OrderHelper.cs. Change the code to the following:
using System.Linq;
namespace OrderSystem
{
public class OrderHelper
{
public static double Cost(Order order)
{
return order.Items.Sum(i => i.Price * i.Quantity);
}
}
}
Rerunning the test we get:
Useful functionality
Ok, we've taken you through so-called Red-Green testing where you start out with writing a test, ensures it fails, it turns Red. Then we write the implementation. Finally, we rerun the test and it passes, it turns Green.
Many times we end up writing tests that tests the same flow through code, we just need to vary the input to make sure our production code really really works.
Let's take our CalculatorTest.cs file as an example. First, let's give it a real implementation Calculator.cs in the app project. Now give it the following content:
// Calculator.cs
namespace app
{
public class Calculator
{
public static int Add(int lhs, int rhs)
{
return 0;
}
}
}
Update our CalculatorTest.cs to now say:
// CalculatorTest.cs
using app;
using Microsoft.VisualStudio.TestTools.UnitTesting;
namespace app_test
{
[TestClass]
public class CalculatorTest
{
[TestMethod]
public void Add()
{
var actual = Calculator.Add(0,0);
Assert.AreEqual(actual,0);
}
}
}
Running the test at this point means it passes. Now, can we trust our code at this point to work? I mean the test passes right? Because the implementation is so small we can tell that it WON'T work if we change the input. With larger implementations it might be harder to tell.
Let's try to find an approach!
Change your code to the following:
// CalculatorTest.cs
using app;
using Microsoft.VisualStudio.TestTools.UnitTesting;
namespace app_test
{
[TestClass]
public class CalculatorTest
{
[DataTestMethod]
[DataRow(0)]
public void Add(int value)
{
var actual = Calculator.Add(value, value);
var expected = value + value
Assert.AreEqual(actual, expected);
}
}
}
Note how we remove TestMethod and replace it with DataTestMethod. Also, note how we add DataRow which takes an argument. Now, this argument becomes the input parameter value. Running the test right now means it calculates 0 + 0 and thereby our test should still pass. Let's add some more scenarios though so the code looks like this:
using System;
using app;
using Microsoft.VisualStudio.TestTools.UnitTesting;
namespace app_test
{
[TestClass]
public class CalculatorTest
{
[DataTestMethod]
[DataRow(0)]
[DataRow(1)]
[DataRow(2)]
public void Add(int value)
{
Console.WriteLine("Testing {0} + {0}", value);
var actual = Calculator.Add(value, value);
var expected = value + value;
Assert.AreEqual(actual,expected);
}
}
}
Running the test now we get:
From the above we can see that 0 + 0 passes whereas 1 + 1 and 2 + 2 fails. Our code don't work so we need to fix it. Let's change Calculator.cs to the following:
namespace app
{
public class Calculator
{
public static int Add(int lhs, int rhs)
{
return lhs + rhs;
}
}
}
Now if we run our tests we get:
What we just did is called data-driven testing. Let's stop here, the article is long enough already
Summary
We've taken you through creating a test project, authoring some tests and learn to run our test. Additionally, we have discussed some good practices in naming and how to improve your tests even further with data-driven testing.
I hope this was useful and that you are looking forward to the next article that aims to go more in-depth.
Top comments (1)
A good tool that I want to see in the next article about this topic is Coverlet