State versus Behaviour Verification

When reasoning about types of automated tests, I find it quite useful to reason about two different categories, namely solitary and sociable tests. Also see the test pyramid for a more detailed explanation.

Focusing on solitary tests, there are generally two different styles that are being used:

• Solitary tests that perform state verification.
• Solitary tests that perform behaviour verification.

State verification

Applying state verification means that we first exercise the Subject Under Test by calling one or more methods on an object. Then we use assertions to verify the state of the object. We may also verify any results returned by the method.

This is also known as the Detroit School of TDD. This nickname comes from its origins out of Extreme Programming, a well known development methodology used by Chrysler’s C3 project at the end of the 1990s. Kent Beck’s book Test-Driven Development By Example best describes this approach.

Let’s have a look at an example of state verification.

public class Resume
{
    private readonly IList<Experience> _experiences;
    public IEnumerable<Experience> Experiences => _experiences;

    public Resume()
    {
        _experiences = new List<Experience>();
    }

    public void AddExperience(string employer, string role, DateTime from, DateTime until)
    {            
        var newExperience = new Experience(employer, role, from, until);
        _experiences.Add(newExperience);
    }
}

public class Experience
{
    public string Employer { get; }
    public string Role { get; }
    public DateTime From { get; }
    public DateTime Until { get; }

    public Experience(string employer, string role, DateTime from, DateTime until)
    {
        Employer = employer;
        Role = role;
        From = from;
        Until = until;
    }
}

Here we have a system for managing online résumés. A user can add one or more experiences to a résumé. Therefore we have a Resume class with a method AddExperience. This method adds an Experience object to a collection.

[TestFixture]
public class When_adding_experience_to_a_resume
{
    [Test]
    public void Then_the_specified_experience_should_now_appear_on_the_resume()
    {
        var experienceFrom = new DateTime(2014, 09, 12);
        var experienceUntil = new DateTime(2017, 12, 31);

        var resume = new Resume();
        resume.AddExperience("Google", "Data analyst", experienceFrom, experienceUntil);

        var addedExperience = resume.Experiences.SingleOrDefault();
        Assert.That(addedExperience, Is.Not.Null);
        Assert.That(addedExperience.Employer, Is.EqualTo("Google"));
        Assert.That(addedExperience.Role, Is.EqualTo("Data analyst"));
        Assert.That(addedExperience.From, Is.EqualTo(experienceFrom));
        Assert.That(addedExperience.Until, Is.EqualTo(experienceUntil));
    }
}

Looking at the implementation of the corresponding unit test, we see that a Resume object is created and that the AddExperience method is being called. Then we verify whether a correct Experience object has been added to the collection of experiences. With this test we verify the new state of the Resume object.

State verification tests do not instrument the Subject Under Test to verify its interactions with other parts of the system. We only inspect the observable state of an object and the direct outputs of its methods. This approach tests the least possible implementation detail. It has the most notable advantage that these tests will continue to pass even if the internals of the SUT’s methods are changed without altering their observable behaviour.

There are also two slightly different styles of state verification, namely Procedural State Verification and Object State Verification. We’ll cover these two styles more in-depth in a later blog post.

Behaviour verification

Verifying the behaviour of the Subject Under Test implies the ability for instrumenting and verifying its interactions with other objects or other parts of the system. This is mostly done by using test doubles, either written manually or by using a framework.

This is also known as the London School of TDD. This nickname comes from the practices applied by the Extreme Programming community in London. The concepts of this process is most clearly described by the book Growing Object Oriented Software Guided By Tests, written by Steve Freeman and Nat Pryce, also known as the GOOS book.

Let’s have a look at an example of behaviour verification.

public class RegisterUserHandler
{
    private readonly IUserRepository _userRepository;
    private readonly IEmailSender _emailSender;

    public RegisterUserHandler(
        IUserRepository userRepository,
        IEmailSender emailSender)
    {
        _userRepository = userRepository;
        _emailSender = emailSender;
    }

    public void Handle(RegisterUser command)
    {
        var user = new User(command.Email);
        _userRepository.Save(user);

        var emailMessage = new EmailMessage(user.Email, "Confirm email", "...");
        _emailSender.Send(emailMessage);
    }
}

public class RegisterUser
{
    public string Email { get; }

    public RegisterUser(string email)
    {
        Email = email;
    }
}

Here we have a system that manages user registration. We have a class named RegisterUserHandler that creates and saves a new user. Also a confirmation email is sent to verify the existence of the specified email address.

[TestFixture]
public class When_registering_a_new_user
{
    [Test]
    public void Then_the_new_user_should_be_registered_in_the_system()
    {
        var userRepository = Substitute.For<IUserRepository>();
        var emailSender = Substitute.For<IEmailSender>();

        var sut = new RegisterUserHandler(userRepository, emailSender);

        var command = new RegisterUser("[e-mail]");
        sut.Handle(command);

        userRepository.Received().Save(Arg.Any<User>());
    }

    [Test]
    public void Then_a_confirmation_email_should_be_sent()
    {
        var userRepository = Substitute.For<IUserRepository>();
        var emailSender = Substitute.For<IEmailSender>();

        var sut = new RegisterUserHandler(userRepository, emailSender);

        var command = new RegisterUser("[e-mail]");
        sut.Handle(command);

        emailSender.Received().Send(Arg.Any<EmailMessage>()); 
    }
}

Looking at the implementation of the unit tests, notice that we’re using the NSubstitute mocking library to verify that the collaborators of the RegisterUserHandler class are being called.

We pass those test doubles to the RegisterUserHandler class using constructor injection. For more information on Dependency Injection, you can check out the excellent book Dependency Injection by Mark Seemann. With these tests we verify the behaviour of the RegisterUserHandler class.

Applying this approach is a bit more complicated and harder to reason about than using state verification. These tests are also more brittle as they imply that the test code has intimate knowledge about the implementation details of the Subject Under Test. Changing the implementation details without changing the overall functionality will more likely result in failing tests.

There are a few techniques and design principles that can somewhat reduce this brittleness. The most important one is to strive for the least amount of collaborators. This way interactions can be verified effectively without sacrificing maintainability.

From this explanation, we might come to the conclusion that only applying state verification is the best approach for writing tests. And to a certain degree this is true. We should favour state verification over behaviour verification most of the time. But in practice, we actually need both kinds of verifications. Not every object in our system has the same role or responsibility. Some are more algorithmic and involves logic based code, while others involve more interactions between objects.

As you might have guessed, state verification is most useful for verifying algorithms and business logic, like domain objects, validators, static functions like helper or extension methods, etc. … Behaviour verification is most useful for verifying interactions, like services, controllers, gateways, etc. …

Being aware about state and behaviour verification is the first step to learn about how to write maintainable unit tests.