DEV Community: Jan Van Ryswyck

Tales Of TDD: One Test Double To Rule Them All

Jan Van Ryswyck — Wed, 11 Mar 2026 08:05:43 +0000

So I’ve encountered this specific test case, which is quite elaborate.

What do you mean by that?

Well, it contains a lot of code, and also looks more difficult to understand compared to the other test cases.

Yeah, that’s because of the Fake we’re using in our test code.

So, the test code is complex because we’re using a Fake?

Yes, I’m afraid so. The test scenario embodies a specific edge case. The Subject Under Test should initiate a recovery only when the collaborator raises an InvalidValueException.

That does align with the functionality we discussed earlier.

I had to do all this setup in the test method just to force the Fake to throw an InvalidValueException.

I see. I agree that it’s sometimes not easy to make a Fake behave exactly as you want.

Tell me about it. It took me almost an hour to figure out how to make it throw an exception. I really like using Fakes in our tests because they behave just as the real implementation. But sometimes, they can be a real pain to work with as well.

Have you considered using a Stub instead of a Fake?

A Stub? No, not really. We decided in a team meeting to use Fakes instead of Stubs, Spies, or Mocks

I understand. I’m not suggesting you stop using Fakes altogether. But have you considered not using a Fake just for this specific test case? Would it make the test easier to understand if you used a Stub that raises an InvalidValueException instead?

Yeah, but that’s not what we agreed on in the team meeting.

I’m sure what was agreed on was more of a guideline than a strict rule. How about we just try it out? Let’s change the test code to use a Stub instead of a Fake.

OK. We can do that.

… After short while …

Now the test is just four lines of code. What do you think?

Yeah, it’s a lot more concise and readable.

Do you want to keep it like this?

Sure! But shouldn’t we change all the other tests to use a Stub for that collaborator as well?

Why do you want to do that?

Because that way, it’s more consistent.

If consistency is what we’re after, then I agree with you. But what do we ultimately want from our tests?

That they fail when we have an issue in the production code.

Yes, and what else?

That they are readable and easy to understand.

Exactly! There’s no reason to be dogmatic about test doubles. We can just use the one that best fits our purpose. So it’s perfectly fine to use Fakes for most of the tests, while using a Stub for this specific test case as a one-off.

I see that now. So I don’t need to stick with one specific type of test double? That’s very cool. I can’t wait to show this off at our next team meeting.

Perspectives On Software Quality

Jan Van Ryswyck — Tue, 03 Feb 2026 20:16:22 +0000

The quality of software, or any product, can be viewed from different perspectives. This is because there are usually multiple stakeholders, each bringing their own unique views and interests to the table.

Some of these perspectives include:

The quality of a product as experienced by customers.
The quality of a product as perceived by the user, which can be highly subjective and personal.
The quality of the process used to build the product, which is particularly important to members of a software team.

A common issue in software organisations is the tendency to prioritize one specific perspective over other quality perspectives. For example, when customer or user quality is considered more important than process quality. However, this is a fallacy. Although these quality perspectives are often represented by different people, they are inextricably intertwined and each cannot stand on its own.

Focusing solely on customer or user quality while neglecting process quality can lead to a deteriorating codebase, making it difficult or impossible to add new features. Conversely, prioritizing only process quality, while neglecting customer or user quality, may result in a technically impressive product that fails to attract users and ultimately serves no purpose.

Organisations that empower their software teams to invest equally in all quality perspectives will outperform all others who choose not to apply this balance.

The Five Underplayed Premises Of TDD

Jan Van Ryswyck — Fri, 08 Aug 2025 17:42:05 +0000

Today I got reminded of a great video of GeePaw Hill about the “Five Underplayed Premises Of TDD”.

1. The money premise:

When writing tests, we don’t do it for the art of it or other moral reasons. We do TDD because it lets us build more features faster.

2. The judgement premise:

There’s no one-size-fits all approach to quality software. Developers applying TDD use human judgement while building `software systems.

3. The chain premise:

We prefer testing very tiny subsystems that make up our larger system. Why? Because small tests are not only easier to read, comprehend and write, but also execute a lot faster than large-scale end-to-end tests. This is a preference, not a hard rule.

4. The correlation premise:

The internal quality of the code for a system is directly correlated to the productivity of the developers working in this code base. They go up together, and they go down together.

5. The driving premise:

Tests and testability help drive the design of a system. A code base that is highly testable exposes certain characteristics that are generally considered as good design. People new to TDD often resist the idea of changing their design to facilitate tests. However, this is inevitable when doing TDD. This how this small practice drives the design.

These 5 premises are at the foundation of every step a developer using TDD makes. Check out GeePaw’s video if you want to learn more. The best 10 minutes you’ll probably spend this week. I promise 😄

Contract Tests - Parameterised Test Cases

Jan Van Ryswyck — Wed, 28 Jun 2023 09:10:38 +0000

This is the final installment of a three-part series about contract tests. In the first blog post we’ve discussed the rationale behind contract tests. Next we’ve looked at how to implement contract tests using Abstract Test Cases. In this blog post, we’re going to look into an alternative approach to Abstract Test Cases by using parameterised tests instead.

With this alternative approach, we no longer rely on inheritance, where concrete classes are derived from an abstract base class that contains the test cases. Instead, Sociable tests are added to a regular test class, just as we would normally do. We apply the “Abstract Factory” design pattern to create the respective Subject Under Test, either the real implementation or the fake implementation. Then we use this factory to execute every parameterised test case for each subject that the factory instantiates.

Let’s have a look at an example to demonstrate this approach. Although we’ve used Java in our example, the same pattern can be implemented in a similar way by using other object-oriented languages like C#, Python, Ruby, etc. …

We’re going to implement the same example as we’ve used in the previous blog post. The Subject Under Test is still a repository for storing and retrieving employee data to and from a database.

We start out by defining an interface.

interface EmployeeRepositoryStrategy extends AutoCloseable {
    EmployeeRepository getSubjectUnderTest();
}

This interface defines a method for creating the Subject Under Test, which in our example is an instance of an EmployeeRepository. This interface also extends the AutoCloseable interface which provides a close method. This method can be used to perform some cleanup.

The following piece of code shows the implementation of the SQLiteEmployeeRepositoryStrategy, which implements the EmployeeRepositoryStrategy interface.

static class SQLiteEmployeeRepositoryStrategy implements EmployeeRepositoryStrategy {

    private final NamedParameterJdbcTemplate jdbcTemplate;
    private final SQLiteEmployeeRepository sqliteEmployeeRepository;

    public SQLiteEmployeeRepositoryStrategy() {
        var database = getClass().getClassLoader().getResource("database.db");
        var connectionUrl = String.format("jdbc:sqlite:%s", database);

        var sqliteDataSource = new SQLiteDataSource();
        sqliteDataSource.setUrl(connectionUrl);

        this.jdbcTemplate = new NamedParameterJdbcTemplate(sqliteDataSource);
        this.sqliteEmployeeRepository = new SQLiteEmployeeRepository(jdbcTemplate);
    }

    @Override
    public EmployeeRepository getSubjectUnderTest() {
        return sqliteEmployeeRepository;
    }

    @Override
    public void close() {
        jdbcTemplate.getJdbcOperations().execute("DELETE FROM Employee");
    }
}

The constructor initialises an instance of the SQLiteEmployeeRepository. This instance is returned by the implementation of the getSubjectUnderTest method. The close method simply removes all records from the Employee table in the database.

The following piece of code shows the implementation of the FakeEmployeeRepositoryStrategy, which also implements the EmployeeRepositoryStrategy interface.

static class FakeEmployeeRepositoryStrategy implements EmployeeRepositoryStrategy {

    private final FakeEmployeeRepository fakeEmployeeRepository;

    public FakeEmployeeRepositoryStrategy() {
        fakeEmployeeRepository = new FakeEmployeeRepository();
    }

    @Override
    public EmployeeRepository getSubjectUnderTest() {
        return fakeEmployeeRepository;
    }

    @Override
    public void close() {
        fakeEmployeeRepository.clear();
    }
}

The constructor initialises an instance of the FakeEmployeeRepository. This instance is also returned by the implementation of the getSubjectUnderTest method. The close method removes all the data from the repository by calling the clear method.

The contract tests for the EmployeeRepository now look like this:

public class EmployeeRepositoryTests {

    @ParameterizedTest
    @ArgumentsSource(EmployeeRepositoryStrategyProvider.class)
    public void Should_return_nothing_for_a_non_existing_employee(EmployeeRepositoryStrategy strategy) {

        var unknownId = UUID.fromString("753350fb-d9a2-4e4b-8ca4-c969ca54ef5f");

        var SUT = strategy.getSubjectUnderTest();
        var retrievedEmployee = SUT.get(unknownId);

        assertThat(retrievedEmployee).isNull();
    }

    @ParameterizedTest
    @ArgumentsSource(EmployeeRepositoryStrategyProvider.class)
    public void Should_return_employee_for_identifier(EmployeeRepositoryStrategy strategy) {

        var SUT = strategy.getSubjectUnderTest();

        var employee = new Employee(
                UUID.fromString("13e420a7-3bfd-4c6b-adde-d673c6ee1469"),
                "Dwight", "Schrute",
                LocalDate.of(1966, 1, 20));
        SUT.save(employee);

        var retrievedEmployee = SUT.get(UUID.fromString("13e420a7-3bfd-4c6b-adde-d673c6ee1469"));
        assertThat(retrievedEmployee).usingRecursiveComparison().isEqualTo(employee);
    }

    @ParameterizedTest
    @ArgumentsSource(EmployeeRepositoryStrategyProvider.class)
    public void Should_save_employee(EmployeeRepositoryStrategy strategy) {

        var SUT = strategy.getSubjectUnderTest();

        var newEmployee = new Employee(
                UUID.fromString("55674e0b-4a1f-4cd1-be96-bcdc67fd4ded"),
                "Dwight", "Schrute",
                LocalDate.of(1966, 1, 20));
        SUT.save(newEmployee);

        var persistedEmployee = SUT.get(UUID.fromString("55674e0b-4a1f-4cd1-be96-bcdc67fd4ded"));
        assertThat(persistedEmployee).usingRecursiveComparison().isEqualTo(newEmployee);
    }
}

Here we have three parameterised tests. Each test receives a particular EmployeeRepositoryStrategy instance when it gets executed. The tests themselves interact with the SUT through this specified interface. After the test has been executed, the test runner will automatically call the close method as it recognises that the parameter implements the AutoCloseable interface.

Notice that we provide an instance of the EmployeeRepositoryStrategyProvider as an argument source. The implementation of this factory class looks like this:

static class EmployeeRepositoryStrategyProvider implements ArgumentsProvider {

    @Override
    public Stream<? extends Arguments> provideArguments(ExtensionContext extensionContext) {
        return Stream.of(
            Arguments.of(new SQLiteEmployeeRepositoryStrategy()),
            Arguments.of(new FakeEmployeeRepositoryStrategy())
        );
    }
}

The provideArguments method simply creates and returns an instance of the SQLiteEmployeeRepositoryStrategy class and the FakeEmployeeRepositoryStrategy respectively. Applying this provider as the parameter source ensures that each test is executed twice; once for the SQLiteEmployeeRepository and once for the FakeEmployeeRepository. The test runner will therefore execute six tests in total.

The advantage of using this approach is that we use composition over class inheritance as we no longer rely on subclasses. A disadvantage to this approach is that it’s slightly more complicated compared to Abstract Test Cases, which can be a debatable subject.

To conclude, I would like to thank Mario Pio Gioiosa for teaching me about the Parameterised Test Cases approach.

Contract Tests - Abstract Test Cases

Jan Van Ryswyck — Wed, 12 Apr 2023 13:12:33 +0000

In the previous blog post, we’ve discussed the rationale behind contract tests. They are used for exercising those parts of an application that communicate with other parts of the system by crossing the process boundary. In this blog post, we’re going to have a look at how to implement contract tests in practice. The most common approach you’ll likely encounter is Abstract Test Cases.

With this approach, we write Sociable tests as one normally would write them. However, instead of adding them to a regular test class we’ll add them to an abstract base class instead. Then we derive a subclass for each implementation. A subclass thereby inherits all the test cases from the base class. In essence, this is the “Template Method” design pattern in action, where each test becomes a template method. Concrete subclasses implement primitive operation methods for creating and interacting with their respective Subject Under Test, either the real implementation or the fake implementation.

Let’s have a look at an example to demonstrate this. Although we’ve used Java in our example, the same pattern can be implemented in a similar way by using other object-oriented languages like C#, Python, Ruby, etc. …

The following example shows a Subject Under Test that is a repository for storing and retrieving employee data to and from a database. The contract tests for the EmployeeRepository look like this:

abstract class EmployeeRepositoryTests {

    private EmployeeRepository SUT;

    @BeforeEach
    public void setUp() {
        SUT = getSubjectUnderTest();
    }

    @AfterEach
    public void tearDown() {
        cleanup();
    }

    @Test
    public void Should_return_nothing_for_non_existing_employee() {

        var unknownId = UUID.fromString("753350fb-d9a2-4e4b-8ca4-c969ca54ef5f");
        var retrievedEmployee = SUT.get(unknownId);
        assertThat(retrievedEmployee).isNull();
    }

    @Test
    public void Should_return_employee_for_identifier() {

        var employee = new Employee(
            UUID.fromString("13e420a7-3bfd-4c6b-adde-d673c6ee1469"),
            "Dwight", "Schrute",
            LocalDate.of(1966, 1, 20));
        SUT.save(employee);

        var retrievedEmployee = SUT.get(UUID.fromString("13e420a7-3bfd-4c6b-adde-d673c6ee1469"));
        assertThat(retrievedEmployee).usingRecursiveComparison().isEqualTo(employee);
    }

    @Test
    public void Should_save_employee() {

        var newEmployee = new Employee(
            UUID.fromString("55674e0b-4a1f-4cd1-be96-bcdc67fd4ded"),
            "Dwight", "Schrute",
            LocalDate.of(1966, 1, 20));
        SUT.save(newEmployee);

        var persistedEmployee = SUT.get(UUID.fromString("55674e0b-4a1f-4cd1-be96-bcdc67fd4ded"));
        assertThat(persistedEmployee).usingRecursiveComparison().isEqualTo(newEmployee);
    }

    abstract EmployeeRepository getSubjectUnderTest();
    abstract void cleanup();
}

Notice that the EmployeeRepositoryTests class is abstract. It also defines two abstract methods: getSubjectUnderTest and cleanup. These abstract methods are being called by the setUp and tearDown method respectively, which in turn are executed before and after each test. The tests themselves interact with the SUT through the EmployeeRepositoryinterface.

public interface EmployeeRepository {
    Employee get(UUID id);
    void save(Employee employee);
}

We’ve derived two concrete classes from EmployeeRepositoryTests, one for the real repository implementation (SQLiteEmployeeRepository) and one for the fake repository implementation (FakeEmployeeRepository). Both of these implement the EmployeeRepository interface.

The following piece of code shows the implementation of the SQLiteEmployeeRepositoryTests, which exercises the code of the SQLiteEmployeeRepository.

class SQLiteEmployeeRepositoryTests extends EmployeeRepositoryTests {

    private final NamedParameterJdbcTemplate jdbcTemplate;
    private final SQLiteEmployeeRepository sqliteEmployeeRepository;

    public SQLiteEmployeeRepositoryTests() {
        var database = getClass().getClassLoader().getResource("database.db");
        var connectionUrl = String.format("jdbc:sqlite:%s", database);

        var sqliteDataSource = new SQLiteDataSource();
        sqliteDataSource.setUrl(connectionUrl);

        this.jdbcTemplate = new NamedParameterJdbcTemplate(sqliteDataSource);
        this.sqliteEmployeeRepository = new SQLiteEmployeeRepository(jdbcTemplate);
    }

    @Override
    EmployeeRepository getSubjectUnderTest() {
        return sqliteEmployeeRepository;
    }

    @Override
    void cleanup() {
        jdbcTemplate.getJdbcOperations().execute("DELETE FROM Employee");
    }
}

The constructor initialises an instance of the SQLiteEmployeeRepository. This instance is returned by the implementation of the getSubjectUnderTest method. The cleanup method simply removes all records from the Employeetable in the database.

The implementation of the SQLiteEmployeeRepository itself looks like this:

public class SQLiteEmployeeRepository implements EmployeeRepository {

    private final NamedParameterJdbcTemplate jdbcTemplate;

    public SQLiteEmployeeRepository(NamedParameterJdbcTemplate jdbcTemplate) {
        this.jdbcTemplate = jdbcTemplate;
    }

    @Override
    public Employee get(UUID id) {
        var sql = "SELECT Id, FirstName, LastName, BirthDate " +
            "FROM Employee " +
            "WHERE Id = :id";

        var parameters = Map.of("id", id);

        try {
            return jdbcTemplate
                .queryForObject(sql, parameters, SQLiteEmployeeRepository::mapEmployeeFromResultSet);
        } catch (EmptyResultDataAccessException e) {
            return null;
        }
    }

    private static Employee mapEmployeeFromResultSet(ResultSet resultSet, int rowNumber) 
        throws SQLException {

        return new Employee(
            UUID.fromString(resultSet.getString("Id")),
            resultSet.getString("FirstName"),
            resultSet.getString("LastName"),
            LocalDate.parse(resultSet.getString("BirthDate"))
        );
    }

    @Override
    public void save(Employee employee) {
        var sql = "INSERT INTO Employee (Id, FirstName, LastName, BirthDate) " +
            "VALUES(:id, :firstName, :lastName, :birthDate)";

        var parameters = Map.of(
            "id", employee.getId(),
            "firstName", employee.getFirstName(),
            "lastName", employee.getLastName(),
            "birthDate", employee.getBirthDate());

        jdbcTemplate.update(sql, parameters);
    }
}

The following piece of code shows the implementation of the FakeEmployeeRepositoryTests, which exercises the code of the FakeEmployeeRepository.

public class FakeEmployeeRepositoryTests extends EmployeeRepositoryTests {

    private final FakeEmployeeRepository fakeEmployeeRepository;

    public FakeEmployeeRepositoryTests() {
        fakeEmployeeRepository = new FakeEmployeeRepository();
    }

    @Override
    EmployeeRepository getSubjectUnderTest() {
        return fakeEmployeeRepository;
    }

    @Override
    void cleanup() {
        fakeEmployeeRepository.clear();
    }
}

Again, the constructor initialises an instance of the FakeEmployeeRepository. This instance is also returned by the implementation of the getSubjectUnderTest method. The cleanup method removes all data from the repository by calling the clear method. This method, in turn, simply clears the internal employees map as shown by the code of the FakeEmployeeRepository.

public class FakeEmployeeRepository implements EmployeeRepository {

    private final Map<UUID, Employee> employees;

    public FakeEmployeeRepository() {
        employees = new HashMap<>();
    }

    @Override
    public Employee get(UUID id) {
        return employees.get(id);
    }

    @Override
    public void save(Employee employee) {
        employees.put(employee.getId(), employee);
    }

    public void clear() {
        employees.clear();
    }
}

That’s it. When running the EmployeeRepositoryTests, the test runner will execute six tests; three tests for the SQLiteEmployeeRepository and the same three tests for the FakeEmployeeRepository.

To conclude, Abstract Test Cases are a quick and easy way to get started with contract tests. However, as the Gang of Four already expressed in their well-known book Design Patterns, we should favour composition over class inheritance. Following this principle brings us to another approach for implementing contract tests, which we’re going to discuss in the next blog post.

Contract Tests

Jan Van Ryswyck — Wed, 01 Feb 2023 09:54:49 +0000

One of my favorite awesome talks of all time is Integrated Tests Are a Scam by J.B. Rainsberger. I must admit that the title has a clickbait vibe to it. However, I very much recommend watching the video.

So what’s the deal with integrated tests? Why should they be considered as a scam? First of all, let’s clarify what an integrated test actually is. An integrated test is a high-level test that exercises multiple different behaviours of the system all together in one go. At first glance this doesn’t sound like a huge problem. We need tests like that, don’t we? Well, it kind of depends on the audience of the integrated test.

Remember from my previous post about the Testing Quadrant, that there are two quadrants of tests that support the team:

Developer tests: this is the quadrant where the Test Pyramid resides.
Functional tests: these are the kind of tests, as described by Behaviour-Driven Development (BDD) or Acceptance Test-Driven Development (ATDD), that prove to business stakeholders that certain features are available in the system.

When it comes to using integrated “functional” tests for providing the necessary confidence to business stakeholders, there’s not much of a problem there. Such tests often do require different parts of the system working together to provide the requested capabilities.

However, an issue arises when software developers start using integrated “developer” tests for proving the correctness of the system. So why do such integrated tests become an issue in this situation? For all the same reasons as when having too much Sociable tests:

They can be grudgingly slow.
They are quite often very non-deterministic. This undermines trust and confidence.
They are overly verbose as they require a lot of code to set up the system under test. This requires a serious investment when it comes to time and effort.
They don’t provide useful feedback about the design of the system.
They are highly susceptible to cascading failures.

Therefore, my advice, ad nauseam, is to adhere to the Test Pyramid as it’s still a highly valuable model to think about when it comes to the developer tests quadrant.

What about integration tests? Are those a scam as well? It kind of depends on how you use them. An integration test, as opposed to an integrated test, only verifies whether the different components for a single path running through the system correctly work together. If a sociable test is used for that purpose alone, then things are just fine. However, troubles usually start appearing the moment software developers decide to start using Sociable tests, or even worse integrated tests, as the sole means to verify correctness of the system at the expense of Solitary tests.

“ Strong integration tests, consisting of collaboration tests (clients using test doubles in place of collaborating services) and contract tests (showing that service implementations correctly behave the way clients expect) can provide the same level of confidence as integrated tests at a lower total cost of maintenance.

— J.B. Rainsberger - Clearing Up the Integrated Tests Scam

What J.B. suggests here is the use of contract tests for exercising those parts of the application that cross the process boundary to communicate with other parts of the system like a database, a queue, the file system, etc. … The purpose of a contract test is to verify whether an adapter for a collaborating service correctly implements a particular interface. This implies that the same suite of tests verifies the outside-observable behaviour of both the real implementation and a fake implementation.

In the context of an hexagonal architecture, this suite of contract tests ensures that the fake adapter behaves exactly the same as the real adapter. We can then use this fake implementation for Sociable “integration” tests that verify whether the different components of the application correctly work together.

In the next post we’ll discuss how to approach contract tests from a practical point of view. In the meantime, you can check out the Integrated Tests Are a Scam blog series to learn more first-hand.

The Testing Quadrant

Jan Van Ryswyck — Wed, 15 Jun 2022 09:31:24 +0000

Back in 2003, Brian Marick wrote an excellent article series about agile testing. There he described the concept of the Testing Quadrant. To my recollection this used to be a well-know model that most software developers were familiar with. Although still very much relevant today, I’ve noticed over the years that the knowledge about the different types of testing as described by the Testing Quadrant somewhat got lost.

The four quadrants as shown in the diagram above each match a specific perspective of testing. These are:

At the bottom left: Developer tests. This is the quadrant where the test pyramid lives. These tests are typically automated.
At the top left: Functional tests. These are customer story tests as described by Behaviour-Driven Development (BDD) or Acceptance Test-Driven Development (ATDD). These tests are usually automated.
At the top right: Exploratory tests and usability tests. By definition these are manual tests.
At the bottom right: Performance tests, load tests, security tests, … etc. These tests are mostly semi-automated, semi-manual.

Now that we’ve named the four quadrants, let's have a look at their vertical and horizontal position in the quadrant.

The top two quadrants are concerned with tests that are obvious to business stakeholders. These tests should indicate whether the system actually provides a solution to a business problem. The bottom two quadrants on the other hand are meaningful for technical people.

“ We will write tests before we code, minute by minute. We will preserve these tests forever, and run them all together frequently. We will also derive tests from the customer’s perspective.

— Kent Beck - Extreme Programming Explained (1st Edition - Chapter 18)

The left two quadrants are focused on preventing defects before and during the coding process. This matches the quote from the opening paragraph of chapter 18 in Kent Beck’s excellent book Extreme Programming Explained (1st Edition). In essence, by writing tests before writing the code, we build in a certain quality into the system right from the start. These kinds of tests therefore prevent defects instead of merely finding them.

The right two quadrants are focused on evaluating for defects or finding missing features. Note that this kind of inspection is still necessary. These two quadrants merely reflect whether we have a low or high quality system.

Most software systems will involve tests from across all four quadrants. Also, testing shouldn’t be a single person’s

responsibility. The quality of a software system is in fact a team effort. Every team member should be involved in these activities related to the four quadrants. Please note that the Testing Quadrant doesn’t prescribe any order of tests, nor the importance of certain quadrants and definitely not the amount of tests for each quadrant.

“ All models are wrong, but some are useful.

— George Box

The Testing Quadrant is a useful model to help software teams plan their testing efforts, while also creating awareness that it’s generally a good idea to cover all four bases.

Tales Of TDD: The Big Refactoring

Jan Van Ryswyck — Wed, 02 Feb 2022 09:22:05 +0000

I’ve heard that you’ve been working on this new feature for the payroll system based on the newly voted government legislation?

Yes, I am.

How is it going? Seems like a major change.

Well, I’m glad you asked. I’ve refactored a large part of the domain based on some new insights I’ve had. This morning I’ve pushed all the code to the git repository. It took me almost two weeks to make the necessary changes.

Great!

However, I could use some help though.

Sure thing. How can I help?

You see, in order to refactor the code base I commented out most of the tests. I just hate it when they get in the way. Anyway, can you fix these tests? There shouldn’t be more than 350 of them. That way I’m able to start working on the next user story.

Uhm, … that’s … uhm … interesting.

What’s interesting?

A couple of things come to mind.

OK. Tell me.

For starters, you mentioned the term refactoring a couple of times. I’m interested in the process that you call refactoring.

A process?

Yes. How did you take on this endeavour?

Well, I did what I always do. I started out by changing a bunch of existing methods on several classes, adding and removing parameters, moving code around, etc. … I’ve also added a couple of new classes. Just the usual stuff.

How soon did you execute the tests?

After a couple of hours I wanted to compile the source code. Then I noticed that a bunch of the tests didn’t compile anymore. There were just too many, so I’ve put them in comments. You know, this bothers me the most about tests. They prevent me from refactoring the code. Maybe we should just get rid of these unit tests. We still have the acceptance tests, right?

So you didn’t run the tests?

No. They didn’t even compile.

Did you execute the acceptance tests?

Are you kidding me? They take more than two hours to run. I don’t have time for that. Besides, that’s why we pay the QA folks. It’s their job to make sure that the acceptance tests run smoothly. They’re also a mess. I won’t go anywhere near those.

In summary, you’ve been “restructuring” code for two weeks without executing any kind of automated tests?

I’m afraid so. I could have done a better job on that department, which reminds me to try running the application later on to see if it still works. But I’m curious, you just said “restructuring the code”. Isn’t that the same as refactoring?

Refactoring is a form of restructuring, but they aren’t necessarily the same thing. Refactoring is making small adjustments to the code without altering its observable behaviour. By adding up all these small changes you basically end up with a bigger change.

I don’t get it. That’s exactly what I’ve been doing. It’s not that I changed thousands of lines of code at once.

There are some very important aspects that I didn’t mention yet.

Ok. What are those aspects then?

The most important aspect of all when it comes to refactoring is to execute the tests after each and every change, no matter how small. That way you know that the system still works as expected. Whenever a test fails, you also know exactly where to find and fix the issue.

I’ve tried that once while doing a code kata with a bunch of people. It made me feel ridiculous. I mean, the code changes became so small that it was almost impossible to mess them up.

That’s what you want, right?

I don’t know. Maybe. But who has time for such things?

You just mentioned that you want to try running the application to see if most of the features still work, right?

Yes

What are you going to do when you encounter something that doesn’t work?

In that case, I’ll try to find the code for that broken feature, set out some breakpoints and start debugging to see what’s going on.

Do you have time for that?

Uhm, … I guess so.

More often than not, a debugging session can easily waste a lot of time. Alternatively, by enabling a tight feedback loop we keep ourselves out of such a mess.

Suppose that I only make small changes as you suggest, and I’m unable to find the issue whenever a test fails. Then I still have to debug the code right?

Another important aspect is to also commit the code changes after each cycle. That way you’ll be able to revert to the last good commit and start over, only now using even smaller steps.

So you’re saying that besides making only tiny changes, I have to commit those changes as well? Isn’t that a lot of overhead?

Not if you want to have a fast feedback loop in place.

Who ever said that I want to have a feedback loop?

Well, didn’t you just mention that you wanted to move on to the next user story?

Yes

Why’s that? You didn’t even finish your current user story.

I just want to work on something else. I’m kind of fed up with the current user story.

You could have moved on if you’d only made small refactorings, ensured that all the tests still passed and committed the code after each short cycle.

I don’t understand.

Well, if you’d rigorously followed this short feedback cycle that I’ve been describing, then the code would always be in a releasable state, wouldn’t it?

Yeah, maybe that’s true.

Be quick but don’t hurry.

You always say things like that. I still have one question though.

Shoot!

What if I don’t want to refactor a particular part of the code? What if I just want to replace it with some entirely new implementation? Surely I can’t fall back to having a short feedback cycle in that case, right?

When you want to make changes on a larger scale, you can use the “Branch by Abstraction” technique.

What’s that all about?

Going back to your specific case, you develop the new part of the domain just as you would implement it from scratch. This way you can flesh out the design of the new code using Test-Driven Development alongside the existing implementation. When this is finished, you can gradually replace the old implementation with the new implementation at the “seams”.

What’s a seam? I never heard about this in the context of software development before.

This is a term that Michael Feathers uses in his excellent book Working Effectively with Legacy Code. According to his definition, a seam is a place where you can alter behavior in your program without editing in that place. This involves that some kind of abstraction layer is in place that is used by the client code. This abstraction layer enables you to swap out the old implementation with the new implementation.

That way I can work on the new implementation using short feedback cycles. I’ll be able to commit the code without affecting the existing implementation, while also keeping the application in a deployable state.

Exactly.

I’ll have to try that some time for one of the next user stories.

Let me know how that works out.

I will. Now, about those 350 failing tests …

Implementing Approval Tests For PDF Document Generation

Jan Van Ryswyck — Wed, 15 Dec 2021 09:24:26 +0000

In the previous blog post, we discussed how to use Approval Tests for verifying generated PDF documents. In this blog post I’m going to show how to extend the Approval Test library for Java in order to support PDF documents. Let’s just dive right into the code.

The first thing that needs to happen is making a new implementation of the ApprovalApprover interface, which is provided by the Approval Test library. The following code demonstrates how this can be implemented.

public class PdfFileApprover implements ApprovalApprover {

    public static final Field<String> PDF_DIFF_OUTPUT_DIRECTORY =
        new Field<>("PdfDiffOutputDirectory", String.class);

    private final ApprovalNamer namer;
    private final ApprovalWriter writer;

    private final double allowedDiffInPercent;
    private final List<PageArea> excludedAreas;

    private File received;
    private final File approved;

    public PdfFileApprover(ApprovalWriter writer, PdfFileOptions options) {
        this.writer = writer;
        this.allowedDiffInPercent = options.getAllowedDiffInPercent();
        this.excludedAreas = options.getExcludedAreas();

        namer = options.getParent().forFile().getNamer();

        received = namer.getReceivedFile(writer.getFileExtensionWithDot());
        approved = namer.getApprovedFile(writer.getFileExtensionWithDot());
    }

    public VerifyResult approve()
    {
        received = writer.writeReceivedFile(received);
        return approvePdfFile(received, approved);
    }

    public void cleanUpAfterSuccess(ApprovalFailureReporter reporter)
    {
        received.delete();
        if(reporter instanceof ApprovalReporterWithCleanUp) {
            ((ApprovalReporterWithCleanUp) reporter)
                .cleanUp(received.getAbsolutePath(), approved.getAbsolutePath());
        }
    }

    public VerifyResult reportFailure(ApprovalFailureReporter reporter)
    {
        reporter.report(received.getAbsolutePath(), approved.getAbsolutePath());
        if (reporter instanceof ReporterWithApprovalPower)
        {
            ReporterWithApprovalPower reporterWithApprovalPower = (ReporterWithApprovalPower) reporter;
            return reporterWithApprovalPower.approveWhenReported();
        }

        return VerifyResult.FAILURE;
    }

    public void fail()
    {
        throw new Error(String.format("Failed Approval\n Approved:%s\n Received:%s", 
            approved.getAbsolutePath(), received.getAbsolutePath()));
    }

    private VerifyResult approvePdfFile(File received, File approved) {
        try {
            SimpleEnvironment environment = new SimpleEnvironment();
            environment.setAllowedDiffInPercent(this.allowedDiffInPercent);

            PdfComparator<CompareResultImpl> pdfComparator = new PdfComparator<>(approved, received)
                .withEnvironment(environment);

            excludedAreas.forEach(pdfComparator::withIgnore);

            CompareResultImpl comparisonResult = pdfComparator.compare();
            if(comparisonResult.isNotEqual()) {
                String outputFileName = determineDiffOutputFileName();
                comparisonResult.writeTo(outputFileName);
            }

            return VerifyResult.from(comparisonResult.isEqual());
        } catch (IOException e) {
            return VerifyResult.FAILURE;
        }
    }

    private String determineDiffOutputFileName() {
        var outputDirectory = PackageLevelSettings.getValueFor(PDF_DIFF_OUTPUT_DIRECTORY);
        if(null == outputDirectory) {
            outputDirectory = namer.getSourceFilePath();
        }

        return Path.of(outputDirectory, "diffOutput").toString();
    }
}

The PdfFileApprover class provides the core implementation for supporting PDF documents. Most of the code is quite similar to the FileApprover class of the Approval Test library. However, the approvePdfFile method is the most important part. This method expects two arguments; the received PDF file and the approved PDF file. The purpose of the approvePdfFile method is to compare both incoming PDF files. For doing the actual comparison we make use of the PDFCompare library.

When there’s a difference between these PDF files, we save the result of the comparison to a PDF file so that we can visually inspect the differences as well as instruct the framework to fail the test. Also notice that the PDFCompare library has the ability to exclude certain areas within a PDF file from the comparison as well as allowing a certain percentage of differences.

public class PdfApprovals {

    public static void verify(ByteArrayOutputStream outputStream) {
        verify(outputStream, PdfFileOptions.DEFAULT_ALLOWED_DIFF_IN_PERCENT, Collections.emptyList());
    }

    public static void verify(ByteArrayOutputStream outputStream, List<PageArea> excludedAreas) {
        verify(outputStream, PdfFileOptions.DEFAULT_ALLOWED_DIFF_IN_PERCENT, excludedAreas);
    }

    public static void verify(ByteArrayOutputStream outputStream, double allowedDiffInPercentage) {
        verify(outputStream, allowedDiffInPercentage, Collections.emptyList());
    }

    public static void verify(ByteArrayOutputStream outputStream, double allowedDiffInPercentage, 
                              List<PageArea> excludedAreas) {
        ByteArrayInputStream inputStream = new ByteArrayInputStream(outputStream.toByteArray());
        ApprovalBinaryFileWriter binaryFileWriter = new ApprovalBinaryFileWriter(inputStream, "pdf");

        PdfFileOptions options = new PdfFileOptions()
            .withAllowedDiffInPercent(allowedDiffInPercentage)
            .withExcludedAreas(excludedAreas);

        PdfApprovals.verify(binaryFileWriter, options);
    }

    private static void verify(ApprovalWriter writer, PdfFileOptions options) {
        PdfFileApprover pdfFileApprover = new PdfFileApprover(writer, options);
        Approvals.verify(pdfFileApprover, options.getParent());
    }
}

The PdfApprovals class provides a number of static helper methods that can be used by the tests themselves. These helper methods ultimately use the PdfFileApprover class to perform the actual comparison. A number of overloaded methods are available to provide the ability of excluding certain areas and/or tweaking the allowed percentage of differences when performing the comparison.

These can be used as follows:

PdfApprovals.verify(result);

...

PdfApprovals.verify(result, 0.18);

...

var excludedAreas = Arrays.asList(
    new PageArea(4, 4, 12, 18),
    new PageArea(35, 38, 42, 45)
);
PdfApprovals.verify(result, excludedAreas);

...

PdfApprovals.verify(result, 0.18, excludedAreas);

For completeness, the following code shows the implementation of the PdfFileOptions and PackageSettings classes. These are necessary to provide the PdfFileApprover class with the necessary configuration settings for performing the comparison and saving the output files.

public class PdfFileOptions {

    public static final double DEFAULT_ALLOWED_DIFF_IN_PERCENT = 0.001;

    private enum CustomFields {
        ALLOWED_DIFF_IN_PERCENT,
        EXCLUDED_AREAS
    }

    private final Map<CustomFields, Object> customFields;
    private final Options options;

    public PdfFileOptions() {

        customFields = new HashMap<>();
        customFields.put(CustomFields.ALLOWED_DIFF_IN_PERCENT, DEFAULT_ALLOWED_DIFF_IN_PERCENT);
        customFields.put(CustomFields.EXCLUDED_AREAS, Collections.emptyList());

        options = new Options();
        options.forFile().withExtension(".pdf");
    }

    public double getAllowedDiffInPercent() {
        return (double) customFields.get(CustomFields.ALLOWED_DIFF_IN_PERCENT);
    }

    public List<PageArea> getExcludedAreas() {

        return (List<PageArea>) customFields.get(CustomFields.EXCLUDED_AREAS);
    }

    public Options getParent() {
        return options;
    }

    public PdfFileOptions withAllowedDiffInPercent(double allowedDiffInPercent) {
        customFields.put(CustomFields.ALLOWED_DIFF_IN_PERCENT, allowedDiffInPercent);
        return this;
    }

    public PdfFileOptions withExcludedAreas(List<PageArea> excludedAreas) {
        customFields.put(CustomFields.EXCLUDED_AREAS, excludedAreas);
        return this;
    }
}

public class PackageSettings {
    public static String ApprovalBaseDirectory = "../resources";
    public static String PdfDiffOutputDirectory = 
        String.format("%s/build/tmp", System.getProperty("user.dir"));
}

That’s all there is to it. With just this tiny bit of code we’re able to use Approval Tests for PDF documents.

Approval Tests For PDF Document Generation

Jan Van Ryswyck — Thu, 07 Oct 2021 11:32:50 +0000

A while back I was confronted with a part of a legacy system that generates PDF documents. This legacy system used a well known library for generating the requested PDF files. The good news was that there were a decent amount of tests available. The not so good news was that these tests made heavy use of test doubles for swapping out most of the types provided by the third-party API. I strongly believe that using test doubles in such cases is not a good choice. In the past I already wrote about why to avoid using test doubles for types that you don’t own.

Tests like these might become a large impediment whenever we upgrade to a newer version of the third-party library. Major versions quite often introduce breaking changes to existing API’s. Whenever tests are strongly coupled to such an API, they basically need to be rewritten during the upgrade.

In order to reduce the coupling of these tests, I decided to replace them with Approval Tests instead. This technique is also known as “Golden Master” or “Characterization Test”. The idea behind an Approval Test is that after the first test run, some output needs to be visually verified and approved. During subsequent test runs, the approved output will be compared to current output. When there’s a difference in the output, the test will fail.

This is especially useful whenever we have to deal with code of a legacy system. You can check out this video from Emily Bache where she demonstrates the Gilded Rose refactoring kata using Approval Tests. Highly recommended!

Usually the output of Approval Tests is captured in plain text files, which has nothing to do with PDF files. So I decided to extend the Java Version of an Approval Test libraryto support PDF documents as well. Let’s have a look at some example code to demonstrate this extension.

Suppose that we have a small application that generates a PDF document. A generated document contains the refrain of a well-known song lyric. The following code shows a possible implementation.

public class SingAlongPdfGenerator {

    public ByteArrayOutputStream generate(SingAlongData pdfData) {

        var outputStream = new ByteArrayOutputStream();
        PdfDocument pdf = new PdfDocument(new PdfWriter(outputStream));

        Document document = new Document(pdf);
        document.add(new Paragraph("Let's sing-a-long:"));

        List list = new List()
            .setSymbolIndent(12)
            .setListSymbol("\u2022");

        for(var refrainLine : pdfData.getRefrainLines()) {
           list.add(new ListItem(refrainLine));
        }

        document.add(list);
        document.close();

        return outputStream;
    }
}

public class SingAlongData {

    private final List<String> refrainLines;

    public SingAlongData(List<String> refrainLines) {

        this.refrainLines = refrainLines;
    }

    public List<String> getRefrainLines() {
        return refrainLines;
    }
}

The SingAlongPdfGenerator class provides a method named generate that accepts a SingAlongData instance as its only parameter. The SingAlongData class is merely a DTO that provides a list of refrain lines. The generatemethod creates a new document containing a paragraph of text and a list of the refrain lines.

Let’s have a look at the code of the corresponding Approval Test.

public class SingAlongPdfGeneratorTests {

    @Test
    public void generateRickRollPdf() {

        var refrainLines = Arrays.asList(
            "Never gonna give you up",
            "Never gonna let you down",
            "Never gonna run around and desert you",
            "Never gonna make you cry",
            "Never gonna say goodbye",
            "Never gonna tell a lie and hurt you"
        );
        var data = new SingAlongData(refrainLines);

        var pdfGenerator = new SingAlongPdfGenerator();
        var result = pdfGenerator.generate(data);

        PdfApprovals.verify(result);
    } 
}

First we create an instance of the SingAlongData DTO, specifying some test data. Next we create an instance of the Subject Under Test, which in this case is the SingAlongPdfGenerator class and call the generate method. This returns a ByteArrayOutputStream containing the data of the PDF document. Then we verify the result by calling the PdfApprovals.verify method. Notice that the anatomy of an Approval Test is identical to any other type of test as it also adheres to the Arrange, Act, Assert pattern.

A new Approval Test always fails the very first time that it gets executed. After the initial test run, a PDF file is generated that needs to be visually verified and approved. So when we first run the generateRickRollPdf test, a file with the name SingAlongPdfGeneratorTests.generateRickRollPdf.received.pdf appears which has the following content:

After we verified the PDF document, we approve it using the following command:

mv ~/src/test/resources/pdfapproval/SingAlongPdfGeneratorTests.generateRickRollPdf.received.pdf 
~/src/test/resources/pdfapproval/SingAlongPdfGeneratorTests.generateRickRollPdf.approved.pdf

At this point we have a PDF file named SingAlongPdfGeneratorTests.generateRickRollPdf.approved.pdf. When we now execute our test again, it passes as the newly generated PDF document matches the approved PDF document.

Note that we also have to make sure to commit the approved PDF file alongside our test code. Otherwise we have to approve the output of the test again when executed on another machine.

Let’s say that we want to make a change to the implementation of our SingAlongPdfGenerator class. For example, we’re going to make a small change to the text inside the paragraph.

document.add(new Paragraph("Let's sing-a-long shall we?"));

When we execute our test again, it fails due to the change that we’ve made.

Failed Approval
  Approved:~/src/test/resources/pdfapproval/SingAlongPdfGeneratorTests.generateRickRollPdf.approved.pdf
  Received:~/src/test/resources/pdfapproval/SingAlongPdfGeneratorTests.generateRickRollPdf.received.pdf

The test created a “received” PDF file again alongside the existing PDF file that we’ve approved earlier. We now have to visually compare the received and the approved file side-by-side. Needless to mention that this is going to be quite cumbersome. For this reason I’ve added the capability that in case of failing test, a third PDF file is generated that contains the annotated differences between the two PDF files.

The purple markers indicate where the changes are located in the document. The green colon indicates what is expected (approved). The red text that we’ve added to the paragraph shows the actual text found in the document (received). We now have to decide whether we want to approve the changes that we’ve made or not. Let’s say that we are happy with the change that we’ve made. To do that we simply run the approval command again as shown earlier. And that’s it.

With just a handful of these Approval Tests I was able to eliminate all the tightly coupled tests. Generating PDF files is more often than not an infrastructure concern. Sociable tests tests are much more appropriate in this case compared to solitary tests. Using Approval Tests this way turned out to be a very valuable approach.

Ignoring Tests

Jan Van Ryswyck — Tue, 22 Jun 2021 17:34:43 +0000

Most test frameworks out there have the capability to disable tests. This is usually done by adding some kind of annotation that instructs the test runner to ignore an individual test method.

[TestFixture]
public class SomeTests {

    [Test]
    public void RegularTest() 
    {
        ...
    }

    [Test]
    [Ignore] // Ignore a single test
    public void IgnoredTest() 
    {
        ...
    }
}

It’s also possible to disable all tests of an entire test fixture by adding an annotation at the class level.

[TestFixture]
[Ignore] // Ignore all tests
public class IgnoredTests {

    [Test]
    public void IgnoredTest() 
    {
        ...
    }

    [Test]
    public void AnotherIgnoredTest() 
    {
        ...
    }
}

A while back, I came to realise that I never use this kind of functionality to prevent the execution of tests. I became aware of this when I started observing other people’s development workflow. For some developers, being able to ignore tests seems to be a very important feature of a test framework. I’ve come to believe that the need for ignoring tests is more prevalent in a Test-After approach than with a Test-First approach. Let’s have a look at a very simple code example.

public class Task
{
    public Person Assignee { get; private set; }

    public void Assign(Person personToAssign)
    {
        Assignee = personToAssign;
    }
}

Suppose that we’ve built an application that provides an online Kanban board service to our customers. In the domain of this application there’s a class named Task. Currently, a task can be assigned to a particular person. This is how the test for the Assign method looks like.

[TestFixture]
public class TaskTests
{
    [Test]
    public void It_should_be_able_to_assign_a_person_to_a_task()
    {
        var person = new Person("Joe");

        var SUT = new Task();
        SUT.Assign(person);

        Assert.That(SUT.Assignee, Is.SameAs(person));
    }
}

These days, being able to assign only a single person to a task is a relic from the past. Therefore, in light of everything “ensemble”, the business requested whether it would be possible to assign multiple people to a single task.

Let’s see how a Test-First approach would look like. Obviously, we would start by writing a failing test. So we’ll add the following test method to the TaskTests test fixture.

[Test]
public void It_should_be_able_to_assign_multiple_people_to_a_task()
{
    var people = new[]
    {
        new Person("Joe"),
        new Person("Annie")
    };

    var SUT = new Task();
    SUT.Assign(people);
}

This new code doesn’t compile at this point as the Assign method currently accepts only a single Person object instead of a collection. This is a good thing because a test method that doesn’t compile qualifies as a failing test. Let’s fix this by adding a new overload of the Assign method.

public class Task
{
    public Person Assignee { get; private set; }

    public void Assign(Person personToAssign)
    {
        Assignee = personToAssign;
    }

    public void Assign(IEnumerable<Person> peopleToAssign)
    {

    }
}

We’re going to leave the implementation of this new method empty for the time being. Now the code of the application compiles again. Also, all the tests pass when we run them. We have working software again. However, it’s still not very useful as we didn’t completely finish the implementation of the new test method. An assert statement is still missing, so we’re going to add that as our next step.

[Test]
public void It_should_be_able_to_assign_multiple_people_to_a_task()
{
    var people = new[]
    {
        new Person("Joe"),
        new Person("Annie")
    };

    var SUT = new Task();
    SUT.Assign(people);

    Assert.That(SUT.Assignees, Is.SameAs(people));
}

By adding the assert statement we find out that the code doesn’t compile anymore. This is a good thing because a test method that doesn’t compile qualifies as a failing test. In order to fix this, we’re need to add an Assignees property to the Task class.

public class Task
{
    public Person Assignee { get; private set; }
    public IEnumerable<Person> Assignees { get; private set; }

    public void Assign(Person personToAssign)
    {
        Assignee = personToAssign;
    }

    public void Assign(IEnumerable<Person> peopleToAssign)
    {

    }
}

Now we’re able to compile the code again. However, when we run all the tests again we find out that the assert statement that we’ve just added fails the test. In order to make this test pass, we need to add the necessary implementation to the newly overloaded Assign method.

public class Task
{
    public Person Assignee { get; private set; }
    public IEnumerable<Person> Assignees { get; private set; }

    public void Assign(Person personToAssign)
    {
        Assignee = personToAssign;
    }

    public void Assign(IEnumerable<Person> peopleToAssign)
    {
        Assignees = peopleToAssign;
    }
}

When we run the test suite again, we notice that all the tests are green. We have working software again.

The rest of the code base still uses the Assign method that accepts a single Person object. So our next step is to migrate all the client code of the Task class to use the new version of the Assign method. We go about this migration by writing failing tests and making them pass by using the Assign method that accepts a collection of Person objects. When the original Assign method has been fully replaced, we can remove the corresponding test as well as the method itself.

Notice that during this workflow, we were always able to run all the tests without ignoring any existing ones. Let’s compare the Test-First approach to a Test-Last approach using the same example. We’ll immediately start off by making the necessary changes in the Task class.

public class Task
{
    public IEnumerable<Person> Assignees { get; private set; }

    public void Assign(IEnumerable<Person> peopleToAssign)
    {
        Assignees = peopleToAssign;
    }
}

We’ve renamed the Assignee property to Assignees while we also changed its type to IEnumerable<Person>. Likewise,
we've changed the parameter of the Assign method to IEnumerable<Person> peopleToAssign. By making this change the
way we did, we broke the contract of the Task class. The result is that the code doesn't compile anymore.

First, the code of the test that verifies the Assign method needs to be changed. We complain a bit to our colleagues about how tests are holding us back while we’re developing code. After we’re done complaining, we comment out the line SUT.Assign(person); and slap the Ignore attribute on the test method. We tell ourselves that we’re going to fix this test later.

[TestFixture]
public class TaskTests
{
    [Test]
    [Ignore("Meh")]
    public void It_should_be_able_to_assign_a_person_to_a_task()
    {
        var person = new Person("Joe");

        var SUT = new Task();
        //SUT.Assign(person);

        Assert.That(SUT.Assignees, Is.SameAs(person));
    }
}

Next, we find out that there are other places in the production code that need attention as well. There could be a single caller of the Assign method, or there could be several places throughout the code base that call this method. Suppose that there are three places in the code that need to be altered to reflect the changes that we’ve made. After adding some additional Ignore attributes for disabling some other tests, everything compiles now.

Suppose that we’re now dragged into a meeting. When the meeting has ended, we (conveniently) forgot about the ignored tests. After all, we want to continue working on our feature. “Stressed and always in a hurry”.

Notice that after this endeavour we don’t even have a clue whether we have working software or not. Please note that the simple example that has been presented here in and of itself is not important. We could have made the necessary changes using the refactoring capabilities of any modern IDE. That’s not the point. The point is the difference between both workflows.

Whenever we feel the need to disable the execution of a test, we might want to consider finding a better approach. Tests should not be there to stand in our way. Rather they can help guide us throughout the development process. Test-Driven Development is not only about writing tests first. It’s about being able to work in very short iterations. The important part is that by the end of every iteration, we have code that works. Writing tests first is just the means that guides us through that workflow.

Writing Maintainable Unit Tests In Print

Jan Van Ryswyck — Fri, 23 Apr 2021 08:17:16 +0000

I’m very happy to announce that Writing Maintainable Unit Tests is also available as a paper book. Both the ebook and the paper book have been completely self-published. I must say that this whole endeavour was quite an educational experience.

The ebook is still available on LeanPub. The paper book can be purchased at most leading book stores:

For more information, you can have a look at the full table of contents.