How to write meaningful test assertions that help you not break your code base

Introduction

Testing is an important part of software development. It helps us to ensure the quality and functionality of our code. However, if not written well some tests will not ensure quality. One factor is that developers can write too generic assertions (it happens in mockito verify too) when writing a test scenario. Neglecting to write assertions that match the expected behavior of the code, allows many bugs to make their way to production code.

In this article, I will show how you can avoid the mistake of writing too generic assertions. This will help you reduce a lot the amount of bugs detected in production.

What are meaningful assertions and verifications?

Assert and Verify are statements that help us to check the expected behavior of our code. For example, you can use an assertion **to check if a method returns the correct value or to verify if a mock was called with the right arguments.

But, even if your tests have some assertion and verify statements, that doesn’t mean they are checking the correct behavior. They have to be meaningful and specific. If your tests have a lot of P assertNotNull, assertTrue, or verify(any()), this can be a sign that maybe you have too many generic assertions.

Meaningful assertions check for specific things. They look for the exact match of the code output. They tell you exactly the exact result you are looking for. It verifies if the parameters passed into a function in the correct order, with a correct value. It checks how many times it was executed. These assertions have a meaning to be there, they are assuring that your code is behaving as expected as much as possible.

Why are meaningful assertions and verifications important?

Meaningful assertions are important for:

• Detect bugs and errors early: By writing specific assertions and verifications, you can catch more bugs and errors in your code base before they reach production. This can save you time, as well as prevent potential security risks and customer complaints.
• Improve code quality and readability: By writing specific assertions and verifies, you can make your code more clear and understandable. You are telling in your test scenarios exactly the behavior expected when interacting with a part of the code. It helps to understand the code better and can be helpful as a kind of documentation.
• Increase confidence and trust: By writing relevant assertions and verifies, you can increase your confidence and trust in your code. You can make changes and refactor and be more confident that your changes will not break the code or bring more bugs into it.

Common mistakes when writing assertions

These simple mistakes can make your tests inefficient:

• Avoid using too generic assertion: methods like assertNotNull, assertTrue, or verify(mockObject.testMethod(any())). They are useful in some cases but don’t tell you much about the expected behavior. Use more descriptive ones like assertEquals, assertThat, verify(mockService.checkDocument(eq(customerDocument)).
• Avoid using generic variable names: Your testing variables and mock names should be descriptive. They have to tell you more about the behavior and business rules. Avoid using, test, value, number, and x. Use more descriptive names like: validDocument, invalidEmail, mockCustomerService.
• Don’t assert too much or too little: Test the code’s behavior results and its effects. Check only the parts that have changed or are new.

Examples

Here are some examples to illustrate the problems, and how to fix them. For example:

We have to test the behavior of the method getFullNumber from this Contactclass:

public class Contact {

    private String countryCode;
    private String areaCode;
    private String phoneNumber;

    public Contact(String countryCode, String areaCode, String phoneNumber) {
        this.countryCode = countryCode;
        this.areaCode = areaCode;
        this.phoneNumber = phoneNumber;
    }

    public String getFullNumber() {
        return "+" + countryCode + " " + areaCode + " " + phoneNumber;
    }
}

It’s a simple behavior, but it will serve well to illustrate. If we write a bunch of generic assertions like this. Then write a test that aims for the specific behavior:

    @Test
    public void testGenericAssertions() {
        Contact contact = new Contact("55", "20", "9999 9999");
        assertAll(
                () -> assertNotNull(contact.getFullNumber()),
                () -> assertFalse(contact.getFullNumber().isEmpty()),
                () -> assertDoesNotThrow(contact::getFullNumber),
                () -> assertTrue(contact.getFullNumber().contains("9999 9999")),
                () -> assertTrue(contact.getFullNumber().contains("55")),
                () -> assertTrue(contact.getFullNumber().contains("20"))
        );
    }

    @Test
    public void testWithSpecificAssertion() {
        Contact contact = new Contact("55", "20", "9999 9999");
        assertEquals("[phone]", contact.getFullNumber());
    }

If we run the tests, it works, because everything is correct with our code.

But if a developer introduces by accident hits a key and changes the code creating a bug:

public String getFullNumber() {
    return "+" + countryCode + " a" + areaCode + " " + phoneNumber;
}

The generic tests are still working just fine. But the one with the specific assertion will break.

The test with a meaningful assertion on the code behavior caught a bug in our code. That’s the power of this technique.

That’s the way to write meaningful assertions. You have to make them assert specific results as much as possible.

Now from the verify statement. The verify method is used with Mockito and helps us to check interactions with Mockito. Usually, developers can use Mockito combined with any() methods. Which is a high-risk combination.

Let’s see why in an example:

So you have a service class that uses a repository to find some information. The code can be something like this:

public interface Repository {
    String find(String name, String email);
}

public class Service {

    private Repository repository;

    public Service(Repository repository) {
        this.repository = repository;
    }

    public String getUserInfo(String name, String email) {
        // Some business logic here
        return repository.find(name, email);
    }
}

Now testing your code you can test in these two ways:

    @Test
    public void testGetUserInfoWithAny() {
        Repository repository = mock(Repository.class);
        Service service = new Service(repository);

        when(repository.find(any(), any())).thenReturn("Dummy");

        String result = service.getUserInfo("Alice", "[e-mail]");

        verify(repository).find(any(), any());
    }

    @Test
    public void testGetUserInfoWithSpecific() {
        Repository repository = mock(Repository.class);
        Service service = new Service(repository);

        when(repository.find("Alice", "[e-mail]")).thenReturn("Alice's info");

        String result = service.getUserInfo("Alice", "[e-mail]");

        verify(repository).find("Alice", "[e-mail]");
    }

If you observe, we are verifying using any() methods in the first test. In the second one, we are looking for the exact value.

If we run it, both should pass:

Now let’s introduce a simple bug and make things a little more interesting. Let’s say a developer changed the order of the parameters in the method getUserInfo:

public String getUserInfo(String email, String name) {
        // Some business logic here
        return repository.find(name, email);
    }

With this change, the code will still compile, because they are both strings. The code is still syntactic and semantic correct. All methods that use this method from the service will not break even if the parameters are in the wrong order now.

So when we run the tests:

That’s something to look for. The verify statement can be seen as a form of assertion too. So if you are using mocks. Make sure that they call a method properly.

The same applies to when methods. It should match exactly the parameters expected from your code behavior.

Conclusion

So we saw how important is to write meaningful assertions. That can save you from breaking your production code. It will make you confident that your tests are efficient. You can trust your code base. And make a refactor without worrying if you gonna miss something that would cause a headache.

Now it’s your turn, to check the test methods of your project, are you seeing some of the mistakes that we described here? Fix it and tell your team how dangerous this was and how they can avoid making this mistake too.

Do that and become a top developer on your team. The one that never breaks the code base!

Don’t forget to follow me on social media to be the first to read when my next article comes out!

Willian Moya (@WillianFMoya) / X (twitter.com)

Willian Ferreira Moya | LinkedIn