Comparing Spock and JUnit

JUnit is considered the standard framework to test Java applications. I don't think anybody will challenge its position in the short term. In addition JUnit 5 will be released soon, and it will support the shiny Java 8 features, something that has been a bit of a handicap recently with JUnit 4.

However, a new testing framework is pushing hard. It was initially created to test Groovy code, but given that Groovy runs on the JVM it can be used to test Java code too!

Spock

Spock is an acronym for "Specification and Mock". I guess now it's easy to figure out what it allows us to do: creating specifications of our systems, adding capabilities to create mocks of our dependencies. I say specifications and not tests, something that can be a bit confusing at the beginning, but hopefully will be well understood with the examples. In few words, a specification is just a test class on steroids.

My intention with this post is showing in a clear example the main differences between Spock and JUnit. I don't want to go into too many details about what Spock has to offer, you have the official documentation for that.

The class we need to test

We're going to create a naive class called Calculator, that we want to test:

public class Calculator {

    private Audit audit;

    public Calculator(Audit audit) {
        this.audit = audit;
    }

    public long add(int operand1, int operand2, Mode mode) {
        audit.register(String.format("%d + %d (%s)",
                        operand1,
                        operand2,
                        mode));

        if (mode == Mode.ABSOLUTE) {
            operand1 = Math.abs(operand1);
            operand2 = Math.abs(operand2);
        }

        return operand1 + operand2;
    }

    public static enum Mode {ABSOLUTE, STRAIGHT;}
}

The features of this class (which is not a great example of best practices, in any case) are clear.

Tests in JUnit

This would be the JUnit class that would cover all the features in Calculator:

@RunWith(Parameterized.class)
public class CalculatorTest {

    @Mock
    private Audit audit;

    @InjectMocks
    private Calculator calculator;

    @Before
    public void setUp() throws Exception {
        MockitoAnnotations.initMocks(this);
    }

    private int operand1;
    private int operand2;
    private long expectedResultStraight;
    private long expectedResultAbsolute;

    @Parameterized.Parameters
    public static Collection data() {
        Object[][] data = new Object[][] {
                { 2, 2, 4, 4 },
                { -2, 2, 0, 4 },
                { -3, -3, -6, 6 },
                { 0, 0, 0, 0 }
        };

        return Arrays.asList(data);
    }

    public CalculatorTest(int operand1,
                          int operand2,
                          long expectedResultStraight,
                          long expectedResultAbsolute) {

        this.operand1 = operand1;
        this.operand2 = operand2;
        this.expectedResultStraight = expectedResultStraight;
        this.expectedResultAbsolute = expectedResultAbsolute;
    }


    @Test
    public void testAddStraight() throws Exception {
        long sum = calculator.add(operand1, operand2, STRAIGHT);
        assertThat(sum, is(expectedResultStraight));

        verify(audit).register(
                String.format("%d + %d (STRAIGHT)", operand1, operand2));
    }

    @Test
    public void testAddAbsolute() throws Exception {
        long sum = calculator.add(operand1, operand2, ABSOLUTE);
        assertThat(sum, is(expectedResultAbsolute));

        verify(audit).register(
                String.format("%d + %d (ABSOLUTE)", operand1, operand2));
    }

}

I see several inconveniences here:

• We're adding two libraries on top of JUnit to improve our test code: Hamcrest, to improve readability, and Mockito, to create mocks of our dependencies.
• In order to use Data Driven Testing appropriately we need our tests to be run by the Parameterized runner, instead of the JUnit default runner.
• The quantity of boilerplate code that we need to write out of our tests methods is considerable. We need to prepare a data matrix using a certainly ugly approach (in the data method), we need a setUp method to initialize dependencies, a constructor to initialize the data in every test, etc.

All this issues are not bad by themselves, we all know that Java is not popular for being a concise language. Its verbosity can be good in some occasions actually, because it forces good practices and thinking carefully about our designs. But when we need to write tests, wouldn't it be better to skip some of these problems?

Spock to the rescue

Let's see how we would implement our specification for Calculator in Spock:

class CalculatorSpec extends Specification {

    Audit audit = Mock()

    @Subject
    Calculator calculator = new Calculator(audit)

    def "Calculator can add operands in straight mode"() {
        when: "We add two operands in straight mode"
        long sum = calculator.add(operand1, operand2, STRAIGHT)

        then: "The result of the sum matches the expected one"
        sum == expectedResult

        where:
        operand1 | operand2 || expectedResult
        2        | 2        || 4
        -2       | 2        || 0
        -3       | -3       || -6
    }

    def "Calculator can add operands in absolute mode"() {
        when: "We add two operands in absolute mode"
        long sum = calculator.add(operand1, operand2, ABSOLUTE)

        then: "The result of the sum matches the expected one"
        sum == expectedResult

        where:
        operand1 | operand2 || expectedResult
        2        | 2        || 4
        -2       | 2        || 4
        -3       | -3       || 6
    }

    def "Audit object intercepts all calls to the Calculator"() {
        when: "We add two operands in any mode"
        calculator.add(2, 2, ABSOLUTE)
        calculator.add(2, 2, STRAIGHT)

        then: "The Audit object registers the call"
        1 * audit.register("2 + 2 (ABSOLUTE)")
        1 * audit.register("2 + 2 (STRAIGHT)")
    }
}

We can see that:

• The tests are written in Groovy.
• The name of the test class has the Spec suffix and extends Specification. This names actually means that our class is not only testing code, it's actually generating specifications readable by a human being.
• We can create a mock invoking the method Mock, which is part of the framework.
• The Subject annotation tells us which class we're testing / describing
• Our methods can be named using a String without any restriction. This means we don't need to follow any convention given by programming languages, so it's much easier to describe what we're going to test without being afraid of creating long and unreadable method names.
• The test methods are clearly divided in sections, following the Arrange Act Assert pattern or "Given When Then". Our example is so simple that we don't need to do anything in the given: section, so we skip it.
• Every section can have its own description, so we can explain exactly what we're preparing, running or verifying there.
• The then: section contains assertions which will be verified using Groovy Truth. We don't need to use assert methods, just boolean expressions.
• The verifications using mocks are very clear: 1 * audit.register("2 + 2 (ABSOLUTE)") verifies that we're invoking the register method in audit one single time with those parameters, and it will fail if that doesn't happen. Simulating behaviors with mocks is very simple too, take a look at the official documentation for further details.
• The data matrices are very concise, as we can see in where: sections. No need to create two dimensional arrays anymore.

Finally, let's see how the framework shows assertion errors, as it does it in a graphical way, where we can see the error, the values of all the variables involved, etc. Here's an example:

Condition not satisfied:

sum == expectedResult
|   |  |
4   |  5
    false

Spock vs JUnit

I'm not trying to convince anybody here about using one framework against the other, just wanted to compare the differences between both. The benefits of using Spock are clear, but it's true that it forces us to add a new language to our technology stack, for example. In my experience, companies are quite reluctant to do this, as it makes hiring a bit more complicated. I don't think learning Groovy and Spock if you're a Java developer is very challenging, anyway.

To use Spock in a Maven project it's necessary to configure our POM file properly (more info here). It's of course easier if we use Gradle.

Comments

[Mateus Costa]

Very cool post.

Tests should be easy to read and for me spock is cleaner than junit. I already work with both and one trade off to make is that groovy is a little bit slower than junit.

Another bad thing about spock and junit be more consolidated in the market is that for most of the places that support java, they don't support groovy.

I'm doing a lot of exercises these days (like hackerrank) and is not possible to code in java and test with spock =/

groovy is groovy =)

[Jean-Rodney Larrieux]

Great post! Been learning Spock in the past month. You just thought me about "subject".

I used to try hard (and reluctant) TDD, but with Spock and BDD, testing is exciting to me. I go back and look at old projects just to add additional test by adding spock.... Loving it!