Some surprising things about the order in which Spek runs things

These are some of the surprising things I learned about in what order Spek (the Android testing framework) runs things, especially the describe, it, afterGroup, afterEachGroup, and afterEachTest. This blog was originally posted on daniellevass.com.

Introduction

We are using Spek at work for our testing framework for our Android library. I spent too long today trying to work out why my newly written tests were failing and I learned the order that Spek runs my instructions were not what I expected.

Example App

I have created a example app which simplifies what I was trying to do in the office.

The app has a singleton for a current account - CurrentAccountImplementation. You need to call connect to get the current balance (always 0) and disconnect. You then have two actions to add money and take money. You can't send any actions if you aren't connected.

First Description Block

I have created a Spek test file CurrentAccountTests. My intention was to use the describe and it together, so that a describe block would group a bunch of the instructions I wanted to flow through e.g.

describe("end to end flow of typical user") {
     CurrentAccountImplementation.connect()

     it("balance starts at 0") {
         assertThat(CurrentAccountImplementation.getCurrentBalance()).isEqualTo(0)
     }

     it("balance is 10 when I add 10") {
         CurrentAccountImplementation.addMoney(10)
         assertThat(CurrentAccountImplementation.getCurrentBalance()).isEqualTo(10)
     }

     it("balance is 5 when I withdraw 5") {
         CurrentAccountImplementation.takeMoney(5)
         assertThat(CurrentAccountImplementation.getCurrentBalance()).isEqualTo(5)
     }
 }

Good testing etiquette means I should ensure that I set up and tear down each test, so for this example I started with adding afterEachGroup and ensuring I called CurrentAccountImplementation.disconnect().

Second Description Block

This worked fine until I added another describe block with some more it tests:

describe("error handling for insufficient funds") {
    CurrentAccountImplementation.connect()

    it("balance starts at 0") {
        assertThat(CurrentAccountImplementation.getCurrentBalance()).isEqualTo(0)
    }

    it("should throw an error with insufficient funds") {
        val exception = assertThrows<Exception>("Should throw an Exception") {
            CurrentAccountImplementation.takeMoney(100)
        }

        assertThat(exception).hasMessageThat().isEqualTo("insufficient funds")
    }
}

I was surprised to find that my CurrentAccount had been disconnected before the second description blocks it functions started running. What happened?

Order Spek Methods Were Called

I ended up adding a bunch of logs to see in which order each methods were being called:

• entering describe typical user
• entering describe error handling insufficient funds
• entering it balance 0
• entering it balance is 10
• entering it balance is 5
• AFTER EACH GROUP ~~ resetting current account
• entering it balance 0

My assumption that a description block would be called, then it's inner its was incorrect. However, the afterEachGroup was called when it finished all the its in a description block.

Conclusion

I've since learned that it's not sensible to keep state in a description block, each it test should be responsible for it's full lifecycle. You can use the assertThat to explain what you're doing without needing the it description to give more context.

However the behaviour of all the description blocks being called before any of the its and in relation to the afterEachGroup was unexpected to me and so maybe is useful to others.

Summary (for future Danielle):

• all description blocks are called first, then each it function in order
• afterGroup will call after all the its in the entire file
• afterEachGroup will call after all the it functions in a given description
• afterEachTest will call after each it has completed
• don't keep state in the description block, each it should be responsible for it's own state.