📸 Snapshot Testing with Kotlin

Snapshot testing is a test technique where first time the test is executed the output of the function being tested is saved to a file, the snapshot, and future executions of the test will only pass if the function generates the very same output

This seems very popular in the frontend community but us backends we can use it too! I use it whenever I find myself manually saving test expectations as text files 😅

In this PoC we will use two different snapshot testing libraries JVM compatible:

1. Java Snapshot Testing - loved by lazy productive devs!
2. Selfie - are you still writing assertions by hand?

rogervinas / snapshot-testing

📸 Snapshot Testing with Kotlin

Let's start!

• Implementation to test - test results should be deterministic!
• Using Java Snapshot Testing
  • Serialize to JSON
  • Parameterized tests
• Using Selfie
  • Serialize to JSON
  • Parameterized tests

Implementation to test

Imagine that we have to test this simple MyImpl:

class MyImpl {

  private val random = Random.Default

  fun doSomething(input: Int) = MyResult(
    oneInteger = input,
    oneDouble = 3.7 * input,
    oneString = "a".repeat(input),
    oneDateTime = LocalDateTime.of(
      LocalDate.of(2022, 5, 3),
      LocalTime.of(13, 46, 18)
    )
  )

  fun doSomethingMore() = MyResult(
    oneInteger = random.nextInt(),
    oneDouble = random.nextDouble(),
    oneString = "a".repeat(random.nextInt(10)),
    oneDateTime = LocalDateTime.now()
  )
}

data class MyResult(
  val oneInteger: Int,
  val oneDouble: Double,
  val oneString: String,
  val oneDateTime: LocalDateTime
)

Notice that:

• doSomething function is testable as its results are deterministic ✅
• doSomethingMore function is not testable as its results are random ❌

So first we need to change doSomethingMore implementation a little bit:

class MyImpl(
  private val random: Random,
  private val clock: Clock
) {

  fun doSomething() { }

  fun doSomethingMore() = MyResult(
    oneInteger = random.nextInt(),
    oneDouble = random.nextDouble(),
    oneString = "a".repeat(random.nextInt(10)),
    oneDateTime = LocalDateTime.now(clock)
  )
}

So we can create instances of MyImpl for testing that will return deterministic results:

myImplUnderTest = MyImpl(
  random = Random(seed=1234),
  clock = Clock.fixed(
    Instant.parse("2022-10-01T10:30:00.000Z"),
    ZoneId.of("UTC")
  )
)

And create instances of MyImpl for production:

myImpl = MyImpl(
  random = Random.Default, 
  clock = Clock.systemDefaultZone()
)

Using Java Snapshot Testing

To configure the library just follow the Junit5 + Gradle quickstart guide:

• Add required dependencies
• Add required src/test/resources/snapshot.properties file. It uses by default output-dir=src/test/java so snapshots are generated within the source code (I suppose so we don't forget to commit them to git) but I personally use output-dir=src/test/snapshots so snapshots are generated in its own directory

We can write our first snapshot test MyImplTestWithJavaSnapshot:

@ExtendWith(SnapshotExtension::class)
internal class MyImplTestWithJavaSnapshot {

  private lateinit var expect: Expect

  private val myImpl = MyImpl()

  @Test
  fun `should do something`() {
    val myResult = myImpl.doSomething(7)
    expect.toMatchSnapshot(myResult)
  }
}

It will create a snapshot file MyImplTestWithJavaSnapshot.snap with these contents:

org.rogervinas.MyImplTestWithJavaSnapshot.should do something=[
MyResult(oneInteger=7, oneDouble=25.900000000000002, oneString=aaaaaaa, oneDateTime=2022-05-03T13:46:18)
]

And if we re-execute the test it will match against the saved snapshot

Serialize to JSON

By default, this library generates snapshots using the ToString serializer. We can use the JSON serializer instead:

@Test
fun `should do something`() {
  val myResult = myImpl.doSomething(7)
  expect.serializer("json").toMatchSnapshot(myResult)
}

Don't forget to add the required com.fasterxml.jackson.core dependencies and to delete the previous snapshot

Then the new snapshot file will look like:

org.rogervinas.MyImplTestWithJavaSnapshot.should do something=[
  {
    "oneInteger": 7,
    "oneDouble": 25.900000000000002,
    "oneString": "aaaaaaa",
    "oneDateTime": "2022-05-03T13:46:18"
  }
]

We can also use our own custom serializers just providing in the serializer method one of the serializer class, the serializer instance or even the serializer name configured in snapshot.properties

Parameterized tests

We can create parameterized tests using the scenario method:

@ParameterizedTest
@ValueSource(ints = [1, 2, 3, 4, 5, 6, 7, 8, 9])
fun `should do something`(input: Int) {
  val myResult = myImpl.doSomething(input)
  expect.serializer("json").scenario("$input")
    .toMatchSnapshot(myResult)
}

This way each execution has its own snapshot expectation:

org.rogervinas.MyImplTestWithJavaSnapshot.should do something[1]=[
  {
    "oneInteger": 1,
    "oneDouble": 3.7,
    "oneString": "a",
    "oneDateTime": "2022-05-03T13:46:18"
  }
]

...

org.rogervinas.MyImplTestWithJavaSnapshot.should do something[9]=[
  {
    "oneInteger": 9,
    "oneDouble": 33.300000000000004,
    "oneString": "aaaaaaaaa",
    "oneDateTime": "2022-05-03T13:46:18"
  }
]

Using Selfie

To configure the library follow Installation and Quickstart guides and just add required dependencies with no extra configuration

We can create our first snapshot test MyImplTestWithSelfie:

internal class MyImplTestWithSelfie {
  @Test
  fun `should do something`() {
    val myResult = myImpl.doSomething(7)
    Selfie.expectSelfie(myResult.toString()).toMatchDisk()
  }
}

It will create a snapshot file MyImplTestWithSelfie.ss with these contents:

╔═ should do something ═╗
MyResult(oneInteger=7, oneDouble=25.900000000000002, oneString=aaaaaaa, oneDateTime=2022-05-03T13:46:18)

And if we re-execute the test it will match against the saved snapshot

Anytime the snapshot does not match we will get a message with instructions on how to proceed:

Snapshot mismatch / Snapshot not found
- update this snapshot by adding `_TODO` to the function name
- update all snapshots in this file by adding `//selfieonce` or `//SELFIEWRITE`

Serialize to JSON

If instead of matching against .toString() we want to serialize to JSON we can customize a Camera and use it:

private val selfieCamera = Camera<Any> { actual ->
  val mapper = ObjectMapper()
  mapper.findAndRegisterModules()
  Snapshot.of(
    mapper
      .writerWithDefaultPrettyPrinter()
      .writeValueAsString(actual)
  )
}

@Test
fun `should do something`() {
  val myResult = myImpl.doSomething(7)
  Selfie.expectSelfie(myResult, selfieCamera).toMatchDisk()
}

Then the new snapshot file will look like:

╔═ should do something ═╗
{
  "oneInteger" : 7,
  "oneDouble" : 25.900000000000002,
  "oneString" : "aaaaaaa",
  "oneDateTime" : [ 2022, 5, 3, 13, 46, 18 ]
}

Parameterized tests

We can use parameterized tests passing a value to identify each match:

@ParameterizedTest
@ValueSource(ints = [1, 2, 3, 4, 5, 6, 7, 8, 9])
fun `should do something`(input: Int) {
  val myResult = myImpl.doSomething(input)

 Selfie.expectSelfie(
  myResult, selfieCamera).toMatchDisk("$input")
}

Then snapshots will be saved this way:

╔═ should do something/1 ═╗
{
  "oneInteger" : 1,
  "oneDouble" : 3.7,
  "oneString" : "a",
  "oneDateTime" : [ 2022, 5, 3, 13, 46, 18 ]
}

...

╔═ should do something/9 ═╗
{
  "oneInteger" : 9,
  "oneDouble" : 33.300000000000004,
  "oneString" : "aaaaaaaaa",
  "oneDateTime" : [ 2022, 5, 3, 13, 46, 18 ]
}

Thanks and happy coding! 💙

Comments

[Ned Twigg]

Great testing series! I added a PR to your example which adopts the selfie snapshot testing library. It's advantages are

• it can do snapshots on disk or as inline literals
• it garbage-collects unused disk snapshots automatically
• you don't need to manipulate snapshot files manually to control read/write
• (advanced) you can snapshot multiple facets of an entity, and then assert some on disk and others inline to tell a story

[Roger Viñas Alcon]

Thanks! I will take a look