Snapshot Testing in Python

Have you ever found yourself writing long, tedious tests that take in a large data structure, and test the output of a similar, validated data structure? Whether it's converting a dictionary to a named tuple or serializing an object, if you've ever tried testing a data model or a GraphQL model for an API, you've probably experienced this.

Writing tests and replicating the same response we expect can be a tedious and monotonous task. Snapshot testing can be a great way to ease the pain of testing these data structures. As our APIs evolve, we need to know when our changes introduce any breaking changes, and a snapshot test will tell us exactly that.

What is Snapshot testing?

Put simply, A snapshot is a single state of your output, saved in a file.

A typical snapshot test case for a mobile app renders a UI component, takes a screenshot, then compares it to a reference image stored alongside the test. The test will fail if the two images do not match: either the change is unexpected, or the screenshot needs to be updated to the new version of the UI component.

A similar approach can be taken when it comes to testing your APIs or data structures. Instead of rendering the graphical UI, you can use a test renderer, such as the SnapshotTest library, to quickly generate a serializable value for your API response.

Snapshots lets us write all these tests in a breeze, as it automatically creates the snapshots for us the first time the test is executed. Then, every subsequent time the tests are run, it will compare the output with the file snapshot. If they are different, our tests will fail, immediately alerting us of the change.

How to use the SnapshotTest library

First, we install the library:

pip install snapshottest

I'll provide code snippets below for both the python built-in unittest library and pytest, but you can probably use this with any testing framework you choose.

We'll begin with a simple namedtuple that we'd like to test.

'''Our Clown Code'''

from collections import namedtuple

Clown = namedtuple('Clown', 'name', 'nose_type'))

With unittest:

from snapshottest import TestCase

class TestClown(TestCase):

    def test_clown(self):
      clown = Clown('bozo', 'red and round')                                  
      self.assertMatchSnapshot(clown)  

# >> python -m unittest

With pytest:

import pytest

def test_clown(snapshot):
      clown = Clown('bozo', 'red and round')                                  
      snapshot.assert_match(clown) 

# >> pytest

This will create a snapshot directory and a snapshot file the first time the test is executed.

>> pytest
=================== SnapshotTest summary ===================
1 snapshot passed.
1 snapshot written in 1 test suites.

Next time we run our tests, it will compare the results of our test with the snapshot. Since nothing changed, we get the following:

>> pytest
=================== SnapshotTest summary ===================

1 snapshot passed.

If our data structure or output changes, the snapshot tests will fail. We can then compare the changes and determine whether they are valid. To update the snapshot we can run:

pytest --snapshot-update

Obviously the above is a super simple test case, and we're not really testing any logic. But this same pattern can be applied for testing API requests.

import requests

def test_my_api(snapshot):
    response = request.get('my_api.com')
    snapshot.assert_match(response)

Or for testing a GraphQL API

from graphene.test import Client

def test_hey(snapshot):
    client = Client(my_schema)
    response = client.execute('''{ hey }''')
    snapshot.assert_match(response)

Snapshot testing may not be a replacement for unit or integration tests, but it's a great way to quickly and effectively determine whether the state of your API or data serialization has changed.

Comments

[briwoto]

Hi,
When using with pytest, there is a parameter named "snapshot" passed into every def. Where is that parameter coming from? The PyTest example does not contain any "import" commands

[Leah Einhorn]

Good question! This is part of the pytest "magic". The short answer is that if you install snapshottest, that parameter will automatically be available when you run your tests with pytest. There is no need to import or define anything else!

The longer answer is that pytest uses fixtures as function arguments; if you're not familiar with it and want to learn more, you can read the docs here.
It looks for fixtures in several locations (and so you don't need to import them), a common one being conftest.py. In the case of the snapshottest library, the fixture is defined here and configured so that pytest knows where to look for it.

Hope that helps!

[Mike-Rossi]

Hello,

In the last image, you have the line client = Client(my_schema), what exactly would go into my_schema, would it be the name of your schema file or would it be the query code itself?

[Leah Einhorn]

Ah, it would be the GraphQL schema object. The example provided assumes usage of graphene, so the following would be a simple example of the schema:

from graphene import ObjectType, String, Schema

class Query(ObjectType):

    hello = String()

    def resolve_hello(root, info, name):
        return f'Hello {name}!'

my_schema = Schema(query=Query)

You would then pass my_schema into the Client.

You can read more about graphene here.