Test beyond your code with docker & pytest

Docker opens up access to an incredible ecosystem of build-up applications. We find containers for everything … We can run a postgres database, run a redis database or launch an ftp server.

Docker is a great way to run dependency our code needs when running system integration tests that will validate our application.

In this article, we will discover how to use docker in our python tests by writing a minimum of boilerplate while keeping the same CI/CD pipeline.

Testing an application beyond its code thanks to docker

A bug can hide in the use of a database, for example, a badly written SQL query. We need a database to test that. We can’t validate this behavior with just the code.

It is an important topic. Django, a major python framework, provides a partial answer to this. It only manages a test workflow for the database. This workflow also requires you to launch the database by yourself before launching your tests.

To solve this topic, our tests must be able to launch containers by themselves.

To mount the containers from our tests, we will boost our tests with Fixtup. Our tests will be able to start and stop containers at the right time by themselves.

If a container is expensive to start, it can be mounted once and reused. pytest will stop it after playing the last test it has to play.

Install the requirements

To follow this article, we need to prepare our developper station.

1 . We need to be able to use docker and docker-compose from a terminal.

$ docker --version
$ docker-compose --version

2 . We need poetry to create the python project

$ poetry

Create a python project

$ poetry init
$ poetry add psycopg2-binary
$ poetry add --dev pytest fixtup

We will create a docker container for postgresql as a fixture to use in our tests.

$ poetry run fixtup init
Choose a directory to store fixture templates : tests/fixtures
Python manifest (pyproject.toml) [pyproject.toml]

$ poetry run fixtup new
Choose a fixture identifier : postgres
Mount environment variables on this fixture (y/n) [n]
Mount docker container on this fixture (y/n) [n] y

tests/fixtures/postgres/docker-compose.yml

version: "3.9"
services:
    postgresql:
        init: true
        image: postgres
        ports:
            - "5432:5432"
        environment:
            - POSTGRES_PASSWORD=1234
        volumes:
            - /var/lib/postgresql

We will activate the hook_started hook to wait for the database server to be loaded before starting our test.

tests/fixtures/postgres/.hooks/hook_started.py

from time import sleep
import fixtup. helper

fixtup.helper.wait_port(5432, timeout=2000)

# we wait 1 second to ensure the database is ready
sleep(1)

Using postgresql database in a test

Now we are going to install the driver for postgresql and write our first test.

test/test_postgresql_database.py

import fixtup
import psycopg2

def test_postgres_should_work():
    with fixtup.up('postgres'):
        conn = psycopg2.connect("host=127.0.0.1 dbname=postgres user=postgres password=1234")
        core = conn.cursor()
        cur.execute('SELECT 1+1')
        res = cur.fetchall()
        assert res[0][0] == 2
        conn.close()

Use the same postgresql database for all tests

Containers can take a long time to start, that’s why we would like to reuse them between several tests. By default, fixtup stops and starts them between each test. By activating the keep_up policy of fixtup, the containers will remain up as soon as they have been launched. They will be deleted when pytest has finished running.

tests/fixtures/postgres/fixtup.yml

# This flag control if a fixture stay up and running between every test. The fixture is stop and
# unmount when the test process stop
#
# This attribute allow to start a database only once and stop the container only when unittest has finished to run
# the test suite. It may be interested to improve the performance if your start and stop process is too slow
keep_up: true

Wait for database

Instead of waiting for port availability and a timer after to wait database to be ready, we will rewrite the hook_started to attempt to connect to the database regularly.

tests/fixtures/postgres/.hooks/hooks_started.py

from time import monotonic
import psycopg2

start = monotonic()
connected = False
timeout = 5
while not connected:
    try:
        conn = psycopg2.connect("host=127.0.0.1 dbname=postgres user=postgres password=1234")
        connected = True
        conn.close()
    except Exception:
        if monotonic() - start > timeout:
            raise TimeoutError()

Going further

In this article, we have seen how to use docker containers in our tests with pytest and fixtup.

If your test depends on an application that works in docker, pytest can mount these containers through Fixtup. It can be a redis base, a localstack stack for AWS, or an ftp server...

If you're stepping through a test from pycharm or vscode, you can debug your test and also the fixture hooks.

We haven't seen how to mount the database schema at startup, nor how to clean up the database between each test. Fixtup hooks allow you to do this.

I plan to develop a fixtup plugin for sqlalchymie and another for django that will allow you to use a database without writing a specific hook.

• Fixtup
• Fixtup - Documentation
• Fixtup - Discord

Comments

[Lucy Linder]

I usually use testcontainers. It is a really mature library with built-in support for many well-known services such as kafka, postgres, mysql etc. There is also a "reuse" option and it is available for multiple languages (I use it extensively in java/kotlin).

Have you ever tried it? How does it compare to fixtup?

[Fabien Arcellier]

I didn't know it. I haven't test it so I will try to explain what is similar and what is different regards to the documentation I have read. Thanks for sharing.

Testcontainer implement clearly the promise of this blog post. The solution looks great and pretty mature. The solution is cross environment and that's awesome !

Fixtup is a solution to manage test environment based on directory. Every fixture is a directory. That's the major difference. Fixtup is designed arround the fact you will describe the environment you want in a directory, called a fixture. It can be configuration files, data files ... This environment is cloned in tmp directory during the test. It will be clean-up afterwards. The docker management is done by a plugin, not fixtup itself. Fixtup implements a plug-in to manage environment variables through .env file.

Testcontainers propose container ready to use as a function. That's avoid to learn the usage and specifity of docker compose.

To manage livecycle through test, you will have to rely on pytest fixture scope. Fixtup run with other test framework as unittest or behave and allow to use the keep_up policy to reuse same container. I am not sure testcontainers has implemented a similar policy.

[Lucy Linder]

Thanks for the analysis! As stated before, the keep_up policy is also available, called reuse in testcontainers :)

[Fabien Arcellier]

I think it's not developped yet in python. The issue is still open : github.com/testcontainers/testcont....

reuse is missing from source code itself. As many python projects rely on pytest, I think missing this feature is not critical because the lifecycle of a container in test container may be managed thanks to pytest fixture scope.