Isolating Integration Tests

When we write integration tests at Super we follow some principles:

1. Each test should work independently of any other test
2. It should only act and assert at the boundary of the service (i.e. how the service is interacted with by other services or users)

In practice this means we have to do some work to manage isolation (or lack of state-bleed) between each test.

(Note: if you want to skip to a working code example here is an example repo: https://github.com/sam-super/example-db-test-isolation)

When thinking about isolation it's handy to have a good mental model of how tests are executed in the most popular testing frameworks:

This means our test suites run in parallel, but (by default) each test in the suite run in sequence). So it's important that we use this model to make sure our tests can run in parallel and stay isolated.

Below is an example of testing a simple fastify app that has a DB with a single table for cars:

So for example:

import {FastifyInstance} from "fastify";
import knexFactory, {Knex} from "knex";
import {$} from 'execa';
import {buildApp} from "../src/app";
import {expect} from 'expect';
import {Client} from "pg";
import {randomString} from "./helpers/utils";

async function startContainers() {
  await $`docker compose up --remove-orphans --wait -d postgres`;
}

async function initDb(testSpecificDbName: string) {
  const start = Date.now();

  const connOpts = {
    // these need to match your docker-compose.yml
    host: '127.0.0.1',
    port: 54323,
    password: 'whatever',
    user: 'root',
  };
  const client = new Client({
    database: 'postgres',
    ...connOpts,
  });
  await client.connect();
  await client.query('CREATE DATABASE ' + testSpecificDbName);
  await client.end();

  const knex = knexFactory({
    client: 'pg',
    connection: {
      ...connOpts,
      database: testSpecificDbName,
    },
    migrations: {
      directory: __dirname + '/../src/migrations',
      tableName: 'knex_migrations',
    }
  });

  await knex.migrate.latest();

  console.log(`Created DB ${testSpecificDbName} (in ${(Date.now() - start) / 1000}s)`);
  return knex;
}

describe('cars api', function () {
  let app: FastifyInstance;
  let knex: Knex;

  before(async function () {
    await startContainers();
  });
  beforeEach(async function () {
    process.env.DB_NAME = `test_${Date.now()}_${randomString()}`;
    knex = await initDb(process.env.DB_NAME);
    app = buildApp(knex);
  });
  afterEach(async function () {
    await app.close();
    await knex.destroy();
  })

  it('can get a car it creates', async function () {
    const postRes = await app
      .inject()
      .post('/cars')
      .headers({'content-type': 'application/json'})
      .body({make: 'ford'});
    expect(postRes.statusCode).toEqual(200);
    expect(postRes.json()).toEqual({make: 'ford'});

    const getRes = await app
      .inject()
      .get('/cars')
      .headers({'content-type': 'application/json'});
    expect(getRes.statusCode).toEqual(200);
    expect(getRes.json()).toEqual([{make: 'ford'}]);
  });

  it('gets no cars if none created', async function () {
    const getRes = await app
      .inject()
      .get('/cars')
      .headers({'content-type': 'application/json'});
    expect(getRes.statusCode).toEqual(200);
    expect(getRes.json()).toEqual([]);
  });
});

What is this doing:

1. once, before any tests run, we start our containers (postgres - see docker compose file in GH repo)
2. before each test we create a new database with a random name and run the knex migrations (which creates the 'cars' table).
3. create a new instance of the fastify app (which is bound to the random db name/instance)
4. make requests to our api using the inbuilt .inject() method to issue http requests to fastify (without having to start a http server)
5. we can see the second test doesn't have the state (db row) created in the first test (otherwise it would fail)

In future articles we can go into more depth on how to optimize our tests and how we work with isolation when using localstack (dynamo, sqs queues etc).

FAQs

Why not just re-use the same database?

We could re-use the same db for each test and truncate the data between each test. We have to be careful tho, because, although in our example we have a single test file/suite, in practice we want our suites to be able to run in parallel. When we have many test-suites, it is advantageous to re-use databases on a per-thread basis (to save the time to create/migrating the DB) and then just truncate the tables between tests.

Isn't this slow?

On an M1 Macbook it's about 10ms to create each DB and run the migrations. It's only 1 migration, and as the number grows so will the time. However, for us it's worth the trade of for what we are trying to achieve and the guaranteed isolation it gives us.
There are also a few strategies to speed it up:

1. As above we can re-use databases per-worker-thread
2. we can maintain a single sql dump file (generated from the migrations themselves) and use it populate the databases on creation (rather than running each individual migration).

Shouldn't you be deleting the DBs and removing the containers at the end?

Re-using the containers speeds up our tests (postgres is fast to start, but it helps a lot if you have something like localstack which takes a while to start). Since our tests are isolated, it shouldn't matter what state we leave lying around on our containers. Eventually we could fill up the disk with all our test DBs, but that will take a long time and is easily fixed by killing our containers: docker compose down -v. Then next test run will bring up clean containers.

There is also the added benefit of having the DB left around after each test to manually inspect after a failed test run.