DEV Community

Sebastian Chikán
Sebastian Chikán

Posted on

How to Run Jest Integration Tests in Parallel Using Isolated SQL Schemas

#javascript #testing #performance #jest

In one of our TypeScript/Node.js services, most of our confidence comes from integration tests, with a deliberately thin layer of unit tests applied only where business logic is sufficiently complex to require focused verification. Tests are executed using Jest, backed by a Postgres instance running in Docker as the test database, with Prisma providing the data access layer.

Most of the service's responsibility lies in communication rather than decision-making: maintaining SSE streams, handling HTTP requests, and relaying data to other internal services.

At Dakai.io, we initially ran our integration tests in parallel without clearing the database, relying instead on practical isolation by creating test data with unique UUIDs. This allowed tests to execute safely at the same time - up to a point.

Why not clear the database?

When running tests with Jest in parallel, test files can execute concurrently, while tests within the same file run sequentially. If multiple test files operate on the same tables, clearing the database between tests can cause one test to accidentally remove data that another test is still using, leading to intermittent and hard-to-debug failures.

This is why simply wiping tables between tests isn't always a safe solution.

Tests with UUIDs

To address this, we generate unique UUIDs for all test data. Each test operates on its own records, preventing interference with other parallel tests and making it safe to run tests concurrently without wiping the tables.

The following codes are very simple examples to illustrate the concept - these are not full production-grade integration tests.

it('getUserById returns the proper user', async () => {
  // unique ID for this test
  const testId = uuidv4();

  // unique name to make sure the user we retrieve
  // is exactly the one created in this test
  const testUserName = `Test User ${uuidv4()}`;

  await createTestUser({ id: testId, name: testUserName });

  const user = await getUserById(testId);

  expect(user.name).toEqual(testUserName);
});
Enter fullscreen mode Exit fullscreen mode

This approach works perfectly for tests that create and retrieve individual records. However, it breaks down when testing endpoints like listUsers, which return global entities that cannot be scoped or filtered by UUID. In such cases, tests can interfere with each other, and parallel execution may lead to flaky results - like in this one:

it('listUsers returns all the users', async () => {
  const testUserName1 = `Test User1 ${uuidv4()}`;
  const testUserName2 = `Test User2 ${uuidv4()}`;

  await createTestUser({ id: uuidv4(), name: testUserName1 });
  await createTestUser({ id: uuidv4(), name: testUserName2 });

  const usersResponse = await listUsers();

  const userNames = usersResponse.map(u => u.name);
  expect(userNames.length).toEqual(2);
  expect(userNames).toContain(testUserName1);
  expect(userNames).toContain(testUserName2);
});
Enter fullscreen mode Exit fullscreen mode

At this point, if we want fully deterministic results with strong assertions, there is effectively no alternative to resetting the relevant tables before each test run.

The hard decision

Matrix

As we saw earlier, cleaning the tables while running tests in parallel doesn't work well on a shared database. This seems to leave us with two unappealing options: run tests sequentially, or weaken our assertions so they no longer require a clean database.

Jest Under the Hood: Running Tests in Parallel

When running Jest tests in parallel, the number of workers is determined based on the configuration and/or the number of CPU cores available on the host machine.

Each worker runs in its own separate Node.js process and handles one test file at a time, so the tests within a single file are executed sequentially, as we discussed earlier.

This setup allows Jest to fully utilize the available CPU cores, but it also introduces a challenge: multiple workers can access the same database tables simultaneously, which may lead to interference between tests if the data isn't properly isolated.

Solution: separate DB schema for each worker

Big picture of the process

By creating a separate schema for each worker and configuring them to read and write only to their assigned schema, we can ensure an isolated environment where tests don't interfere with each other. Each worker still runs the tests in its own process sequentially, so parallel execution across workers is safe and isolated. This may sound more complicated than it actually is.

First we needed to create the setup that starts the database and calculates the number of workers that will be needed. This can be easily done since we use maxWorkers = 50% and the exact implementation is available.

Even if you rely on Jest's default worker calculation, this setup still works the same way - you only need to understand how many workers will be spawned so you can create the corresponding schemas.

After this we create a schema for each worker (for example: test_worker_1) and run the migrations on those schemas. This setup/migration has to be run only once.

const os = require('os')
const { Client } = require('pg');
const path = require('path');
const { execSync } = require('child_process');
const fs = require('fs');
const { v2: dockerCompose } = require('docker-compose')
const waitOn = require('wait-on')

const repoRoot = path.resolve(__dirname);
const delay = (ms) => new Promise((res) => setTimeout(res, ms))

function loadJestConfig() {
  const configPathJS = path.join(repoRoot, 'jest.config.js');
  const configPathTS = path.join(repoRoot, 'jest.config.ts');

  let config;

  if (fs.existsSync(configPathJS)) {
    config = require(configPathJS);
  } else if (fs.existsSync(configPathTS)) {
    config = require(configPathTS);
  } else {
    throw new Error('Cannot find jest.config.js or jest.config.ts');
  }

  return config;
}

/*
  Jest calculates the number of workers (parallel test processes) based on CPU cores:
    - If maxWorkers is a number → use that value directly.
    - If maxWorkers is a percentage (e.g. "50%") → use that percentage of available CPU cores, rounded down.
    - Otherwise (default) → workers = CPU core count.
  Reference:
  https://github.com/facebook/jest/blob/main/packages/jest-worker/src/index.ts#L102-L110
*/
function getWorkerCount(jestConfig) {
  if (jestConfig.maxWorkers) {
    if (typeof jestConfig.maxWorkers === 'number') return jestConfig.maxWorkers;

    if (typeof jestConfig.maxWorkers === 'string') {
      if (jestConfig.maxWorkers.endsWith('%')) {
        // e.g. '50%'
        const pct = Number(jestConfig.maxWorkers.replace('%', ''));
        const cores = os.cpus().length;
        return Math.max(1, Math.floor((pct / 100) * cores));
      }

      return Number(jestConfig.maxWorkers);
    }
  }

  // fallback: number of CPU cores
  return require('os').cpus().length;
}

const baseUrl =
  process.env.DATABASE_URL_BASE ||
  'postgres://postgres:root@127.0.0.1:5433/postgres';

async function startDb() {
  await dockerCompose.upOne('test-db', {
    config: '../../docker-compose-dev-dependencies.yml'
  })
  await waitOn({ resources: ['tcp:127.0.0.1:5433'], timeout: 30_000 })
  // unfortunately even if the port 5432 is open postgres is still not able to run queries, so we wait an arbitrary amount (1s)
  await delay(1000)
}

async function createSchema(schemaName) {
  const client = new Client({ connectionString: baseUrl });

  try {
    await client.connect();
    await client.query(`CREATE SCHEMA IF NOT EXISTS "${schemaName}";`);
  } finally {
    await client.end();
  }
}

function runPrismaFor(schema) {
  process.env.DATABASE_URL = `${baseUrl}?schema=${schema}`;

  console.log(`\n🟦 Migrating schema: ${schema}`);
  execSync(`npx turbo run prisma:migrate --filter @api/storage`, {
    stdio: 'inherit',
    cwd: repoRoot,
  });

  console.log(`🟩 Seeding schema: ${schema}`);
  execSync(`npx turbo run prisma:seed --filter @api/storage`, {
    stdio: 'inherit',
    cwd: repoRoot,
  });
}

async function main() {
  console.log('\n=== Jest Worker Schema Setup ===');

  await startDb()

  const jestConfig = loadJestConfig();
  const workerCount = getWorkerCount(jestConfig);

  console.log(`Detected Jest workers: ${workerCount}\n`);

  for (let i = 1; i <= workerCount; i++) {
    const schema = `test_worker_${i}`;
    console.log(`\n🔧 Setting up schema: ${schema}`);

    await createSchema(schema);
    runPrismaFor(schema);
  }

  console.log('\n✅ All worker schemas ready.\n');
}

main().catch((err) => {
  console.error('❌ Setup failed:', err);
  process.exit(1);
});
Enter fullscreen mode Exit fullscreen mode

The Database and its schemas are ready, but how will each worker know which schema to reach out to?

In the Jest config, we need to introduce a setup to ensure each worker is assigned the correct schema.

setupFiles: ['./jest-setup.js'],
Enter fullscreen mode Exit fullscreen mode

SetupFiles runs once for each worker after its process is initiated, but before any tests are executed. Here is what is inside jest-setup.js:

const isCI = require('is-ci')
if(!isCI) { // on CI we fall back to sequential tests!
  process.env.DATABASE_URL = `postgres://postgres:root@127.0.0.1:5433/postgres
    ?schema=test_worker_${process.env.JEST_WORKER_ID}`
}
Enter fullscreen mode Exit fullscreen mode

Each Jest worker runs in its own Node.js process, with its own copy of process.env, so setting a different DATABASE_URL in each worker keeps the database configuration completely isolated per process.

The most important thing to look out here is the JEST_WORKER_ID environment variable. This is the ID of the given worker and since we used to create schemas like test_worker_1 … test_worker_n this way we can tell each worker which schema to connect to via the DATABASE_URL environment variable that is used by Prisma.

⚠️ Important: this solution is for running local only, but also could work on CI/CD pipelines that have a strong machine under them. The is-ci package is used here to detect if the test was run in CI/CD and to skip this part. Most CI/CD machines are relatively weak, so the recommended approach there is to run tests sequentially. Read more about Jest runner performance.

Conclusion

In this article, we explored the challenges of running Jest integration tests in parallel. Using unique UUIDs works well for individual records, but when dealing with global entities, tests can interfere with each other. By implementing a separate schema for each worker, we can create an isolated environment where tests don't interfere, allowing safe parallel execution.

The key takeaway: proper isolation is essential for reliable parallel tests, and understanding how Jest workers interact with your database can save you from flaky test results.

Give this setup a try in your own projects!

Top comments (0)