Stop Mocking Everything in Integration Tests: Why Testcontainers Changed My CI Strategy

#cicd #programming #devops

Every backend developer knows the classic trade-off when writing tests:

Unit tests with mocks are lightning fast, but they lie to you about database constraint violations, subtle SQL dialect quirks, and ORM mapping bugs.
End-to-end (E2E) tests catch real bugs, but they are notoriously flaky, require complex staging environments, and slow down CI pipelines.

For years, the standard compromise was using lightweight, in-memory alternatives for testing—like running H2 or SQLite in your test suite while production runs on PostgreSQL or MySQL.

This approach creates a false sense of security.

The In-Memory Database Trap

Running an in-memory substitute in tests while using a real relational database in production frequently introduces subtle bugs that bypass CI entirely:

JSON/JSONB Operators: Native JSON querying behaves completely differently across engine dialects.
Concurrency & Locking: Row-level locking (SELECT ... FOR UPDATE), transaction isolation levels, and deadlocks cannot be reliably reproduced on in-memory databases.
PostgreSQL Extensions: Features like pgvector, PostGIS, or custom full-text search indexes don't exist in H2 or SQLite.

Mocking your persistence layer doesn't test your database interaction—it tests your assumptions about how the database works.

Enter Testcontainers: Ephemeral Infrastructure in Code

Testcontainers flips this paradigm. Instead of relying on mocks or shared staging databases, it allows you to spin up real, disposable Docker containers directly from your test code (available in Node.js, Go, Python, Java, .NET, and more).

Here is what a true integration test looks like in TypeScript using Testcontainers and PostgreSQL:

import { PostgreSqlContainer, StartedPostgreSqlContainer } from '@testcontainers/postgresql';
import { Pool } from 'pg';

describe('User Repository Integration Test', () => {
  let container: StartedPostgreSqlContainer;
  let dbPool: Pool;

  // Spin up a real PostgreSQL 16 container before tests run
  beforeAll(async () => {
    container = await new PostgreSqlContainer('postgres:16-alpine')
      .withDatabase('test_db')
      .withUsername('test_user')
      .withPassword('test_pass')
      .start();

    dbPool = new Pool({
      connectionString: container.getConnectionUri(),
    });

    // Run database migrations against the real engine
    await runMigrations(dbPool);
  }, 30000);

  // Tear down the ephemeral container after tests complete
  afterAll(async () => {
    await dbPool.end();
    await container.stop();
  });

  it('should enforce unique email constraint on user insertion', async () => {
    const user = { email: 'dev@example.com', name: 'Alice' };

    await dbPool.query('INSERT INTO users(email, name) VALUES($1, $2)', [user.email, user.name]);

    // Test real PostgreSQL duplicate key error handling
    await expect(
      dbPool.query('INSERT INTO users(email, name) VALUES($1, $2)', [user.email, user.name])
    ).rejects.toThrow(/duplicate key value violates unique constraint/);
  });
});

Why This Pattern Scales

1. Zero Staging Pollution

Because every test run provisions its own ephemeral container on a random host port, tests run in complete isolation. You no longer suffer from dynamic test order dependencies or leftover dirty state in shared staging databases.

2. Multi-Service Integration

Testcontainers isn't limited to SQL. You can orchestrate multi-container setups inside a single test file:

Spin up Redis to verify cache invalidation logic.
Spin up Kafka/RabbitMQ to verify async event-driven consumers.
Spin up LocalStack to test AWS S3 uploads locally.

3. Deterministic CI Environments

Since the container image tag is explicitly defined in your test code (postgres:16-alpine), your local dev environment and your GitHub Actions / GitLab CI runner run on the exact same byte-for-byte engine.

Best Practices for CI Performance

Running real containers in CI comes with performance considerations. To keep build times fast:

Reuse Containers Across Test Suites: Enable container reuse (withReuse(true)) during local development so you don't pay the startup penalty on every code change.
Use Lightweight Base Images: Stick to minimal tags like alpine or slim variants to keep Docker image pull times down.
Run on Ramdisk (tmpfs): Mount the database data directory inside the container to tmpfs (/var/lib/postgresql/data) to bypass physical disk I/O bottlenecks during test runs.

Summary

In-memory database substitutes and heavy mocking were necessary compromises a decade ago. Today, containerized testing makes it seamless to validate your code against the exact technologies running in production.

If you aren't testing against real infrastructure, you aren't really integration testing!

DEV Community