Improving Integration/E2E testing using NestJS and TestContainers

Introduction

Testing javascript based services in backend context is a required practise to ensure the consistency of our application, however testing the integrity with other services is most of the time done by mocking returned values of methods or by using some automated scripts or tests databases inside piplines for example.

Today we are going to walk you through this tutorial on how you can improve these kind of test cases using TestContainers

Setup

First we start by creating a Nest.js application :
1 - Under command line execute npm i -g @nestjs/cli to add Nest CLI
2- Create Nest project nest new project-name

3- Follow the steps and create your project
4- For this project we choose to use Prismaas our ORM , so under our directory run
npm install prisma @prisma/client --save-dev

Now we should be able to start Coding !!

Getting Started

Using command line run npx prisma init
This should create a prisma/schema.prisma file in your root folder
inside that folder we will create our entity Car

generator client {
  provider = "prisma-client-js"
  output   = "./generated/prisma-client-js"
}

datasource db {
  provider = "postgresql"
  url      = env("DATABASE_URL")
}


model Car {
  id    Int     @default(autoincrement()) @id
  model  String  @unique
  color  String?
}

No go to your .envfile and add your DATABASE_URL
then go to your command line and run

 $ npx prisma migrate dev --name init

A migration file should be generated under /primsa/migrations

Creating our Resource

After setting up our database now we can start generating our resource in Nestjs app
using CLI provided from Nestjs run

$ nest generate resource

Choose the name Carfor the resource then pick REST API

your resource should be generated under src

After that we should align our DTO's and Entitieswith the right types.
Hopefully Prisma client provides us with the generated types automatically from the generated model so the files should be aligned this way

//create-car.dto.ts
import { Prisma } from "@prisma/client";

export type CreateCarDto = Prisma.CarCreateInput;

//update-car.dto.ts
import { Prisma } from '@prisma/client';

export type UpdateCarDto = Prisma.CarUpdateInput

//car.entity.ts
import { Prisma } from "@prisma/client";

export type Car =  Prisma.CarSelect

Now let's create our PrismaService.
Under project directory run

$ nest generate service prisma 

No go to Prisma.service.ts and add these lines

import { Injectable, OnModuleInit } from '@nestjs/common'
import { PrismaClient } from '@prisma/client'

@Injectable()
export class PrismaService extends PrismaClient
    implements OnModuleInit {

    async onModuleInit() {
        await this.$connect();
    }

}

let's now create our carService

//cars.service.ts
import { Injectable } from '@nestjs/common';
import { Car as CarModel, Prisma } from '@prisma/client'
import { PrismaService } from '../prisma/primsa.service';

@Injectable()
export class CarsService {

  constructor(private readonly prismaService: PrismaService) { }
  async create(data: Prisma.CarCreateInput): Promise<CarModel> {
    try {
      const car = await this.prismaService.car.create({ data })
      return car;
    } catch (error) {
      throw error
    }
  }

  async findAll(): Promise<CarModel[]> {
    try {
      return await this.prismaService.car.findMany()
    } catch (error) {
      throw error
    }
  }

  async findOne(where: Prisma.CarWhereUniqueInput) {
    try {
      return await this.prismaService.car.findUnique({ where })
    } catch (error) {
      throw error
    }
  }

  async update(where: Prisma.CarWhereUniqueInput, data: Prisma.CarUpdateInput) {
    try {
      return await this.prismaService.car.update({ data, where })
    } catch (error) {
      throw error
    }
  }

  async remove(where: Prisma.CarWhereUniqueInput) {
    try {
      return await this.prismaService.car.delete({ where })
    } catch (error) {
      throw error
    }
  }
}

and now our controller

import { Controller, Get, Post, Body, Patch, Param, Delete } from '@nestjs/common';
import { CarsService } from './cars.service';
import { CreateCarDto } from './dto/create-car.dto';
import { UpdateCarDto } from './dto/update-car.dto';

@Controller('cars')
export class CarsController {
  constructor(private readonly carsService: CarsService) { }

  @Post()
  create(@Body() createCarDto: CreateCarDto) {
    return this.carsService.create(createCarDto);
  }

  @Get()
  findAll() {
    return this.carsService.findAll();
  }

  @Get(':id')
  findOne(@Param('id') id: string) {
    return this.carsService.findOne({ id: Number(id) });
  }

  @Patch(':id')
  update(@Param('id') id: string, @Body() updateCarDto: UpdateCarDto) {
    return this.carsService.update({ id: Number(id) }, updateCarDto);
  }

  @Delete(':id')
  remove(@Param('id') id: string) {
    return this.carsService.remove({ id: Number(id) });
  }
}

And finally we should add PrismaService to cars.module.ts

import { Module } from '@nestjs/common';
import { CarsService } from './cars.service';
import { CarsController } from './cars.controller';
import { PrismaService } from '../prisma/primsa.service';


@Module({
  controllers: [CarsController],
  providers: [CarsService, PrismaService],
})
export class CarsModule { }

Before start testing we can try to test some endpoints,thus we can use some tools like Postman

Everything works fine

Setup TestContainers for testing

In order to start working in our tests we should first install testContainers

$ npm install --save-dev @testcontainers/postgresql testContainers

TestContainers is a npm dependency that allow us to run database images programatically inside docker containers, these databases are lighweight and super efficent for tests

NOTE: for this tutorial you need to have docker already installed

In order to start testing let's create a Setup file for our tests.
Inside /test create a file setupTests.e2e.ts.

NOTE: The idea of this file is to setup a database for our test environement and initialize a Prisma instance that will be used inside our tests. This database will be created just before running our tests and then will be stopped and unmounted once all tests are executed.
This way we make sure that we connect only once to our database and we will get rid of extra anoying lines of code inside our tests.

import { PostgreSqlContainer, StartedPostgreSqlContainer } from '@testcontainers/postgresql';
import { Client } from 'pg';
import { PrismaService } from '../src/prisma/primsa.service';
import { execSync } from 'child_process';

let postgresContainer: StartedPostgreSqlContainer;
let postgresClient: Client;
let prismaService: PrismaService;

beforeAll(async () => {
  //connect our container 
  postgresContainer = await new PostgreSqlContainer().start();

  postgresClient = new Client({
    host: postgresContainer.getHost(),
    port: postgresContainer.getPort(),
    database: postgresContainer.getDatabase(),
    user: postgresContainer.getUsername(),
    password: postgresContainer.getPassword(),
  });

  await postgresClient.connect();
   //Set new database Url 
  const databaseUrl = `postgresql://${postgresClient.user}:${postgresClient.password}@${postgresClient.host}:${postgresClient.port}/${postgresClient.database}`;
    // Execute Prisma migrations
    execSync('npx prisma migrate dev', { env: { DATABASE_URL: databaseUrl } });
//Set prisma instance
  prismaService = new PrismaService({
    datasources: {
      db: {
        url: databaseUrl,
      },
    },
    log: ['query']

  },
  );
  console.log('connected to test db...');
})

afterAll(async () => {
//Stop container as well as postgresClient 
  await postgresClient.end();
  await postgresContainer.stop();
  console.log('test db stopped...');
});
// add some timeout until containers are up and working 
jest.setTimeout(8000);
export { postgresClient, prismaService };

Now go to jest.e2e.ts and set our config

{
  "moduleFileExtensions": ["js", "json", "ts"],
  "rootDir": ".",
  "roots": ["../src"],
  "setupFilesAfterEnv": ["./setupTests.e2e.ts"],
  "testEnvironment": "node",
  "testRegex": ".e2e-spec.ts$",
  "transform": {
    "^.+\\.(t|j)s$": "ts-jest"
  }
}

Now let's try to create a test case for our car service
we need to make sure that SQL query to the database are performing in the right way.
create a file called car.e2e-spec.ts under /src/cars

import { Test, TestingModule } from '@nestjs/testing';
import { CarsService } from './cars.service';
import { PrismaService } from '../prisma/primsa.service';
import { postgresClient, prismaService } from '../../test/setupTests.e2e';


describe('CarsService', () => {
  let service: CarsService;

  beforeEach(async () => {
    const module: TestingModule = await Test.createTestingModule({
      providers: [CarsService, PrismaService],
    }).overrideProvider(PrismaService)
      .useValue(prismaService)
      .compile();

    service = module.get<CarsService>(CarsService);
  });

});

First using the createTestModule utility we should override the Prisma instance because currenty createTestingModule is using the real PrismaServicethat connects to the real database, and for that we need to import the new Prisma instance from ../../test/setupTests.e2e then using :

.overrideProvider(PrismaService)
      .useValue(prismaService)
      .compile();

We inject our new PrismaServiceto override the real Prisma.

Now let's create our first test.
So we will try to test the create method inside our service and here are the testing steps:

1- insert into the database using the createmethod

2- perform a SELECT query for the newly created car
3- test the result using jest assertion

it('should create a car', async () => {
    // Start a transaction
    await postgresClient.query('BEGIN');

    try {
      // Perform the create operation
      const createResult = await service.create({ model: "mercedes", color: "red" });

      // Commit the transaction
      await postgresClient.query('COMMIT');

      // Query the database for the newly created car
      const result = await postgresClient.query('SELECT * FROM "public"."Car"');

      // Log the results
      console.log(result.rows);

      // Assert the create result
      expect(createResult).toEqual({
        id: 1,
        model: "mercedes",
        color: "red"
      });
    } catch (error) {
      // Rollback the transaction in case of an error
      await postgresClient.query('ROLLBACK');
      throw error;
    }
  });

Now under our project directory run

$ npm run test:e2e

Once the command is running we can see that the containers are created and started automatically

And bingo our tests are working !!

Now let's create a test for our controller

import { Test, TestingModule } from '@nestjs/testing';
import { CarsController } from './cars.controller';
import { CarsService } from './cars.service';
import { PrismaService } from '../prisma/primsa.service';
import { prismaService, postgresClient } from '../../test/setupTests.e2e';
import * as request from 'supertest';
import { INestApplication } from '@nestjs/common';

describe('CarsController', () => {
  let controller: CarsController;
  let app: INestApplication;
  beforeEach(async () => {
    const module: TestingModule = await Test.createTestingModule({
      controllers: [CarsController],
      providers: [CarsService, PrismaService],
    }).overrideProvider(PrismaService)
      .useValue(prismaService)
      .compile();

    controller = module.get<CarsController>(CarsController);
    app = module.createNestApplication();
    await app.init();
  });

  it('should be defined', () => {
    expect(controller).toBeDefined();
  });

  it('should create a car', async () => {
    const createCarDto = { model: 'Mercedes', color: 'Red' };

    const response = await request(app.getHttpServer())
      .post('/cars')
      .send(createCarDto)
      .expect(201); 

    expect(response.body).toEqual({
      id: 1,
      model: 'Mercedes',
      color: 'Red',
    });
  });

});

For our controller we tried to use SuperTest in order to perform a real http request to the endpoint POST: /car that points to the method create()

And here our test passing.
For the rest of the CRUD operation it will follow the same strategy for services and also controllers

Summary

In this workaround we tried to show you how to work with TestContainers which is a greate tool for backend applications that can be used for integration tests as well as E2E and makes our tests more consistent and more real without going through mocks or third party tools

Hope you like this tutorial!
Any questions please let us know in your comments ?
Cheers !!