Integration Testing Setup with Nestjs and MongoDB

Originall posted on my blog: https://blog.merckx.fr/integration-testing-setup-with-nestjs-and-mongodb/

Automated testing provides tremendous value when creating software. It enables you to follow a Test Driven Development approach: you can write your tests and design the API and its requirements upfront, then write the code to make the tests pass.

Very often comes the usual questions:

• do I create unit tests, integration tests or E2E tests?
• what should be mocked?
• what am I testing?

While many debates still go on, there are pretty clear rules of thumbs one should follow:

• E2E tests provide a very good answer to the question Can we release? but a very poor one to Why do the tests fail?. They are expensive to write.
• Unit tests do the exact opposite. They are very good to locate the origin of the failure but are bad to know if you can actually release if they all pass. They are effortless to write.
• Integration tests provide a good balance between the two. They are testing the integration points of the components and you can assume that, from one release to the other, if the interfaces between your components didn't change and your integration tests pass, the release can be deployed. They are cheap to write. Not as cheap as unit tests, but far cheaper than E2E tests.

This is why I recommend to focus on integration tests. When writing integration tests, you should always keep in mind that they should be easy to write. Mocking functionnality requires writing code and therefore should be avoided. You should aim at integrating as much as you can from the code that will actually run into your tests.

I use Nestjs in 90% of the projects I work on. In 90% of those projects, I use MongoDB as a database. Writing integration tests for a module should be as easy as writing unit tests without having to mock all the functionnalities around the tested module. Therefore, we must find a way to integrate a real Mongo database when running our integration tests. This is what is going to be presented in the next lines.

Do yourself a favor and create a friendly developer environment

We are going to use Docker to set our development environment up. You might not be friend with Docker but you better become. Docker is a friend that will enable the following:

• once set up, you can work offline. You don't need to host development databases in the cloud.
• you work with containers that you can later on use in production
• you can create many workflows that suit your needs

So go ahead and install Docker on your machine. You need five minutes to do that https://docs.docker.com/get-docker/ and then you can come back.

Containerize your application

At this point, you have Docker running on your machine and a Nestjs project that uses Mongo as a database. We are going to use Docker Compose to create several containers that run different services. The following docker-compose.yml file describes an environment with two services: api and mongo. mongo creates a container that runs a MongoDB server and api is a container that runs NodeJS 14 that exposes ports 3333 and 9229. That way, you can reach your Nest application and the debugger.

version: "3.9"
services:
  api:
    image: node:14
    volumes:
      - .:/app
    ports:
      - 3333:3333
      - 9229:9229
    depends_on:
      - mongo
    environment:
      NODE_ENV: development
  mongo:
    image: mongo:4.4.8
    volumes:
      - ./.docker/mongodb/data/db/:/data/db/
      - ./.docker/mongodb/data/log/:/var/log/mongodb/

You are almost fully set up. Let's describe a few useful workflows.

Start the things you need

We have configured containers. It's time to start and test our NestJS application. Save the following command (as a script in your package.json or as a shell script in your repository:

docker-compose run --service-ports -w /app api bash

This will start the services api and mongo and start a bash shell. In this shell, you can use NPM and Node the same way you would do it without Docker! Try your usual commands npm install and npm start and npm test

I usually save this command in my package.json under the start:env script:

{
  ...
  "scripts": {
    "start:env": "docker-compose run -w /app api bash",
    ...
  }
}

Run the tests once

If you want to run tests in your CI, use the following command:

docker-compose run -w /app api npm run test

where the test script is for instance:

npm install && jest

Write integration tests

Imagine you have a NestJS application with a PostModule that has a controller PostController that has two API endpoints:

• GET /post: returns all the posts stored in the database
• POST /post: creates a new post and stores it in the database

This module also has a service PostService that PostController instantiates and uses. The service accesses the database. To test your module, one can create tests for the PostService and the PostController. The controller doesn't directly access the database and needs the service to do that. When writing tests for the controller, you have the following choices:

1. you can mock the service completely
2. you can mock the database access
3. you can use the real implementation of the whole module, therefore test the integration of its components: the controller, the service, the database access

We are going for the solution 3. For each integration test, we are going to create a completely new database and delete it when we are finished. Have a look at the following spec file:

import { getConnectionToken, MongooseModule } from '@nestjs/mongoose';
import { NestExpressApplication } from '@nestjs/platform-express';
import { Test } from '@nestjs/testing';
import { Connection } from 'mongoose';
import * as supertest from 'supertest';
import { IPost } from '../model/post';
import { PostModule } from '../post.module';

describe('PostController', () => {
  let app: NestExpressApplication;

  const apiClient = () => {
    return supertest(app.getHttpServer());
  };

  beforeEach(async () => {
    const moduleRef = await Test.createTestingModule({
      imports: [
        MongooseModule.forRoot('mongodb://mongo:27017', { dbName: 'test' }), // we use Mongoose here, but you can also use TypeORM
        PostModule,
      ],
    }).compile();

    app = moduleRef.createNestApplication<NestExpressApplication>();
    await app.listen(3333);
  });

  afterEach(async () => {
    await (app.get(getConnectionToken()) as Connection).db.dropDatabase();
    await app.close();
  });

  it('creates a post', async () => {
    await apiClient()
      .post('/post')
      .send({
        content:
          'I am all setup with Nestjs and Mongo for more integration testing. TDD rocks!',
      })
      .expect(201);
    const posts: IPost[] = (await apiClient().get('/post')).body;
    expect(posts[0].content).toBe(
      'I am all setup with Nestjs and Mongo for more integration testing. TDD rocks!',
    );
    expect(posts[0].likes.length).toBe(0);
  });
});

A few notes:

• we do not mock anything
• we test the whole PostModule
• beforeEach and afterEach are our hooks to create and delete databases. We specify to Mongoose which database name to use and we drop the database with (app.get(getConnectionToken()) as Connection).db.dropDatabase()
• we use supertest to create a consumer for our API
• we only add our module PostModule to the app
• if you would to start the whole application, you would be very close to an actual End-to-end test

I believe that you are set up right now. You have at your disposal an environment where you can write as many integration tests as you can. You can now adopt a TDD approach and deploy a fully tested API.

Note: you can find an example of the setup we described at https://github.com/kevinmerckx/nesjts-with-mongo.

KM

Photo by Glen Carrie on Unsplash

Resources

• An example to setup your repository
• TDD
• End-to-End Testing vs Integration Testing
• Unit vs Integration vs System vs E2E Testing
• Unit, Integration and End-To-End Tests - Finding the Right Balance
• The different types of software testing