Building a MongoDB migration system for NestJS with mongoose

This tutorial is assuming:

• You have a working NestJS project
• You are using MongoDB and mongoose

In the recent past, I had an urge to do a bit of refactoring on my discord game.
Still a work in progress, but I couldn't stand anymore the fact that houses were named homes...
I already can hear you, 'just change labels displayed no one care!'.

I do care about the consistency of naming in my codebases. If homes are houses then the next thing you know is:
Canons will become wooden swords and wolves are dogs...

I spent some time online looking for solutions and I finally built something I like.
Let me present to you the result of my work.

I chose to use the migrate library since it is database agnostic, offers an easy up/down logic, and can store the migration status in any form.

Enough speaking about me, let me guide you through this journey.

Install migrate

Go on install the bad guy!

npm i --save migrate

Create a folder to store your migrations!

You will new two folders:

mkdir src/migrations
mkdir src/migrations-utils

The first one will store the update scripts and the seconds will store some utilities.
Let's look into the seconds.

Some little helpers

In the introduction, I told you that migrate is database agnostic.
Therefore you need to write a little mongodb connector:

import { MongoClient } from 'mongodb';
import { configs } from '../config/configuration';

const MONGO_URL = configs.mongoUrl;

export const getDb = async () => {
  const client: any = await MongoClient.connect(MONGO_URL, { useUnifiedTopology: true });
  return client.db();
};

Nice! let's keep going.

Migrate is a tool made in javascript.
And we use typescript, the best thing to do is have a little template with the database already connected.

import { getDb } from '../migrations-utils/db';

export const up = async () => {
  const db = await getDb();
  /*
      Code your update script here!
   */
};

export const down = async () => {
  const db = await getDb();
  /*
      Code you downgrade script here!
   */
};

I had some trouble with ts-node/register in migrate command line.
This little helper solved my transpilation errors!
Do the same! now! do it!

// eslint-disable-next-line @typescript-eslint/no-var-requires
const tsNode = require('ts-node');
module.exports = tsNode.register;

Update package.json

it's time for you to update the package.json of your project in order to have easy to use scripts for the future!

A script to generate migration files

Add this sweet line in the package.json, in the script section!

"migrate:create": "migrate create --template-file ./src/migrations-utils/template.ts --migrations-dir=\"./src/migrations\" --compiler=\"ts:./src/migrations-utils/ts-compiler.js\"",

--template-file ./src/migrations-utils/template.ts provide a template file, it's a necessary thing since we are in a typescript repo.
It also provides you an easy way to bootstrap migration just the way you like it!

--migrations-dir=\"./src/migrations\" Tell migrate where your migration scripts are stored.
By default, it's at the project root.

--compiler=\"ts:./src/migrations-utils/ts-compiler.js\" Explain to migrate how to handle typescript files.

Now, you just need to run this command to create an empty typescript migration file in the correct folder!

npm run migrate:create -- <migration name>

A script for upgrades and a script for downgrades

AAAAAAnd two more lines in the package.json, again in the scripts section!

"migrate:up": "migrate --migrations-dir=\"./src/migrations\" --compiler=\"ts:./src/migrations-utils/ts-compiler.js\" up",
"migrate:down": "migrate --migrations-dir=\"./src/migrations\" --compiler=\"ts:./src/migrations-utils/ts-compiler.js\" down"

No new options here, I already explained them but refreshing is nice.

--migrations-dir=\"./src/migrations\" Tells migrate where to find your migrations!

--compiler=\"ts:./src/migrations-utils/ts-compiler.js\" Tells migrate how to handle typescript...

You can now run update script: npm run migrate:up or downgrade script npm run migrate:down

What will happen when you run a migration?

Migrate will store your migration state in a file at the project root.
This file is called migrate.json.
It looks like this:

{
  "lastRun": "1605197159478-test.ts",
  "migrations": [
    {
      "title": "1605197159478-test.ts",
      "timestamp": 1605197181474
    }
  ]
}

DO NOT COMMIT migrate.json

Questions?

I'll be glad to answers questions in the comments.

If you liked my discord consider joining my coding lair!
☎️Webeleon coding lair on discord

You can also email me and offer me a contract 💰
✉️Email me!

And since I'm a nice guy, here, take this sample repo containing a working codebase!
🎁Get the code of the tuto from github

Documentation

• NestJS
• Install mongoose with nest
• Migrate

Comments

[J.D Nicholls]

Thanks for sharing!

I was wondering how to apply database migrations with Mongoose in a NestJS template but I figure out that I was trying to implement a Database seeding instead because you don't need migrations with a NoSQL Database. This is my template: github.com/proyecto26/MyAPI/tree/n...

Best
Juan

[Julien Prugne]

In production database you might need to apply migration on existing data even in document oriented database.
For example: you can clean some legacy data. (GDPR compliance for example)
Seeding is only for new data or starting a new environment.

[Bsn]

That's true, I see the use of it now thanks!

[Mohsen Heydari]

Thank you for sharing Julien!
I had a problem with relative paths in tsconfig.json and I managed to solve the issue by using the following code in 'ts-compiler.js':

const tsNode = require('ts-node');
const tsConfigPaths = require('tsconfig-paths');

module.exports = function(opts) {
    tsConfigPaths.register();
    return tsNode.register(opts);
};

[Pratik Rane]

Awesome! Thanks for this Julien. :)

[c-neri]

Thank you for sharing!
I have a question, where should i put this code?
const tsNode = require('ts-node');
module.exports = tsNode.register;

Thank you in advance!

[Julien Prugne]

Hello,

I use to place it in a file called ts-compiler.js in a folder called src/migration-utils
You can take a look in the sample repo.

Then you can use as an option in the migration command invocations: --compiler=\"ts:./src/migrations-utils/ts-compiler.js\"

It's in the sample repo, but I'll share the migration commands in the package.json here:

"migrate:create": "migrate create --template-file ./src/migrations-utils/template.ts --migrations-dir=\"./src/migrations\" --compiler=\"ts:./src/migrations-utils/ts-compiler.js\"",
"migrate:up": "migrate --migrations-dir=\"./src/migrations\" --compiler=\"ts:./src/migrations-utils/ts-compiler.js\" up",
"migrate:down": "migrate --migrations-dir=\"./src/migrations\" --compiler=\"ts:./src/migrations-utils/ts-compiler.js\" down"

Hope this nugget will help you :)