DEV Community

Testing Node.js + Mongoose with an in-memory database

Paula Santamaría on October 26, 2019

Last few weeks I've been working on creating Unit Tests for a Node.js and Mongoose application where most of the logic is handled by mongoose and M...

Read full post

yurigiovani • Oct 27 '19 • Edited

Nice way to implement tests.
I will try this way.
I've used mongo in docker container to perform my local tests.
Same in ci/cd.
Do you tried this way in ci/cd, like bitbucket pipelines?
Thanks for sharing.

Paula Santamaría • Oct 27 '19

Yes, one of the first things I tried was executing this tests in a CI pipeline. I was worried to find any unexpected issues, but it worked perfectly!
For CI pipelines I'd recommend using mongodb-memory-server-core and a mongodb + node.js docker image. That way the npm install will not download mongod binaries but use the ones already installed instead.

manuel114 • May 19 '20

Hey, I've been struggling to get mongodb-memory-server to work on Circle Ci, any chance you could share your CircleCi config? I keep getting this error when running tests on the CI pipeline:

Error: Status Code is 403 (MongoDB's 404)
This means that the requested version-platform combination dosnt exist

This is my db configuration:

const mongoose = require('mongoose');
const { MongoMemoryServer } = require('mongodb-memory-server');

const mongod = new MongoMemoryServer();

/**
 * Connect to the in-memory database.
 */
module.exports.connect = async () => {
    const uri = await mongod.getUri();

    const mongooseOpts = {
        useNewUrlParser: true,
        autoReconnect: true,
        reconnectTries: Number.MAX_VALUE,
        reconnectInterval: 1000,
    };

    await mongoose.connect(uri, mongooseOpts);
};

Paula Santamaría • May 20 '20

🤔 looks like MongoMemoryServer is trying to download a binary for the default MongoDB version that is compatible with the OS running on your CircleCI server/container but can't find it.

I've never worked with CircleCI but I'd recommend you to check if the MongoDB version that MongoMemoryServer is trying to download is available for your OS & architecture.
Here's the code where the download URL for the binaries is generated, in case you want to check that out: MongoBinaryDownloadUrl.

Maybe you can set a specific version of MongoDB that exists for your platform, like this:

    mongod = await MongoMemoryServer.create({ binary: { version: '4.2.6' } });
    const uri = await mongod.getConnectionString();

Another way to go would be to activate debug mode so you can get more info on the error. Just set the debug option to "1" in the package.json config section. More info here

Kowshik Sundararajan • Jul 2 '20

Thanks for the suggestion, Paula. I found that for my circleci build, specifying a version like 4.2.6 did not work but specifying latest worked.

Hope that helps!

Jared • Oct 27 '19

This is a very smart method of testing mongodb. much better than my code lol. Time for a refactor!

Paula Santamaría • Oct 27 '19

Thank you! I've had great results using this method so far. Please share your experience if you try it!

Aston Yao • Apr 30 '20

awesome. thanks for sharing this.

I'd also check

const savedProduct = await productService.create(productComplete)
expect(savedProduct._id).toBeDefined()
expect(savedProduct.name).toBe(productComplete.name)

The Stoic Developer • Nov 24 '20

Hi, nice article. I would just like to remind people a few things regarding mongodb-memory-server, please someone correct me if I'm wrong:

1) it's faster (~3x) for a battery of parallel tests, because when you use a real database you're usually constrained to one process (github.com/nodkz/mongodb-memory-se...)

2) it's slower (~4x) for one off tests, because it has to create the database engine every time. This is how I code, I always have one test running many times while TDDing (based on my own testing).

3) it is has somewhat the same speed in the other cases (based on my own testing). Please remember that mongo >= 3.2 runs with a memory cache by default.

docs.mongodb.com/manual/core/wired...

tperrin • Jul 15 '21 • Edited

Thanks for this awsome article. But since mongodb-memory-server version 7.0.0 we can't use anymore const mongoServer = new MongoMemoryServer(); but must use instead const mongo = await MongoMemoryServer.create(); . But I didn't succeed to adapt this to your code since that make me run await at file top level so I got the error "SyntaxError: await is only valid in async function". How would do you migrate your code to version 7?

Emma Walker • Jul 18 '21 • Edited

import mongoose from "mongoose";
import { MongoMemoryServer } from "mongodb-memory-server";

let mongo: MongoMemoryServer;

/**
 * Connect to the in-memory database.
 */
export const connect = async () => {
    mongo = await MongoMemoryServer.create();
    const uri = mongo.getUri();

    const mongooseOpts = {
        useNewUrlParser: true,
        useUnifiedTopology: true,
    };

    await mongoose.connect(uri, mongooseOpts);
}

/**
 * Drop database, close the connection and stop mongod.
 */
export const closeDatabase = async () => {
    await mongoose.connection.dropDatabase();
    await mongoose.connection.close();
    await mongo.stop();
}

/**
 * Remove all the data for all db collections.
 */
export const clearDatabase = async () => {
    const collections = mongoose.connection.collections;

    for (const key in collections) {
        const collection = collections[key];
        await collection.deleteMany({});
    }
}

tperrin • Jul 18 '21 • Edited

of course, simply move MongoMemoryServer.create into the connect method, I should have find this.... Thanks a lot!

Paula Santamaría • Aug 11 '21

Thanks Emma!

Keith Alleman • Nov 13 '19

Excellent write up. I've been working on a MongoDB/GraphQL project to learn with, and have been getting familiar with Jest over the last couple of days. I'm going to try implementing your db-handler tonight!

Paula Santamaría • Jul 15 '20

Thanks, Keith! How did that go?

Georgi Marokov • Feb 19 '20 • Edited

I get this error when using .deleteMany with TS:
Expected 1-3 arguments, but got 0.ts(2554) index.d.ts(945, 16): An argument for 'filter' was not provided.

So I called it with empty obj: await collection.deleteMany({});

parveen99 • Apr 19 '21

Hey amazing article, I am doing my first assignment using node JS express and Mongo DB as a developer and doing Unit testing for the very first time as a newbie developer. This was the most effective article I found. Thank you, Paula. Also how should I run individual tests, npm test runs it on the whole right ? I

Paula Santamaría • Apr 20 '21 • Edited

In glad my article was helpful 😊.

If you follow this guide, npm test will execute all the tests in the "tests" folder (as specified in the package.json). You can however add some parameters (like --testNamePattern) when running npm test to execute only one test. I'd recommend reading my article Mastering NPM Scripts to learn how pass parameters to an npm script.

Also, if your using VSCode, checkout the Jest extension. It let's you run and debug tests individually.

Jim Lynch • Jul 5 '20

Nice article Paula!

I kind of feel like there could be more assertions in your final it block though than just, "expect ... to not throw".

To me it would be great if there was some was to look inside the mongo memory lib and say something like, "expect the fake mongoDb's collection to now have that additional document that I inserted".

Paula Santamaría • Jul 15 '20

Thanks!
I completely agree. I actually added an extra test to check if the product exists after creating it on the repo, but didn't include it here because I wanted to keep the examples for the article simple.

ItBits2 • Sep 5 '20

I implemented this on Windows 10. Unfortunately it is not working. getConnectionString() never returns anything and test times out. This happened even if I increased jest timeout to 1 min. Am I missing something?

ItBits2 • Sep 5 '20

    const mongoose = require('mongoose');
    const { MongoMemoryServer } = require('mongodb-memory-server');
    const mongoUnit = require('mongo-unit');
    const mongod = new MongoMemoryServer();
    jest.setTimeout(60000);

    module.exports.connect = function(callback) {
        mongod.getUri().then((mongoUri) => {

            const mongooseOpts = {
              autoReconnect: true,
              reconnectTries: Number.MAX_VALUE,
              reconnectInterval: 1000,
            };

            mongoose.connect(mongoUri, mongooseOpts);

            mongoose.connection.on('error', (e) => {
              if (e.message.code === 'ETIMEDOUT') {
                console.log(e);
                mongoose.connect(mongoUri, mongooseOpts);
              }
              console.log(e);
            });

            mongoose.connection.once('open', () => {
              console.log(`MongoDB successfully connected to ${mongoUri}`);
              callback();
            });
          });
        }

    module.exports.closeDatabase = async () => {
        await mongoose.connection.dropDatabase();
        await mongoose.connection.close();
        await mongod.stop();
    }


    module.exports.clearDatabase = async () => {
        const collections = mongoose.connection.collections;

        for (const key in collections) {
            const collection = collections[key];
            await collection.deleteMany();
        }
    }

Brian Le • Jun 22 '20

Hi, Great article, I have tried your configuration, and it works in some case.
However, if I use my customvalidate function in my schema , it not works. Is there any way to let the 'mongodb-memory-server' understand what we define in Schema - Model

Paula Santamaría • Jul 15 '20

I haven't had any issues with mongoose validations in this setup. 'mongodb-memory-server' shouldn't prevent mongoose validations from working.
Would you please provide an example of the code that has the issue?

Yudha Putera Primantika • Jun 27 '20 • Edited

it's my first time using this, and I have no idea what going on the screenshot. I tried several test example and the all look like the picture here
dev-to-uploads.s3.amazonaws.com/i/...

Paula Santamaría • Jul 15 '20

It's hard to know without seeing the code, but my best guess is that jest is not finding any tests to run. Try specifying where your test files are, like so: jest ./tests-folder.

Yudha Putera Primantika • Jul 15 '20

That's just the thing, when it ran on my partner's pc it works just fine. Even after we continue building it there, the result on my laptop remains the same.

André Xavier Costa • Aug 8 '20

Many kudos for your nice and neat article! Picky comment is that last Github repo link is broken (which should lead to "more test examples" at the end).

Paula Santamaría • Aug 10 '20

Thanks for letting me know! It's fixed now :)

Luis Pais • Jun 6 '20

I'm guessing this doesn't support multi document transactions ?
Since I think document-level locking is not supported on the memory engine.

Paula Santamaría • Jun 8 '20

Hi Luis!
I haven't tried it myself yet, but apparently mongodb-memory-server does support multi-document transactions. However, for that to work, you must start the mongodb server as a replica set

Kenneth Uyabeme • Jun 7 '20

I love the db-handler module idea, makes for very clean code. Great article.

Paula Santamaría • Jun 8 '20

Thank you, Kenneth! :)

Sebastinez • Apr 21 '21

Thank you so much Paula, I was running against so many walls to make the mongo memory server work with mongoose and your article was the solution!
Saludos!

Paula Santamaría • Apr 21 '21

I'm glad my article was helpful, Sebas! Gracias por tu comentario 😊

Ali Al-Isawi • Jun 26 '20

I want your help

I have applied this procedure on MongoDB locally but I don't find data in the database though I have to remove clean database function

Paula Santamaría • Jul 15 '20 • Edited

Hi there!
The code I provided in this article clears out the database after every test. If that's not what you want, you can delete the contents of the afterEach function to avoid losing your data after every test.
However, keep in mind that if you're using mongodb-memory-server as your test database, every time you run the tests the database is created from scratch, so you'll have to populate any test data you need before running the tests (I'd do it in the beforeAll function, for example).
Hope this helps!

Oak Soe Kyaw • Jan 5 '21

Nice article!

Shivam Ahuja • Sep 29 '21

Hey I wanted to ask how that productModel.create() method works because we dont have any create method in the product.js in models