DEV Community

Santanu Paul
Santanu Paul

Posted on • Originally published at sntnupl.com

How to setup a MongoDB Replica set for development using Docker

#docker #mongodb #tutorial

In this post I will be sharing how I configure and run a MongoDB replica set on my local machine, using Docker.

While there are ample examples on the internet about running MongoDB containers, not many of them focus on how to bring up a MongoDB replica set.

Doing development against single node MongoDB is fine for starting, but as one starts using features like transactions, it becomes imperative that the MongoDB deployment has replica set enabled (or sharded cluster, a topic for future discussion).

Creating and developing against a MongoDB replica set in local development setup helps you learn the topic, and be more confident that your application code is already adapted to run against replica sets in production.

To follow along, you will need to have Docker, and Docker Compose installed in your machine.

To install Docker, I suggest following the official documentation at this link.

Once Docker is installed, install Docker Compose by following the links from this link.

I have created this Github repository, which contains all the required artifacts to follow along with this tutorial.

The Docker Compose file

Once Docker and Docker Compose are both installed on your machine, we can start writing the Docker Compose file, that contains directives to bring up the MongoDB replica set. I will show the full docker-compose.yml file first, and then explain the individual details.

version: '3.9'  # Docker Engine release 19.03.0+ [https://docs.docker.com/compose/compose-file/]

services: 

    # setup MongoDB cluster for production
    mongo-replica-setup:
        container_name: mongo-setup
        image: 'mongo:4.2'
        restart: on-failure
        networks:
            - netApplication
        volumes:
        - ./.docker/mongodb/scripts/mongosetup.sh:/scripts/mongosetup.sh
        entrypoint: ["bash", "/scripts/mongosetup.sh" ]
        env_file:
            - .env
        environment:
            MONGO_INITDB_ROOT_USERNAME: ${MONGO_INITDB_ROOT_USERNAME}
            MONGO_INITDB_ROOT_PASSWORD: ${MONGO_INITDB_ROOT_PASSWORD}
        depends_on:
        - mongo1
        - mongo2
        - mongo3

    mongo1:
        hostname: 'mongo1'
        container_name: 'mongo1'
        image: 'mongo:4.2'
        restart: 'on-failure'
        command: ["-f", "/etc/mongod.conf", "--keyFile", "/auth/file.key", "--replSet", "${MONGO_REPLICA_SET_NAME}", "--bind_ip_all"]
        expose: 
            - 27017
        ports: 
            - 30001:27017 
        networks: 
            - netApplication
        volumes:
            - dataMongo1:/data/db
            - logMongo1:/var/log/mongodb
            - ./.docker/mongodb/initdb.d/:/docker-entrypoint-initdb.d/
            - ./.docker/mongodb/mongod.conf:/etc/mongod.conf
            - ./.docker/mongodb/file.key:/auth/file.key
        healthcheck:
            test: test $$(echo "rs.status().ok" | mongo -u $${MONGO_INITDB_ROOT_USERNAME} -p $${MONGO_INITDB_ROOT_PASSWORD} --quiet) -eq 1
            interval: 30s
            start_period: 60s
        env_file:
            - .env
        environment:
            MONGO_INITDB_ROOT_USERNAME: ${MONGO_INITDB_ROOT_USERNAME}
            MONGO_INITDB_ROOT_PASSWORD: ${MONGO_INITDB_ROOT_PASSWORD}
            MONGO_INITDB_DATABASE: ${MONGO_INITDB_DATABASE}

    mongo2:
        hostname: 'mongo2'
        container_name: 'mongo2'
        image: 'mongo:4.2'
        command: ["-f", "/etc/mongod.conf", "--keyFile", "/auth/file.key", "--replSet", "${MONGO_REPLICA_SET_NAME}", "--bind_ip_all"]
        restart: 'on-failure'
        expose: 
            - 27017
        ports: 
            - 30002:27017  
        networks: 
            - netApplication
        volumes:
            - dataMongo2:/data/db
            - logMongo2:/var/log/mongodb
            - ./.docker/mongodb/mongod.conf:/etc/mongod.conf
            - ./.docker/mongodb/file.key:/auth/file.key
        env_file:
            - .env
        environment:
            MONGO_INITDB_ROOT_USERNAME: ${MONGO_INITDB_ROOT_USERNAME}
            MONGO_INITDB_ROOT_PASSWORD: ${MONGO_INITDB_ROOT_PASSWORD}
            MONGO_INITDB_DATABASE: ${MONGO_INITDB_DATABASE}
        depends_on: 
            - mongo1

    mongo3:
        hostname: 'mongo3'
        container_name: 'mongo3'
        image: 'mongo:4.2'
        command: ["-f", "/etc/mongod.conf", "--keyFile", "/auth/file.key", "--replSet", "${MONGO_REPLICA_SET_NAME}", "--bind_ip_all"]
        restart: 'on-failure'
        expose: 
            - 27017
        ports: 
            - 30003:27017  
        networks: 
            - netApplication
        volumes:
            - dataMongo3:/data/db
            - logMongo3:/var/log/mongodb
            - ./.docker/mongodb/mongod.conf:/etc/mongod.conf
            - ./.docker/mongodb/file.key:/auth/file.key
        env_file:
            - .env
        environment:
            MONGO_INITDB_ROOT_USERNAME: ${MONGO_INITDB_ROOT_USERNAME}
            MONGO_INITDB_ROOT_PASSWORD: ${MONGO_INITDB_ROOT_PASSWORD}
            MONGO_INITDB_DATABASE: ${MONGO_INITDB_DATABASE}
        depends_on: 
            - mongo1

volumes: 
    dataMongo1:
    dataMongo2:
    dataMongo3:
    logMongo1:
    logMongo2:
    logMongo3:

networks: 
    netApplication:
Enter fullscreen mode Exit fullscreen mode

The Setup

Replica set nodes

Here we are trying to setup a MongoDB replica set with 3 nodes.

One of these nodes will be the Primary node - all writes to the database will happen via this node.

The remaining two, are Secondary nodes, providing two-levels of data replication in the setup.

A solid explanation of how data replication works in MongoDB can be found at this link.

Exploring the docker-compose file

Coming back to the yaml file, let's go through the service declaration of mongo1 and see what we are trying to achieve.

The basic stuff 

mongo1:
    hostname: 'mongo1'
    container_name: 'mongo1'
    image: 'mongo:4.2'
    restart: 'on-failure'
    :
    expose: 
        - 27017
    ports: 
        - 30001:27017 
    networks: 
        - netApplication
Enter fullscreen mode Exit fullscreen mode

Nothing fancy here, we are targetting MongoDB server version 4.2 to run these containers.

We have declared hostname and container name, and set the container restart policy to on-failure.

On the ports front, we have exposed 27017 to other docker containers running on the same docker network. Also, this port is mapped to host port 30001.

Lastly, we will be running our replica set on a named network.

Starting the mongod instance 

command: ["-f", "/etc/mongod.conf", "--keyFile", "/auth/file.key", "--replSet", "${MONGO_REPLICA_SET_NAME}", "--bind_ip_all"]
Enter fullscreen mode Exit fullscreen mode

This will start the mongod service with the following:

  • -f /etc/mongod.conf
    • specifies that the runtime configuration options should be picked up from the provided mongod.conf file.
    • we will see the contents of this file later
  • -keyFile /auth/file.key
    • In our replica set, the nodes will use the contents of a shared keyfile to authenticate to each other.
    • Here we are specifying the path to the keyfile
  • --replSet ${MONGO_REPLICA_SET_NAME}
    • This configures the node to run in a replica set.
    • We are also specifying the name of the replica set, which is to be picked up from the environment variable ${MONGO_REPLICA_SET_NAME}
  • --bind_ip_all
    • This specifies the mongod instance to bind to all IPv4 addresses (0.0.0.0).

Container volumes

Before proceeding it is worth mentioning again, that I have created this Github repository to house all the artifacts mentioned here.

volumes:
    - mongoData1:/data/db
    - mongoLog1:/var/log/mongodb
    - ./.docker/mongodb/initdb.d/:/docker-entrypoint-initdb.d/
    - ./.docker/mongodb/mongod.conf:/etc/mongod.conf
    - ./.docker/mongodb/file.key:/auth/file.key
Enter fullscreen mode Exit fullscreen mode

Here we have setup a few Docker managed volumes to store the container's data in host's file system.

  • mongoData1: This will act as the peristent store for MongoDB's data.
  • mongoLog1: All of MongoDB's logs for this container will be stored here.
  • ./.docker/mongodb/initdb.d/
    • When mongod container starts for the very first time, it will look for a directory named /docker-entrypoint-initdb.d, and execute all files with extensions .sh and .js.
    • We map this directory to ./.docker/mongodb/initdb.d/ in host.
    • As we will see later, we will use this directory to store a bash script that allows us to create a user.
  • ./.docker/mongodb/mongod.conf
    • We are essentially providing the path (in host) from where MongoDB will pick up its config file.
  • ./.docker/mongodb/file.key
    • Like above volume, here we are specifying the keyfile that this MongoDB node will be using to authenticate itself to other nodes in the replica set.
    • Steps to create this keyfile is provided in the Github repository that I have created. 
env_file:
    - .env
environment:
    MONGO_INITDB_ROOT_USERNAME: ${MONGO_INITDB_ROOT_USERNAME}
    MONGO_INITDB_ROOT_PASSWORD: ${MONGO_INITDB_ROOT_PASSWORD}
    MONGO_INITDB_DATABASE: ${MONGO_INITDB_DATABASE}
Enter fullscreen mode Exit fullscreen mode

Here we have specified that the docker compose will be using the .env file to read environment variables and inject them into the container.

This allows us to specify that the database's root username/password and initial database name to be picked up from specific environment variables.

In my github repository, I have provided a sample env file that you can use to create your own .env file.

Contents of the sample env file looks like this:

# MongoDB
MONGO_URL=mongodb://mongodb:27017
MONGO_INITDB_ROOT_USERNAME=<root_username>
MONGO_INITDB_ROOT_PASSWORD=<root_password>
MONGO_INITDB_DATABASE=<app_db-name>
MONGO_INITDB_USERNAME=<app_username>
MONGO_INITDB_PASSWORD=<app_password>
MONGO_REPLICA_SET_NAME=rs0
Enter fullscreen mode Exit fullscreen mode

One must make sure not to commit this file in github, as these contains secrests like root username/password.

Tree view of files and directories

The above image provides a tree view of the files and directories discussed here.

Health Check 

healthcheck:
    test: test $$(echo "rs.status().ok" | mongo -u $${MONGO_INITDB_ROOT_USERNAME} -p $${MONGO_INITDB_ROOT_PASSWORD} --quiet) -eq 1
    interval: 30s
    start_period: 60s
Enter fullscreen mode Exit fullscreen mode

Here we are specifying the healthcheck command that docker will use to check if the container is up and running.

To summarize, the mongo1 service's specification looks like this:

mongo1:
    hostname: 'mongo1'
    container_name: 'mongo1'
    image: 'mongo:4.2'
    restart: 'on-failure'
    command: ["-f", "/etc/mongod.conf", "--keyFile", "/auth/file.key", "--replSet", "${MONGO_REPLICA_SET_NAME}", "--bind_ip_all"]
    expose: 
        - 27017
    ports: 
        - 30001:27017 
    networks: 
        - netApplication
    volumes:
        - mongoData1:/data/db
        - mongoLog1:/var/log/mongodb
        - ./.docker/mongodb/initdb.d/:/docker-entrypoint-initdb.d/
        - ./.docker/mongodb/mongod.conf:/etc/mongod.conf
        - ./.docker/mongodb/file.key:/auth/file.key
    healthcheck:
        test: test $$(echo "rs.status().ok" | mongo -u $${MONGO_INITDB_ROOT_USERNAME} -p $${MONGO_INITDB_ROOT_PASSWORD} --quiet) -eq 1
        interval: 30s
        start_period: 60s
    env_file:
        - .env
    environment:
        MONGO_INITDB_ROOT_USERNAME: ${MONGO_INITDB_ROOT_USERNAME}
        MONGO_INITDB_ROOT_PASSWORD: ${MONGO_INITDB_ROOT_PASSWORD}
        MONGO_INITDB_DATABASE: ${MONGO_INITDB_DATABASE}
Enter fullscreen mode Exit fullscreen mode

The other two nodes - mongo1 and mongo2 is a subset of this specification. The only things missing are the healthcheck section, and a volume that maps to docker-entrypoint-initdb.d. This is because we have designed this setup such that mongo1 will become the primary and the rest secondary.

Being secondary nodes, they dont need the initdb part to work.

Create the Mongo Cluster

Once all the nodes in the replica set is up, we will need to initiate Replica set configuration on these nodes.

We will be using a 4th container - mongo-replica-setup to do this.

mongo-replica-setup:
    container_name: mongo-setup
    image: 'mongo:4.2'
    restart: on-failure
    networks:
        - netApplication
    volumes:
    - ./.docker/mongodb/scripts/mongosetup.sh:/scripts/mongosetup.sh
    entrypoint: ["bash", "/scripts/mongosetup.sh" ]
    env_file:
        - .env
    environment:
        MONGO_INITDB_ROOT_USERNAME: ${MONGO_INITDB_ROOT_USERNAME}
        MONGO_INITDB_ROOT_PASSWORD: ${MONGO_INITDB_ROOT_PASSWORD}
    depends_on:
        - mongo1
        - mongo2
        - mongo3
Enter fullscreen mode Exit fullscreen mode

Once this container starts, it will connect to mongo1, and execute a script that will initiate a replica set over mongo1, mongo2 and mongo3.

Once the script is executed this container exits.

Its important to note that we have set the depends_on field of this container to mongo1, mongo2 and mongo3, which ensures that this container will start only after all those 3 containers are up. This is important, as you wont want to run the script without all the nodes of the replica set being up.

The following is the script that this container runs, to initiate the replica set:

#!/bin/bash

MONGODB1=mongo1
MONGODB2=mongo2
MONGODB3=mongo3

echo "**********************************************" ${MONGODB1}
echo "Waiting for startup.."
sleep 30
echo "done"

echo SETUP.sh time now: `date +"%T" `
mongo --host ${MONGODB1}:27017 -u ${MONGO_INITDB_ROOT_USERNAME} -p ${MONGO_INITDB_ROOT_PASSWORD} <<EOF
var cfg = {
    "_id": "rs0",
    "protocolVersion": 1,
    "version": 1,
    "members": [
        {
            "_id": 0,
            "host": "${MONGODB1}:27017",
            "priority": 2
        },
        {
            "_id": 1,
            "host": "${MONGODB2}:27017",
            "priority": 0
        },
        {
            "_id": 2,
            "host": "${MONGODB3}:27017",
            "priority": 0,
        }
    ]
};
rs.initiate(cfg, { force: true });
rs.secondaryOk();
db.getMongo().setReadPref('primary');
rs.status();
EOF
Enter fullscreen mode Exit fullscreen mode

If you look closely, we have set the priority for mongo1 higher than the other two nodes, which ensures that initially, mongo1 will be acting as the Primary node.

Final Result

To start the replica set, run docker-compose up -d from command line.

Once everything comes up, you can run docker-compose status, and see something like this:

Replica set status

To connect to the replica set, you can use mongo client like so:

$ mongo "mongodb://localhost:30001,localhost:30002,localhost:30003/<MONGO_INITDB_DATABASE>" -u <MONGO_INITDB_USERNAME>
Enter fullscreen mode Exit fullscreen mode

Connect to Replica set

Top comments (6)

Collapse
 
maluz profile image
Marco Luzzara •

Hello,
Thank you for this post, very helpful. Where did you find out that chown
999:999 yourfile.key was necessary? The reference on dockerhub does not mention anything like that. Thank you again

Best regards
Marco

Collapse
 
sntnupl profile image
Santanu Paul • • Edited

Hi Marco,

I'm glad you liked the blog post.
The chown thing was necessary due to one of the errors that I faced while creating this setup. I found the solution from here.

Collapse
 
baayfaal profile image
baayfaal • • Edited

Hi,
Thank you very much for this share. I'm trying to test it on my local environment but I have following error :

Error

Could you please advise ?

Regards
Pape

Collapse
 
sntnupl profile image
Santanu Paul •

Hi Pape,

My guess is that the keyfile, which the mongo nodes will be using for authentication, is not created with proper file permissions. Kindly go through the steps as mentioned in the github repo and recreate the 'file.key'.

Both chmod and chown commands are important, and should not be skipped.

Hope this helps.

Santanu

Collapse
 
baayfaal profile image
baayfaal •

Hi Santanu,

Tahnk you for you reply ! I'm on windows 10 and I will try to find how to do same on the file permission and ownership.

Regards,
Pape

Collapse
 
thinkdiff profile image
Amir Salar Pourhasan •

Hi Santanu
Thanks a lot for your artiele.
I faced this problem:

Image description

I did chown. When I check logs of the container "mongo-setup" I see this:

Image description

When I check logs of mongo1, I see nothing!

What is the problem?

Read next

disane profile image

Migration vom Nginx Proxy Manager zu Traefik 🚀

Marco -

emqtech profile image

Running EMQX MQTT Broker on Docker (Bonus: Clustering and Persistence)

EMQ Technologies -

krivanek06 profile image

Creating Custom rxResource API With Observables

Eduard Krivanek -

westtan profile image

Is Docker’s Market Being Eroded? An In-Depth Comparison of Docker, ServBay, Homebrew, Podman, and MAMP

westtan -