Improving the local development setup for running MongoDB Replica Set with Docker

All associated source code used in this blog can be found at the following Github repository:

sntnupl / devcontainers-mongodb-replica-set-with-docker

Docker based development container template to run a MongoDB replica set cluster in local machine.

In my previous post, we created a docker based development setup for running MongoDB Replica Set.

A flaw in that setup, was that although clients could connect to stand-alone nodes of the replica set, they could not connect to the replica set as a whole, as reported by this issue on the github repository.

In this blog post, we will see what can be done, to overcome this problem.

---

Root Cause

As per the MongoDB specifications:

the hostnames used in a replica set config are reachable from the client

In our previous blog, we initialized the following replica set configuration:

MONGODB1=mongo1
MONGODB2=mongo2
MONGODB3=mongo3
:
var cfg = {
    "_id": "rs0",
    ...
    "members": [
        {
            ...
            "host": "${MONGODB1}:27017",
            ...
        },
        {
            ...
            "host": "${MONGODB2}:27017",
            ...
        },
        {
            ...
            "host": "${MONGODB3}:27017",
            ...
        }
    ]
};

In essence, this would require the clients to able to reach the following hosts: mongo1:27017, mongo2:27017, mongo3:27017.

We can update /etc/hosts file to point 127.0.0.1 to mongo1, mongo2, mongo3 hostnames. But the portnumber 27017 would still cause the problem - we can't have 3 nodes (services) running on the same port.

Therefore, solution to the problem would be the following:

1. Update hosts file in client machines, to map hostnames mongo1, mongo2, mongo3 to 127.0.0.1
2. Run mongod service on the 3 nodes of the replica set. in different port numbers

---

Solution

Implementing the 1st step is trivial. One would need to update /etc/hosts file in Linux, or C:\Windows\System32\drivers\etc\hosts in Windows:

127.0.0.1 localhost mongo1 mongo2 mongo3

Let's go through the steps to implement the second step.

Changes to the docker compose file

...
mongo1:
  hostname: 'mongo1'
  container_name: 'mongo1'
  ...
  command: ["-f", "/etc/mongod.conf", "--port", "30001", "--keyFile", "/auth/file.key", "--replSet", "${MONGO_REPLICA_SET_NAME}", "--bind_ip_all"]
  expose: 
    - 30001
  ports: 
    - 30001:30001 
  ...

mongo2:
  hostname: 'mongo2'
  container_name: 'mongo2'
  ...
  command: ["-f", "/etc/mongod.conf", "--port", "30002", "--keyFile", "/auth/file.key", "--replSet", "${MONGO_REPLICA_SET_NAME}", "--bind_ip_all"]
  expose: 
    - 30002
  ports: 
    - 30002:30002 
  ...

mongo3:
  hostname: 'mongo3'
  container_name: 'mongo3'
  ...
  command: ["-f", "/etc/mongod.conf", "--port", "30003", "--keyFile", "/auth/file.key", "--replSet", "${MONGO_REPLICA_SET_NAME}", "--bind_ip_all"]
  expose: 
    - 30003
  ports: 
    - 30003:30003 
  ...

This will bring up mongod services in the 3 nodes in 3 different ports.

Next, we modify the replica set configuration, by changing mongosetup.sh file:

mongo --host ${MONGODB1}:30001 -u ${MONGO_INITDB_ROOT_USERNAME} -p ${MONGO_INITDB_ROOT_PASSWORD} <<EOF
var cfg = {
    "_id": "rs0",
    ...
    "members": [
        {
            ...
            "host": "${MONGODB1}:30001",
            ...
        },
        {
            ...
            "host": "${MONGODB2}:30002",
            ...
        },
        {
            ...
            "host": "${MONGODB3}:30003",
            ...
        }
    ]
};

These are essentially all the necessary changes needed to be done, to solve the problem.

---

Final Result

Once we make these changes, we could connect to the replica set using mongo client:

$ mongo "mongodb://localhost:30001,localhost:30002,localhost:30003/<MONGO_INITDB_DATABASE>?replicaSet=rs0" -u <MONGO_INITDB_USERNAME> --authenticationDatabase admin

OR

$ mongo "mongodb://mongo1:30001,mongo2:30002,mongo3:30003/<MONGO_INITDB_DATABASE>?replicaSet=rs0" -u <MONGO_INITDB_USERNAME> --authenticationDatabase admin

We can also view result of db.isMaster() command to debug details like hosts of the replicaset a client must connect to, if the node is primary, etc.