Plugin multiple Mongodb sources to Prometheus and visualise them on Grafana

Introduction

I ran into a situation where I had to put certain metrics alert for a couple of mongodb instances. This was a challenge as I couldnot find direct resources for the setup we were trying to achieve. So this article will help you plugin multiple mongodb sources and visualise them on Grafana.

Pre-requisites

The setup we are doing would require Docker installed on your system/server.

Create user for all Mongo instances

Go to your mongodb shell and create a user . This user will be used by mongodb exporter to export the metrices and forward them to Prometheus.

>use admin
>db.createUser(
  {
    user: "mongodb_exporter",
    pwd: "your_unique_password",
    roles: [
        { role: "clusterMonitor", db: "admin" },
        { role: "read", db: "local" }
    ]
  }
)

Installing and Running Mongodb Exporter

Here I am using Bitnami's Docker Image for exporting the metrices.
I have two mongodb sources that I need to plugin. So I will be running to mongodb exporters.

For instance 1 and 2:

docker run -d --name m1  -p 9216:9216 bitnami/mongodb-exporter:0.11.2 --mongodb.uri=mongodb://mongodb_exporter:[e-mail]:27017

docker run -d --name m2  -p 9215:9216 bitnami/mongodb-exporter:0.11.2 --mongodb.uri=mongodb://mongodb_exporter:[e-mail]:27017

The key differences above are the name, port on which the services run, and the mongo url.
Note: If you are trying this out on a server, make sure to add rules to your firewall so that the server containing the mongodb exporters can access the mongodb instances.

Prometheus

To install Prometheus using docker, we need to add a configuration file that will then be mounted to the docker container.

prom.yml

scrape_configs:
- job_name: 'prometheus'
    static_configs:
            - targets: ['PRIVATE_IP:9090']
  - job_name: 'mongo-1'
    static_configs:
            - targets: ['PRIVATE_IP:9216']
            - labels:
                instance: 'mongo1'
  - job_name: 'mongo-2'
    static_configs:
            - targets: ['PRIVATE_IP:9215']
            - labels:
                instance: 'mongo2'

So we mentioned targets in the scrap_configs section
i.e our mongodb-exporter instances that are scraping data from mongodb.

Now to run Prometheus

docker run  -d -v /PATH/TO/CONFIG/prom.yml:/etc/prometheus/prometheus.yml --network=host  --name prom prom/prometheus

Grafana

To run Grafana using docker:

docker run -d  --network=host --name grafana grafana/grafana

I have added a makefike for all the docker commands required to run each and every instance.

Note: Prometheus and Grafana here are running on host network and this is not adviceable for production environments. Its better to create a different network or expose only the ports that are to be accessed.

Signin

Go to YOUR_IP:3000. Login (username and password both is admin)

Add source

Go to Configuration > Data Sources > Add Source
Select Prometheus
In the URL field add http :// YOUR_IP:9090. Save the configuration.

See Multiple Grafana instances and toggle between them.

There are several free to import dashboards that you can use and we will be using one of those to see the data and toggle between different instances.

Add a Dashboard

• Hover on Import (+ sign on the left) > Import
• In the Import via Grafana section, for this example, add 7353. Click on load on the right of the input field.
• At the bottom, select source for prometheus and save the dashboard

Here You can see an Instance filter where one of the sources would be selected. From here you can toggle to the other source you have added initially.

And that is all you need to do to add multiple mongodb sources and visualise them on Grafana.I have added all the relevant files here