Effortless Monitoring: Simplifying Deployment of a Powerful Infrastructure Monitoring Tool

Hello everyone, I am very happy to present my first article to you today.

We will discuss the automation & deployment of a monitoring solution and all the tools necessary for its proper functioning.

The solution we are deploying today is Grafana. This opensource tool allows us to set up monitoring of our production or test environments for free.

All our services are dockerized. This approach to containerization of services is a personal choice. I find it very important to containerize the various services, so that we can act in a targeted manner on what we want to modify.

Prerequisites

You can view / clone my work from my Github directory: https://github.com/ArsNQ/deployMonitoring

First, you need to use "ssh-copy-id" to copy your SSH keys to the servers. Without this, ansible will not be able to run. There are a few changes to the code to make in order for it to run properly.

Hosts file

The file "hosts" at the root of the folder is containing the list of servers you will want to monitor. You must add all the servers (their ip addresses, private or public) under the [cluster] line. And you must add below [GrafanaHost] the ip address of the server which will host Grafana.

Email addresses

In the folder roles/deployGrafana/files, you must change the alertmanager.yml file at the following lines:

      - to: "**mailaddress**"
      from: "**mailaddress**"
      smarthost: "smtp.gmail.com:465"
      auth_username: "**mailaddress**"
      auth_identity: "**mailaddress**"
      auth_password: "password

If you have your own SMTP server, you can use it, but you need to change the smarthost line. If you don't have a private SMTP server, you can use Google like I do. It's very simple and many tutorials exist on the internet. After that, you just need to replace all the emailaddresses with your own and you will receive the alerts on them. If you want to receive these alerts on another email address, you just have to change the "to" line.

Monitoring file

You must modify the following file: roles/deployGrafana/files/prometheus.yml. Indeed, this file contains all the services listed to monitor our environments. It allows you to find out which server is running alertmanagers, where the node-exporter are located, etc ...

So we have to modify alertmanagers (line 16), & prometheus (line 20) they are running on the same server as Grafana. so you have to enter the exact ip address on these two fields.

And for the node-exporter (line 23-XX) & cadvisor(line 29-XX) part you have to enter the ip address of the Grafana, and the environments you want to monitor.

After that we have to go and modify the file roles /deployGrafana/files/provisioning/dashboards/datasource.yml & indicate the IP address of the server where Prometheus is running. This is the file that tells Grafana where is located the prometheus datasource. As you can see just below, prometheus allows to link grafana and node-exporter/cadvisor, It has the role of an interval/syncronizer beween every services.

And finally, you need to replace the password in the /deployGrafana/files/provisioning/config.monitoring file, which is the password for your connection to Grafana (you can change it on the platform after).

Explanation

As we indicated at the beginning of this article, I decided to use docker to have better management of my services. This is why each command executed while using ansible is a docker-compose up -d in order to launch the docker-compose.yml previously configured in the files folder of each role. They are deployed on the servers in order to launch the right containers.

First, our script will connect to the different environments, and will execute the first role, deployNodeExporter. It consists of a series of installation of packages & dependencies (like docker, apt-transport-https, etc...) so that the script can run properly afterwards.

Once all our environment is up to date and it runs our first node-exporter container on all servers. We move on to the second role which is called deployCadvisor, which will, as you can guess, deploy cadvisor.

And the last role called deployGrafana will take care of deploying the Prometheus, Grafana, and Alertmanager containers. As well as the configuration files inside the services.

And here is the result after the deployment of your services!

Grafana collects all the data via prometheus and allows you to display it on your web interface!
I thought of importing you two tables to monitor your environments, one called 'Docker Monitoring' & the other 'Infrastructure Monitoring'.
If you want to access it, nothing could be simpler, click on

And click on the dashboard that you want to display.

Overview

Here is a quick overview of the data that you will be able to recover and view.

You can very well imagine this kind of solution in a CI/CD if you want to automate the deployment from a pipeline. It can also be useful to you in a DRP to deploy in an automated way a simple but effective supervision.

Thanks for reading!
I hope you enjoyed this tutorial and if you have any feedback for me, please do not hesitate!