DEV Community

Cover image for DevOps Monitoring and Automation Tool using Jenkins, Prometheus, Grafana and Docker
Nigel Bomett
Nigel Bomett

Posted on

DevOps Monitoring and Automation Tool using Jenkins, Prometheus, Grafana and Docker

In this guide, we will walk through the process of setting up a DevOps monitoring and automation tool and understanding how different tools can be used together for this purpose: Jenkins for continuous integration and continuous deployment(CI/CD), Prometheus for metrics collection, Grafana for visualization, and Docker for containerization.

...

Install Docker

If you haven't already installed Docker, follow the installation instructions for your operating system from the Docker website.

Set up Jenkins

Run Jenkins in a Docker Container:

 ```
 docker run -u 0 --privileged --name jenkins -it -d -p 8080:8080 -p 
 50000:50000 
 \-v/var/run/docker.sock:/var/run/docker.sock 
 \-v $(which docker):/usr/bin/docker 
 \-v /home/jenkins_home:/var/jenkins_home 
 \jenkins/jenkins:latest
```
Enter fullscreen mode Exit fullscreen mode

Access and Setup Jenkins:

  • Open your browser and go to http://localhost:8080/

  • You will be requested to provide a password and there is a guide
    on where to find it however, in case you can not locate the file
    you can run the command: docker logs <docker container id> where
    the 'docker container id' is the id of the docker container running
    Jenkins, the password will be displayed in the logs.

  • For this tutorial, you can select to install suggested plugins.
    We will install any other required plugins later on.

  • From the dashboard add a new item, I will be using, Pipeline.

Install Plugins:

  • Go to "Manage Jenkins" > "Manage Plugins".

  • In the "Available" tab, search for "Docker Plugin", "Docker Pipeline
    Plugin", "Prometheus metrics Plugin" and install them. Restart Jenkins
    if prompted.

Set Up Prometheus and Grafana

  • Create a Docker Network:
    docker network create monitoring

  • Run Prometheus:
    Create a prometheus.yml configuration file:

global:
  scrape_interval: 15s

scrape_configs:
  - job_name: 'prometheus'
    static_configs:
      - targets: ['localhost:9090']

  - job_name: 'jenkins'
    static_configs:
      - targets: ['jenkins:8080']
      metrics_path: '/prometheus'

  - job_name: 'node'
    static_configs:
      - targets: ['node-exporter:9100']

Enter fullscreen mode Exit fullscreen mode
  • Run Prometheus with this configuration:
docker run -d --name=prometheus 
\--network=monitoring 
\-p 9090:9090 
\-v $(pwd)/prometheus.yml:/etc/prometheus/prometheus.yml \prom/prometheus

Enter fullscreen mode Exit fullscreen mode
  • Run Node Exporter:
docker run -d --name=node-exporter 
\--network=monitoring 
\--restart=unless-stopped 
\prom/node-exporter

Enter fullscreen mode Exit fullscreen mode
  • Run Grafana:
docker run -d --name=grafana 
\--network=monitoring 
\-p 3000:3000 
\grafana/grafana

Enter fullscreen mode Exit fullscreen mode

Configure Grafana

  • Access Grafana
    Open your browser and go to http:localhost:3000.Log in with the default username and password(admin).

  • Add Prometheus as a Data Source:
    Go to "Data Sources" on the left sidebar, "Add data source" and select "Prometheus".
    Enter http://prometheus:9090 as the Prometheus server URL.
    Click "Save & Test"

  • Create a Dashboard:
    Select "Dashboards" from the left sidebar to create a new dashboard. You can import dashboard using ID or URL. For example ID: '11159'
    You can edit one of the panels or 'add visualization', then add a query for the data you want to see.
    For this example, we will be looking at how long it takes Jenkins to process a job. Add the query:
    default_jenkins_builds_last_build_duration_milliseconds

Automate application deployment with Jenkins

  • Set Up a Jenkins Pipeline: From the application you had set up in Jenkins, go to "configure" and scroll down to "Pipeline Script" The script below will guide you, you may customize to fit the application you are deploying:
pipeline {
    agent any

    environment {
        DB_HOST = 'your-db-host'
        DB_USER = 'your-db-user'
        DB_PASS = 'your-db-password'
        DB_NAME = 'your-db-name'
    }

    stages {
        stage('Checkout') {
            steps {
                script {
                    checkout([$class: 'GitSCM', branches: [[name: '*/main']],
                      doGenerateSubmoduleConfigurations: false,
                      extensions: [[$class: 'CleanCheckout']],
                      submoduleCfg: [],
                      userRemoteConfigs: [[url: 'https://github.com/yourusername/your-nodejs-typescript-api-repo.git',
                      credentialsId: 'git-credentials-id']]
                    ])
                }
            }
        }

        stage('Install Dependencies') {
            steps {
                sh 'npm install'
            }
        }

        stage('Build') {
            steps {
                sh 'npm run build'
            }
        }

        stage('Test') {
            steps {
                sh 'npm test'
            }
        }

        stage('Build Docker Image') {
            steps {
                script {
                    sh 'docker build -t your-dockerhub-username/your-nodejs-typescript-api:latest .'
                }
            }
        }

        stage('Push Docker Image') {
            steps {
                script {
                    withCredentials([usernamePassword(credentialsId: 'dockerhub-credentials', usernameVariable: 'DOCKER_USERNAME', passwordVariable: 'DOCKER_PASSWORD')]) {
                        sh 'echo $DOCKER_PASSWORD | docker login -u $DOCKER_USERNAME --password-stdin'
                    }
                    sh 'docker push your-dockerhub-username/your-nodejs-typescript-api:latest'
                }
            }
        }

        stage('Deploy') {
            steps {
                script {
                    sh '''
                    docker run -d -p 3000:3000 \
                        -e NODE_ENV=$NODE_ENV \
                        -e DB_HOST=$DB_HOST \
                        -e DB_USER=$DB_USER \
                        -e DB_PASS=$DB_PASS \
                        -e DB_NAME=$DB_NAME \
                        your-dockerhub-username/your-nodejs-typescript-api:latest
                    '''
                }
            }
        }

        stage('Monitor') {
            steps {
                script {
                    // Additional monitoring setup can be added here
                }
            }
        }
    }

    post {
        success {
            echo 'Build succeeded!'
        }
        failure {
            echo 'Build failed!'
        }
    }
}


Enter fullscreen mode Exit fullscreen mode

Credentials

If you will be pulling code from Github you will need to add credentials for git hub and also Docker.
Go to "Manage Jenkins" > "Credentials" select global, then add your credentials for both Github and Docker. Take note of the id as this is what is used as reference in the Pipeline Script.

Build Application

Build your application and view the metrics on the Grafana dashboard 😃.

Image description

Top comments (2)

Collapse
 
nikko1350 profile image
Nikko

Maybe a bit more explanations of what is happening here?

Collapse
 
blenderman profile image
BBM

Great overview of integrating Jenkins, Prometheus, Grafana, and Docker! Could you elaborate more on configuring alerting in Prometheus?