DEV Community

Bidhan Khatri
Bidhan Khatri

Posted on • Updated on • Originally published at

Monitoring ElasticSearch Cluster using Prometheus and Grafana

We are going to monitor our production ElasticSearch Cluster using elasticsearch exporter plugin and store those metrics on the Prometheus server and later visualize it using Grafana Dashboard. The exporter will scrape the metrics from the cluster and make it available on default port 9114(which can be changed) and Prometheus will scrape those metrics from the exporter and store it on its TSD.

ES Cluster >> es_exporter >> Prometheus >> Grafana

ElasticSearch Exporter:

ElasticSearch Exporter is a ElasticSearch stats exporter for Prometheus which is written in GO LANG.

NOTE: Exporter fetches metrics from an ElasticSearch cluster on every scrape, therefore having a too short scrape interval can impose load on ES master nodes, particularly if you run with --es.all and --es.indices.

Link for ElasticSearch Exporter github project.

I'm going to use pre-built binaries here.

tar xvf elasticsearch_exporter-1.1.0.linux-amd64.tar.gz
cd elasticsearch_exporter-1.1.0.linux-amd64
cp elasticsearch_exporter /usr/local/bin/es_exporter
Enter fullscreen mode Exit fullscreen mode

If you need any further information regarding binary then use --help option.

elasticsearch_exporter --help
Enter fullscreen mode Exit fullscreen mode


Now create systemd service file for elasticsearch exporter in your ES server. Define your ES cluster username password if set.

vim /etc/systemd/system/es_exporter.service

Description=Prometheus ES_exporter

ExecStart = /usr/local/bin/es_exporter --es.uri=http://elastic_user:XXXXXXXXXXX@localhost:9200 --es.all --es.indices --es.timeout 20s
ExecStop= /usr/bin/killall es_exporter

Enter fullscreen mode Exit fullscreen mode
systemctl start es_exporter.service
systemctl enable es_exporter.serivce
Enter fullscreen mode Exit fullscreen mode


config for prometheus.yml

vim /etc/prometheus/prometheus.yml
Add below lines.

  - job_name: elasticsearch
    scrape_interval: 60s
    scrape_timeout:  30s
    metrics_path: "/metrics"
    - targets: ['es-01:9114' ]
        service: elasticsearch
    - source_labels: [__address__]
      regex: '(.*)\:9108'
      target_label:  'instance'
      replacement:   '$1'
    - source_labels: [__address__]
      regex:         '.*\.(.*)\.lan.*'
      target_label:  'environment'
      replacement:   '$1'
Enter fullscreen mode Exit fullscreen mode
systemctl restart prometheus
Enter fullscreen mode Exit fullscreen mode


Import Grafana Dashboard and start monitoring your ElasticSearch Cluster metrics. You can either download in JSON format and upload in your grafana server or copy ID and import directly.

Alt Text

Alt Text

Top comments (0)