How to Monitor Your Elasticsearch Cluster with Vigilmon

Elasticsearch powers search functionality for millions of applications — from e-commerce product search to log analytics platforms. But Elasticsearch clusters are notoriously complex to operate: unassigned shards, JVM heap pressure, split-brain scenarios, and node failures can degrade search quality or cause complete outages without any obvious external signal.

This guide shows you how to monitor your Elasticsearch cluster with Vigilmon using Elasticsearch's built-in health APIs.

---

Why Elasticsearch Monitoring Is Different

Elasticsearch exposes cluster health directly through its REST API — no custom code required:

GET /_cluster/health

This returns:

{
  "cluster_name": "my-cluster",
  "status": "green",
  "timed_out": false,
  "number_of_nodes": 3,
  "number_of_data_nodes": 3,
  "active_primary_shards": 25,
  "active_shards": 50,
  "relocating_shards": 0,
  "initializing_shards": 0,
  "unassigned_shards": 0,
  "number_of_pending_tasks": 0
}

The status field is the key metric:

• green — all primary and replica shards are assigned
• yellow — all primaries assigned, but some replicas aren't (data intact, but no redundancy)
• red — some primary shards are unassigned (data loss possible, queries may fail)

---

Step 1: Expose a Health Proxy Endpoint

If your Elasticsearch cluster isn't directly internet-accessible (it shouldn't be), proxy the health check through your application:

Node.js example:

const express = require('express');
const { Client } = require('@elastic/elasticsearch');

const app = express();
const esClient = new Client({ node: process.env.ELASTICSEARCH_URL });

app.get('/health/elasticsearch', async (req, res) => {
  try {
    const health = await esClient.cluster.health({});
    const status = health.status; // 'green', 'yellow', 'red'

    if (status === 'red') {
      return res.status(503).json({ 
        status: 'critical',
        cluster_status: status,
        unassigned_shards: health.unassigned_shards
      });
    }

    if (status === 'yellow') {
      return res.status(200).json({ 
        status: 'degraded',
        cluster_status: status,
        message: 'Replica shards unassigned'
      });
    }

    res.json({ 
      status: 'ok', 
      cluster_status: status,
      nodes: health.number_of_nodes
    });
  } catch (err) {
    res.status(503).json({ status: 'error', message: err.message });
  }
});

Python (FastAPI) example:

from fastapi import FastAPI, Response
from elasticsearch import AsyncElasticsearch
import json

app = FastAPI()
es = AsyncElasticsearch(hosts=["http://localhost:9200"])

@app.get("/health/elasticsearch")
async def health_elasticsearch():
    try:
        health = await es.cluster.health()
        status = health["status"]

        if status == "red":
            return Response(
                content=json.dumps({"status": "critical", "cluster": status}),
                status_code=503,
                media_type="application/json"
            )
        return {
            "status": "ok" if status == "green" else "degraded",
            "cluster_status": status,
            "nodes": health["number_of_nodes"]
        }
    except Exception as e:
        return Response(content=str(e), status_code=503)

---

Step 2: Add HTTP Monitors in Vigilmon

1. Log in to vigilmon.online → Add Monitor
2. Type: HTTP(S)
3. URL: https://your-app.com/health/elasticsearch
4. Interval: 60 seconds
5. Alert if: Status is not 200, or response > 3000ms

Add a second monitor with a stricter threshold if you need to alert on yellow status too — configure the endpoint to return 503 for yellow when running critical workloads.

---

Step 3: Monitor Index Health

For applications that depend on specific indices, add per-index monitoring:

app.get('/health/elasticsearch/index/:name', async (req, res) => {
  try {
    const stats = await esClient.indices.stats({ index: req.params.name });
    const indexHealth = await esClient.cluster.health({
      index: req.params.name
    });

    const docCount = stats._all.total.docs.count;
    const status = indexHealth.status;

    if (status === 'red' || docCount === 0) {
      return res.status(503).json({ status: 'error', index_status: status, docs: docCount });
    }

    res.json({ status: 'ok', index_status: status, docs: docCount });
  } catch (err) {
    res.status(503).json({ status: 'error', message: err.message });
  }
});

Monitor /health/elasticsearch/index/products to detect if your product search index is empty (zero docs) or degraded.

---

Step 4: JVM Heap Alert

Elasticsearch's most common performance failure is JVM heap pressure causing GC pauses. Add a heap check:

app.get('/health/elasticsearch/jvm', async (req, res) => {
  try {
    const stats = await esClient.nodes.stats({ metric: 'jvm' });
    const nodes = Object.values(stats.nodes);

    const highHeap = nodes.filter(n => 
      n.jvm.mem.heap_used_percent > 85
    );

    if (highHeap.length > 0) {
      return res.status(503).json({
        status: 'heap_pressure',
        high_heap_nodes: highHeap.map(n => ({
          name: n.name,
          heap_pct: n.jvm.mem.heap_used_percent
        }))
      });
    }

    res.json({ status: 'ok', nodes: nodes.length });
  } catch (err) {
    res.status(503).json({ status: 'error', message: err.message });
  }
});

---

Elasticsearch Monitoring Coverage Table

Monitor	Endpoint	Alerts On
Cluster health	/health/elasticsearch	Red status or unreachable
Index health	/health/elasticsearch/index/products	Empty index or red
JVM heap	/health/elasticsearch/jvm	Heap > 85%
SSL	yourdomain.com	Cert expiry < 14 days

---

Common Elasticsearch Failure Patterns

Unassigned shards after node restart: A data node rebooted for a kernel update. Shards began reassigning (yellow status). The cluster health monitor alerted within 60 seconds while reassignment was in progress.

JVM heap OOM: A bulk indexing job caused heap to spike to 95% on two nodes. GC pauses began. Search latency went from 50ms to 4000ms. The JVM monitor caught it before users noticed.

Empty index after bad migration: An index swap script deleted the old index before the new one was ready. Zero documents in the products index. The index health monitor fired immediately.

---

Conclusion

Elasticsearch provides rich built-in health APIs. Pair them with Vigilmon's multi-region HTTP monitoring to get real-time alerts on cluster status, index health, and JVM pressure — before search failures affect your users.

Start monitoring your Elasticsearch cluster free at vigilmon.online