How to Monitor Your Elasticsearch Cluster with Vigilmon

Elasticsearch is powerful but notoriously temperamental. Cluster health can degrade from green to red in minutes — shards go unassigned, nodes leave the cluster, memory fills up and the JVM garbage collector starts thrashing. When Elasticsearch goes down, your search functionality disappears and any service depending on it breaks.

This guide shows how to monitor Elasticsearch with Vigilmon using the built-in cluster health API.

Elasticsearch's Built-in Health API

One of Elasticsearch's best features for monitoring: it ships with a comprehensive health API that Vigilmon can poll directly.

# Cluster health
curl -X GET 'https://localhost:9200/_cluster/health?pretty'

Response:

{
  "cluster_name": "my-cluster",
  "status": "green",
  "timed_out": false,
  "number_of_nodes": 3,
  "number_of_data_nodes": 3,
  "active_primary_shards": 15,
  "active_shards": 30,
  "relocating_shards": 0,
  "initializing_shards": 0,
  "unassigned_shards": 0
}

Cluster status meanings:

• green: All primary and replica shards are assigned
• yellow: All primary shards assigned, but some replicas unassigned (cluster is functional, but not fully redundant)
• red: Some primary shards unassigned (search and indexing failures)

Setting Up a Health Proxy Endpoint

If your Elasticsearch isn't publicly accessible (it shouldn't be!), expose it through your application:

Node.js / Express

const axios = require('axios');

app.get('/health/elasticsearch', async (req, res) => {
  try {
    const response = await axios.get(
      `${process.env.ES_URL}/_cluster/health`,
      {
        auth: {
          username: process.env.ES_USERNAME,
          password: process.env.ES_PASSWORD
        },
        timeout: 5000
      }
    );

    const health = response.data;
    const isHealthy = health.status === 'green' || health.status === 'yellow';

    return res.status(isHealthy ? 200 : 503).json({
      status: health.status,
      nodes: health.number_of_nodes,
      unassigned_shards: health.unassigned_shards,
      active_shards: health.active_shards
    });
  } catch (err) {
    return res.status(503).json({ status: 'error', error: err.message });
  }
});

Python / FastAPI

import httpx
from fastapi import FastAPI
from fastapi.responses import JSONResponse
import os

app = FastAPI()

@app.get('/health/elasticsearch')
async def health_elasticsearch():
    try:
        async with httpx.AsyncClient() as client:
            response = await client.get(
                f"{os.environ['ES_URL']}/_cluster/health",
                auth=(os.environ['ES_USERNAME'], os.environ['ES_PASSWORD']),
                timeout=5.0
            )
            health = response.json()

            is_healthy = health['status'] in ('green', 'yellow')
            status_code = 200 if is_healthy else 503

            return JSONResponse(
                content={
                    'status': health['status'],
                    'nodes': health['number_of_nodes'],
                    'unassigned_shards': health['unassigned_shards']
                },
                status_code=status_code
            )
    except Exception as e:
        return JSONResponse(
            content={'status': 'error', 'error': str(e)},
            status_code=503
        )

Vigilmon Configuration

1. Sign up at vigilmon.online
2. Add HTTP Monitor → https://yourapp.com/health/elasticsearch
3. Check interval: 1 minute
4. Expected status: 200
5. Alert threshold: 1 failure (Elasticsearch degradation is serious)

Alert on Yellow vs Red

Consider creating two monitors:

• Green+Yellow check (returns 200 for green/yellow, 503 for red): Catches complete cluster failure
• Green-only check (returns 503 for yellow): Catches replica issues before they become full outages

Common Elasticsearch Failure Modes

Unassigned Shards (Yellow)

This usually means:

• A node left the cluster and its shards are waiting to be re-assigned
• Disk space exceeded the watermark threshold
• Not enough nodes to satisfy replica requirements

# Find unassigned shards
curl -X GET 'localhost:9200/_cat/shards?h=index,shard,prirep,state,unassigned.reason&s=state'

JVM Heap Pressure (Slowdown before crash)

When JVM heap fills up:

1. GC runs more frequently (latency spikes)
2. Circuit breakers trip (rejections start)
3. Elasticsearch becomes non-responsive
4. OOM kill

Vigilmon catches step 3-4 as a 503 or timeout.

# Check JVM heap usage
curl -X GET 'localhost:9200/_nodes/stats/jvm?pretty' | grep heap_used_percent

Index Too Large

When a single index gets too large, shard sizes grow beyond optimal (>50GB). Performance degrades before you hit a hard failure.

Vigilmon catches this through response time monitoring — set an alert at 3000ms for your search health endpoint.

Advanced Health Endpoints

// Check specific index health
app.get('/health/elasticsearch/search-index', async (req, res) => {
  try {
    const response = await axios.get(
      `${process.env.ES_URL}/_cat/indices/products?format=json`,
      { auth: { username: ES_USER, password: ES_PASS }, timeout: 5000 }
    );

    const index = response.data[0];
    if (!index) {
      return res.status(503).json({ status: 'error', reason: 'index not found' });
    }

    const isHealthy = index.health !== 'red';
    return res.status(isHealthy ? 200 : 503).json({
      status: index.health,
      docs_count: index['docs.count'],
      store_size: index['store.size']
    });
  } catch (err) {
    return res.status(503).json({ status: 'error', error: err.message });
  }
});

Monitoring Checklist for Elasticsearch

• [ ] Cluster health endpoint exposed via app proxy
• [ ] Vigilmon HTTP monitor at 1-minute intervals
• [ ] Alert on cluster status = red (immediate)
• [ ] Alert on cluster status = yellow (warning)
• [ ] Response time alert at 3000ms
• [ ] Separate monitor for critical indices
• [ ] JVM heap monitoring via CloudWatch/Prometheus

Response Time Thresholds

Response Time	Elasticsearch State	Action
<100ms	Healthy	None
100-500ms	Under load	Monitor closely
500-2000ms	Degraded	Check JVM heap and slow logs
>2000ms	Critical	Likely JVM pressure or disk watermark

Monitor your Elasticsearch cluster for free with Vigilmon →

---

Vigilmon is an uptime monitoring platform for developers. Free tier includes 10 monitors with 1-minute checks from multiple global regions.