How to Monitor Your RabbitMQ Message Queue with Vigilmon

RabbitMQ is one of the most widely deployed message brokers — used for async job processing, microservice communication, event streaming, and decoupling workloads. When RabbitMQ goes down or queues start piling up, the effects ripple through your entire system: emails don't send, background jobs freeze, and user actions silently fail.

This guide shows you how to monitor RabbitMQ effectively with Vigilmon.

---

The RabbitMQ Management HTTP API

RabbitMQ includes a built-in Management Plugin that exposes a REST API. If it's enabled, you have monitoring data available immediately:

# Check cluster health
curl -u guest:guest http://localhost:15672/api/health/checks/aliveness

# Response: {"status":"ok"}

# Check queue depth
curl -u guest:guest http://localhost:15672/api/queues/%2F/my-queue

The management API runs on port 15672 by default and requires authentication.

---

Step 1: Create a RabbitMQ Health Proxy

Never expose RabbitMQ's management port publicly. Instead, proxy a health check through your application:

Node.js (Express) example:

const express = require('express');
const amqp = require('amqplib');

const app = express();
let channel = null;

async function connectRabbit() {
  const conn = await amqp.connect(process.env.RABBITMQ_URL);
  channel = await conn.createChannel();
}

connectRabbit();

app.get('/health/rabbitmq', async (req, res) => {
  try {
    if (!channel) throw new Error('Channel not initialized');

    // Passive check: inspect queue without creating it
    const q = await channel.checkQueue('main-jobs');
    const messageCount = q.messageCount;

    if (messageCount > 10000) {
      return res.status(503).json({
        status: 'queue_overflow',
        queue: 'main-jobs',
        messages: messageCount
      });
    }

    res.json({
      status: 'ok',
      queue: 'main-jobs',
      pending_messages: messageCount,
      consumers: q.consumerCount
    });
  } catch (err) {
    res.status(503).json({ status: 'error', message: err.message });
  }
});

Python (FastAPI + aio-pika) example:

from fastapi import FastAPI, Response
import aio_pika
import json

app = FastAPI()

@app.get("/health/rabbitmq")
async def health_rabbitmq():
    try:
        connection = await aio_pika.connect_robust(
            "amqp://guest:guest@localhost/"
        )
        async with connection:
            channel = await connection.channel()
            queue = await channel.get_queue("main-jobs", ensure=True)
            declaration = await queue.declare(passive=True)
            msg_count = declaration.message_count

            if msg_count > 10000:
                return Response(
                    content=json.dumps({"status": "overflow", "messages": msg_count}),
                    status_code=503
                )

            return {"status": "ok", "pending_messages": msg_count}
    except Exception as e:
        return Response(content=str(e), status_code=503)

---

Step 2: Add an HTTP Monitor in Vigilmon

1. Log in to vigilmon.online → Add Monitor
2. Type: HTTP(S)
3. URL: https://your-app.com/health/rabbitmq
4. Interval: 60 seconds
5. Alert if: Status != 200 or response > 3000ms

Vigilmon's multi-region checks mean you'll know immediately if your message broker is unreachable from multiple geographic locations — not just a probe-side fluke.

---

Step 3: Monitor Consumer Count (Dead Consumer Detection)

Queues piling up with zero consumers is a critical failure mode — jobs accumulate indefinitely:

app.get('/health/rabbitmq/consumers', async (req, res) => {
  try {
    const queues = ['email-queue', 'notification-queue', 'job-queue'];
    const results = {};
    let hasIssue = false;

    for (const qName of queues) {
      const q = await channel.checkQueue(qName);
      results[qName] = {
        messages: q.messageCount,
        consumers: q.consumerCount
      };
      if (q.consumerCount === 0 && q.messageCount > 0) {
        hasIssue = true;
      }
    }

    if (hasIssue) {
      return res.status(503).json({ status: 'dead_consumers', queues: results });
    }

    res.json({ status: 'ok', queues: results });
  } catch (err) {
    res.status(503).json({ status: 'error', message: err.message });
  }
});

This alerts when a consumer process crashes and leaves messages in a queue with no one to process them.

---

Step 4: Heartbeat Monitor for Your Consumer Processes

For background consumers (Celery workers, Sidekiq, custom consumers), add a liveness heartbeat:

# In your RabbitMQ consumer process
import urllib.request
import threading
import time

HEARTBEAT_URL = "https://hb.vigilmon.online/YOUR-HEARTBEAT-ID"

def send_heartbeat():
    while True:
        try:
            urllib.request.urlopen(HEARTBEAT_URL, timeout=5)
        except:
            pass
        time.sleep(60)  # Ping every 60 seconds

# Start heartbeat thread when consumer starts
threading.Thread(target=send_heartbeat, daemon=True).start()

# Your consumer loop
for message in queue:
    process_message(message)

Configure Vigilmon to alert if no heartbeat is received in 5 minutes — catching consumer crashes immediately.

---

RabbitMQ Monitoring Coverage Table

Monitor	Endpoint	Alerts On
Broker liveness	/health/rabbitmq	Broker unreachable
Queue overflow	/health/rabbitmq	Messages > 10,000
Dead consumers	/health/rabbitmq/consumers	Consumer = 0 with pending msgs
Consumer heartbeat	Vigilmon Heartbeat	Consumer process died

---

Common RabbitMQ Failure Patterns

Consumer crash at 3 AM: An email consumer process hit an OOM error and died. Email queue grew from 10 to 50,000 messages before anyone noticed at 9 AM. Heartbeat monitor would have alerted at 3:05 AM.

Broker disk alarm: RabbitMQ's flow control kicks in when disk space drops below a threshold. All publishers block. The health check caught the 503 within 60 seconds.

Memory high watermark hit: RabbitMQ paused all activity when heap exceeded 40% of system RAM. The broker was technically reachable but publishing was blocked. Health check returned 503; Vigilmon paged the team.

---

Conclusion

RabbitMQ failures are invisible until they're catastrophic. With Vigilmon's multi-region HTTP monitoring and heartbeat checks on your consumer processes, you get real-time visibility into broker health, queue depth, and consumer liveness.

Start monitoring your RabbitMQ cluster free at vigilmon.online