DEV Community

Vigilmon
Vigilmon

Posted on

Uptime Monitoring for RabbitMQ Applications (Free, Multi-Region)

Uptime Monitoring for RabbitMQ Applications (Free, Multi-Region)

RabbitMQ is one of the most widely deployed message brokers in the world — reliable, mature, and battle-tested. But it requires active monitoring to use safely in production.

Queues accumulate unacknowledged messages and nobody notices. Consumers crash and the queue depth climbs silently. The management UI shows all nodes green while your application is stuck in a publish loop. This guide shows you how to catch these problems before they become production incidents.


The failure modes that catch RabbitMQ teams off guard

Queue depth explosions — Your consumer goes down. Messages keep arriving. Within hours you have millions of messages in the queue, your broker's memory is exhausted, and the publisher starts blocking. RabbitMQ will trigger the memory alarm and pause all publishers — causing a complete write outage.

Unacknowledged message accumulation — Consumers receive messages but never acknowledge them (a bug in your consumer logic, or a long-running process that holds the channel open). RabbitMQ holds the messages in "unacked" state indefinitely. Eventually the prefetch count fills up and the consumer stops receiving new messages.

Connection churn — Applications that open/close connections frequently (or fail to close them) can trigger RabbitMQ's file descriptor limit. New connections start failing. The broker is healthy; no new connections can be established.

Dead letter queue buildup — Messages rejected or expired by your consumer go to the DLQ. If nothing consumes the DLQ, it grows indefinitely, consuming memory and masking failures in your main queue processing.


Step 1: Use the RabbitMQ Management HTTP API

RabbitMQ's management plugin exposes a REST API at port 15672. This is the fastest way to get health data without a full AMQP connection:

// health/rabbitmq.ts
const RABBIT_API = process.env.RABBITMQ_MANAGEMENT_URL ?? 'http://localhost:15672'
const RABBIT_USER = process.env.RABBITMQ_USER ?? 'guest'
const RABBIT_PASS = process.env.RABBITMQ_PASS ?? 'guest'

const auth = Buffer.from(`${RABBIT_USER}:${RABBIT_PASS}`).toString('base64')
const headers = { Authorization: `Basic ${auth}`, 'Content-Type': 'application/json' }

async function rabbitGet(path: string) {
  const res = await fetch(`${RABBIT_API}/api${path}`, { headers })
  if (!res.ok) throw new Error(`HTTP ${res.status}: ${await res.text()}`)
  return res.json()
}

export async function checkRabbitMQ(): Promise<{
  status: 'ok' | 'warn' | 'error'
  nodeStatus?: string
  alerts?: string[]
  error?: string
}> {
  try {
    // Check node health
    const overview = await rabbitGet('/overview')
    const nodeAlerts: string[] = []

    // Check memory alarm
    if (overview.node?.mem_alarm) {
      nodeAlerts.push('Memory alarm triggered — publishers are blocked')
    }
    // Check disk alarm
    if (overview.node?.disk_free_alarm) {
      nodeAlerts.push('Disk free alarm triggered')
    }

    // Check queue health
    const queues = await rabbitGet('/queues')
    for (const queue of queues) {
      // Warn on deep queues
      if (queue.messages > 10000) {
        nodeAlerts.push(`Queue '${queue.name}' has ${queue.messages.toLocaleString()} messages`)
      }
      // Warn on unacked messages
      if (queue.messages_unacknowledged > 100) {
        nodeAlerts.push(`Queue '${queue.name}' has ${queue.messages_unacknowledged} unacked messages`)
      }
      // Warn if queue has no consumers
      if (queue.consumers === 0 && queue.messages > 0) {
        nodeAlerts.push(`Queue '${queue.name}' has no consumers (${queue.messages} messages waiting)`)
      }
    }

    return {
      status: nodeAlerts.length > 0 ? 'warn' : 'ok',
      nodeStatus: overview.node?.name,
      alerts: nodeAlerts,
    }
  } catch (err: any) {
    return { status: 'error', error: err.message }
  }
}
Enter fullscreen mode Exit fullscreen mode

Step 2: Expose in your health endpoint

// routes/health.ts
import { checkRabbitMQ } from '../health/rabbitmq'

app.get('/health', async (req, res) => {
  const rabbit = await checkRabbitMQ()
  const isOk = rabbit.status === 'ok'

  // Return 503 only on hard errors, 200 with warnings for degraded
  const httpStatus = rabbit.status === 'error' ? 503 : 200

  res.status(httpStatus).json({
    status: rabbit.status,
    checks: { rabbitmq: rabbit },
    timestamp: new Date().toISOString(),
  })
})
Enter fullscreen mode Exit fullscreen mode

Step 3: Python version

# health/rabbitmq.py
import requests
import os

def check_rabbitmq() -> dict:
    base_url = os.environ.get('RABBITMQ_MANAGEMENT_URL', 'http://localhost:15672')
    user = os.environ.get('RABBITMQ_USER', 'guest')
    password = os.environ.get('RABBITMQ_PASS', 'guest')
    auth = (user, password)

    alerts = []
    try:
        overview = requests.get(f'{base_url}/api/overview', auth=auth, timeout=3).json()

        if overview.get('node', {}).get('mem_alarm'):
            alerts.append('Memory alarm active')
        if overview.get('node', {}).get('disk_free_alarm'):
            alerts.append('Disk alarm active')

        queues = requests.get(f'{base_url}/api/queues', auth=auth, timeout=3).json()
        for q in queues:
            if q.get('messages', 0) > 10000:
                alerts.append(f"Queue '{q['name']}' depth: {q['messages']:,}")
            if q.get('consumers', 0) == 0 and q.get('messages', 0) > 0:
                alerts.append(f"Queue '{q['name']}' has no consumers")

        return {
            'status': 'warn' if alerts else 'ok',
            'alerts': alerts,
        }
    except Exception as e:
        return {'status': 'error', 'error': str(e)}
Enter fullscreen mode Exit fullscreen mode

Step 4: Monitor specific queues and DLQs

Add targeted checks for your most important queues:

export async function checkCriticalQueues(queueNames: string[]): Promise<Record<string, any>> {
  const results: Record<string, any> = {}
  for (const name of queueNames) {
    try {
      const queue = await rabbitGet(`/queues/%2F/${encodeURIComponent(name)}`)
      results[name] = {
        status: queue.consumers === 0 ? 'no_consumers' : 'ok',
        depth: queue.messages,
        consumers: queue.consumers,
        unacked: queue.messages_unacknowledged,
      }
    } catch (err: any) {
      results[name] = { status: 'missing', error: err.message }
    }
  }
  return results
}
Enter fullscreen mode Exit fullscreen mode

Check your DLQ explicitly — if it's growing, something in your consumer logic is broken:

const dlqName = 'your-queue.dlq'
const dlqStats = await rabbitGet(`/queues/%2F/${dlqName}`)
if (dlqStats.messages > 0) {
  console.warn(`DLQ has ${dlqStats.messages} messages — investigate consumer errors`)
}
Enter fullscreen mode Exit fullscreen mode

Step 5: Set queue depth thresholds as alerts

Metric Warning Critical
Queue depth > 1,000 > 10,000
Unacked messages > 50 > 200
Consumer count 0 (with depth > 0)
DLQ depth > 0 > 100

Step 6: External monitoring

  1. Go to vigilmon.online — free tier.
  2. Create an HTTP(S) monitor for https://your-app.com/health.
  3. Interval: 60s
  4. Expected status: 200 (or 503 if you want to alert on warnings too)
  5. Regions: 2+ — especially useful if your RabbitMQ cluster is in a specific region

Also monitor the management UI directly if it's exposed:

  • URL: https://rabbitmq.internal.example.com:15671/api/healthchecks/node
  • This built-in RabbitMQ endpoint returns 200 if the node is healthy, 503 otherwise.

Recap

  1. Use RabbitMQ's management HTTP API for fast, lightweight health checks — no full AMQP connection needed.
  2. Check for memory alarms and disk alarms — these cause publishers to block, which your HTTP status endpoint won't reflect.
  3. Monitor queue depth, unacked count, and consumer count per queue.
  4. Check your DLQ explicitly — growth there means silent consumer failures.
  5. Set external monitoring at vigilmon.online for the application layer.

RabbitMQ is reliable. But reliability isn't visibility. Add monitoring and you get both.

Top comments (0)