DEV Community: Swapnil Jaiswal

Monitoring Python RQ jobs: what to watch and how to get alerted

Swapnil Jaiswal — Thu, 09 Jul 2026 18:25:32 +0000

RQ (Redis Queue) is a delightfully simple way to run background jobs in Python. That simplicity is also why teams under-monitor it: it just works, until a downstream API gets slow or a bad deploy ships, and jobs start failing in bulk — quietly. Here's what to watch and how to get alerted before a customer tells you.

RQ failures don't announce themselves

When a job raises, RQ moves it to the FailedJobRegistry and moves on. The worker keeps running; nothing crashes. If you're not looking at that registry, the failure is invisible — the same trap BullMQ, Celery, and every robust queue share. So the job is to reach into the queue's state and turn it into a signal.

The four signals that matter for RQ

Failure count / rate — jobs landing in the FailedJobRegistry over a window.
Backlog — how many jobs are queued vs. being worked; is the worker keeping up?
Latency — how long jobs take, and how long they wait before a worker picks them up.
Worker liveness — are your workers actually alive and heartbeating?

Where to read them

RQ exposes queue and registry state directly:

from redis import Redis
from rq import Queue
from rq.registry import FailedJobRegistry, StartedJobRegistry

redis = Redis()
q = Queue("default", connection=redis)

queued   = len(q)                                   # backlog
failed   = FailedJobRegistry(queue=q)               # failures
started  = StartedJobRegistry(queue=q)              # in-flight

print("queued:", queued)
print("failed:", len(failed))
print("started:", len(started))

Poll this on an interval and store the series — a single snapshot hides the trend, which is the part that matters. For failures specifically, walk the registry to get the actual exceptions:

for job_id in failed.get_job_ids():
    job = q.fetch_job(job_id)
    print(job.id, job.exc_info.splitlines()[-1] if job.exc_info else "")

Two gotchas:

Group by exception, not by job. A thousand jobs failing with the same traceback is one incident. Normalize the message (strip IDs, timestamps, hosts) and group on the rest, or you'll drown in near-identical entries.
Watch worker heartbeats. RQ workers register in Redis; if a worker dies mid-job the job can sit in StartedJobRegistry past its TTL. A rising started-but-never-finished count is a worker-health problem masquerading as a queue problem.

Turning signals into alerts

The metrics above are only useful if something evaluates them and pages a human. Options:

A cron + your own thresholds — cheap, but you build the rate math, grouping, and delivery.
Prometheus + Grafana — export the registry counts (e.g. rq-exporter) and alert in Grafana.
A hosted monitor — send events out and let it handle windows, grouping, and routing.

The principle is identical across every queue: watch from outside the worker, alert on the rate (not a single failure), and group identical failures so a retry storm is one page.

Where PipeRadar fits (and doesn't, yet)

Full disclosure: I work on PipeRadar, a hosted monitor that does exactly this
— failure clustering, latency percentiles, history, and rate-based alerts to Slack/PagerDuty/webhooks
— for BullMQ today. RQ is on the roadmap, not shipping yet, so I'm not going to pretend you can pip install it this afternoon.

What's relevant now: PipeRadar's ingest API is queue-agnostic (it already models an adapter type per queue system), so an RQ adapter is the missing piece, not a rewrite. If RQ monitoring like this is something you'd use, the fastest way to pull it up the roadmap is to say so — you can follow progress and register interest at piperadar.dev. In the meantime, the patterns above work with nothing but Redis and a cron.

More on background-job monitoring — including a deep dive on
why queue jobs fail silently (BullMQ, but the failure model is the same in RQ) — at piperadar.dev/blog.

How to get alerted when a BullMQ job fails (before your users do)

Swapnil Jaiswal — Thu, 09 Jul 2026 15:30:33 +0000

If you run BullMQ in production, you've probably had this moment: a customer tells you their payment/email/export never went through, you go digging, and you find the failed job sitting in Redis — recorded perfectly, retried exactly as configured, and completely silent. Nothing crashed.
No process exited non-zero. Nobody got paged.

That's not a bug. It's how a good queue is supposed to behave. This post is about turning that silence back into an alert, with three concrete approaches and their trade-offs.

Why BullMQ failures are silent by default

To a queue, failed is a normal lifecycle state, right next to completed and waiting. When a job throws and exhausts its attempts, BullMQ moves it into the failed set and picks up the next job. Your worker is healthier for handling it gracefully — which is exactly why none of your usual signals (a dead process, an HTTP 500, a crash loop) ever fire.

Two things make it worse:

Retries hide the early warning. A job with attempts: 5 fails four times quietly before it fails "for real." By the time it lands in the failed set, the queue's been unhealthy for minutes.
The failed event only fires where you're listening. Scale to four worker pods and each one only sees its own failures. Redeploy and the listener is gone until the new process boots.

Approach 1: a `failed` listener (the 15-minute version)

The instinct is to attach a listener:

import { Worker } from 'bullmq'

const worker = new Worker('payments', processPayment)

worker.on('failed', (job, err) => {
  console.error(job?.id, err.message)
  // ...post to Slack?
})

This is better than nothing, but it has three problems every production setup hits:

It only runs in this process. No aggregate view across pods.
It logs, and logs aren't alerts. A console.error goes into a stream nobody tails at 3am.
It restarts with the process. Jobs that fail during a redeploy are recorded in Redis but never emitted to your handler.

If you go this route, at minimum use QueueEvents instead of worker.on — it reads lifecycle events over Redis pub/sub, so it sees every worker's jobs:

import { QueueEvents } from 'bullmq'

const events = new QueueEvents('payments')
events.on('failed', ({ jobId, failedReason }) => {
  // count it, don't just log it — and alert on the *rate*, not a single failure
})

The hard part isn't catching the event. It's everything after: counting failures over a rolling window, deciding what's a real incident vs. noise, grouping 1,000 identical errors into one page, and routing to a human with a cooldown so you're not paged in a loop.

(This is the exact problem we dug into in why BullMQ jobs fail silently)

Approach 2: Prometheus + Grafana

If you already run a metrics stack, you can export queue metrics (there are community exporters like bull-monitor, or roll your own from QueueEvents) and build Grafana alerts.

Good for: full control, unified with your other dashboards.
Cost: days of wiring exporters and tuning alert rules, and you still build failure grouping yourself. Worth it if you have the platform team; heavy if you don't.

Approach 3: a hosted monitor

Dashboards like Taskforce.sh (from the BullMQ team) give you a polished UI with metrics and email alerts by connecting to your Redis. If you want zero-Redis-access and root-cause failure clustering with alerts to Slack/PagerDuty/webhooks, that's what we built PipeRadar for. It's an SDK you drop into your worker — it hooks the same QueueEvents and pushes lightweight metadata out over HTTPS, so it never connects to your Redis or reads your job payloads:

import { PipeRadar } from '@piperadar/bullmq'

const pr = PipeRadar({ apiKey: 'pr_live_...' })
pr.watch(paymentQueue)   // failure rate, latency, backlog & alerts — done

It counts completions and failures per minute, fingerprints failures into incidents (so 1,000 identical errors are one alert, not a thousand), keeps history, and pages you on a rate change. There's a free tier, no credit card.

Which should you pick?

Just need to stop the bleeding today? QueueEvents + a Slack webhook, alert on a failure count.
Already have Grafana and a platform team? Export metrics and build the rules there.
Want alerting + grouping + history without wiring it yourself? A hosted monitor.

Whatever you choose, the principle is the same: watch from outside the worker, alert on the rate (not a single failure), and group identical failures so a retry storm is one page.

I write about running BullMQ in production at piperadar.dev/blog — including a deeper dive on why BullMQ jobs fail silently and the complete monitoring guide.