How to Monitor NestJS Queues and BullMQ Jobs with Vigilmon

#nestjs #bullmq #queues #monitoring

BullMQ queues in NestJS handle critical background work: sending emails, processing payments, generating reports. When a queue worker dies silently, you don't know until users complain. This guide shows how to monitor NestJS queues with Vigilmon heartbeat monitoring.

The Silent Worker Death Problem

BullMQ workers process jobs in the background. If a worker crashes:

Jobs pile up in the queue
Users don't get confirmation emails
Scheduled tasks stop running
No one knows until it's too late

Vigilmon heartbeat monitoring solves this: the worker pings Vigilmon after each job batch. If the ping stops arriving, you get an alert.

Step 1: Create a Heartbeat Service

// heartbeat/heartbeat.service.ts
import { Injectable, Logger } from '@nestjs/common';
import { ConfigService } from '@nestjs/config';

@Injectable()
export class HeartbeatService {
  private readonly logger = new Logger(HeartbeatService.name);
  private readonly heartbeatUrl: string;

  constructor(private config: ConfigService) {
    this.heartbeatUrl = config.get('VIGILMON_HEARTBEAT_URL', '');
  }

  async ping(label?: string): Promise<void> {
    if (!this.heartbeatUrl) return;
    try {
      await fetch(this.heartbeatUrl);
    } catch (error) {
      this.logger.warn('Heartbeat failed:', error.message);
    }
  }
}

Step 2: Add Heartbeat to Your Queue Processor

// email/email.processor.ts
import { Processor, WorkerHost } from '@nestjs/bullmq';
import { Job } from 'bullmq';
import { HeartbeatService } from '../heartbeat/heartbeat.service';

@Processor('email-queue')
export class EmailProcessor extends WorkerHost {
  constructor(private heartbeat: HeartbeatService) { super(); }

  async process(job: Job): Promise<void> {
    try {
      switch (job.name) {
        case 'send-welcome':
          await this.sendWelcomeEmail(job.data);
          break;
      }
      await this.heartbeat.ping(`email:${job.name}`);
    } catch (error) {
      throw error; // Let BullMQ handle retry
    }
  }
}

Step 3: Scheduled Heartbeat for Idle Workers

When queues are empty, add a scheduled heartbeat:

@Injectable()
export class HeartbeatScheduler {
  constructor(
    private heartbeat: HeartbeatService,
    @InjectQueue('email-queue') private emailQueue: Queue,
  ) {}

  @Cron(CronExpression.EVERY_MINUTE)
  async sendWorkerHeartbeat() {
    const ready = await this.emailQueue.isReady();
    if (ready) await this.heartbeat.ping('worker-alive');
  }
}

Step 4: Set Up the Vigilmon Heartbeat Monitor

In Vigilmon:

Create a new Heartbeat monitor
Name it: "NestJS Email Queue Worker"
Period: 2 minutes
Copy the URL to VIGILMON_HEARTBEAT_URL env var

Step 5: Queue Health HTTP Endpoint

@Controller('health/queues')
export class QueueHealthController {
  constructor(@InjectQueue('email-queue') private emailQueue: Queue) {}

  @Get()
  async checkQueues() {
    const [waiting, active, failed] = await Promise.all([
      this.emailQueue.getWaitingCount(),
      this.emailQueue.getActiveCount(),
      this.emailQueue.getFailedCount(),
    ]);

    const degraded = failed > 50;
    return {
      status: degraded ? 'degraded' : 'ok',
      queues: { email: { waiting, active, failed } }
    };
  }
}

Monitor https://api.yourapp.com/health/queues in Vigilmon.

What Vigilmon Catches for BullMQ

Failure	Detection
Worker process crashed	Heartbeat stops arriving
Redis connection lost	Worker can't process, heartbeat stops
Worker stuck on hung job	Heartbeat gap
High failure rate	HTTP health endpoint alerts

Summary

Create HeartbeatService wrapping the Vigilmon ping
Call heartbeat.ping() after successful job processing
Add scheduled heartbeat for idle-but-healthy workers
Create a Vigilmon heartbeat monitor with 2-minute period
Add HTTP health endpoint for queue metrics

Vigilmon — heartbeat and uptime monitoring for NestJS, BullMQ, and every background job system. Free plan available.

DEV Community