DEV Community

Atlas Whoff
Atlas Whoff

Posted on

BullMQ: Production-Grade Job Queues for Node.js

#node #redis #typescript #queues

BullMQ: Production-Grade Job Queues for Node.js

Doing heavy work inside an HTTP request is a recipe for timeouts and poor UX. BullMQ moves work out of the request cycle into background workers — with retries, priorities, and rate limiting built in.

Why a Job Queue

Don't do in a request:

  • Sending emails (slow, can fail)
  • Image processing (CPU-bound)
  • Calling external APIs (unreliable)
  • Generating reports (memory-heavy)

Do instead: add a job to the queue, return 200 immediately, process in a worker.

Setup 

npm install bullmq ioredis
Enter fullscreen mode Exit fullscreen mode

Producer 

import { Queue } from 'bullmq';
import { Redis } from 'ioredis';

const connection = new Redis(process.env.REDIS_URL!, { maxRetriesPerRequest: null });

export const emailQueue = new Queue('emails', { connection });
export const imageQueue = new Queue('images', { connection });

// Add jobs from your API routes
await emailQueue.add('welcome', {
  to: user.email,
  firstName: user.firstName,
}, {
  attempts: 3,
  backoff: { type: 'exponential', delay: 2000 },
});
Enter fullscreen mode Exit fullscreen mode

Worker 

import { Worker } from 'bullmq';

const emailWorker = new Worker('emails', async (job) => {
  switch (job.name) {
    case 'welcome':
      await sendWelcomeEmail(job.data.to, job.data.firstName);
      break;
    case 'receipt':
      await sendReceiptEmail(job.data.to, job.data.amount);
      break;
  }
}, { connection, concurrency: 5 });

emailWorker.on('completed', (job) => console.log(`Job ${job.id} done`));
emailWorker.on('failed', (job, err) => console.error(`Job ${job?.id} failed:`, err));
Enter fullscreen mode Exit fullscreen mode

Scheduled Jobs (Cron) 

await reportQueue.add('daily-report', {}, {
  repeat: { cron: '0 9 * * *' }, // 9 AM every day
  jobId: 'daily-report', // Prevent duplicates
});
Enter fullscreen mode Exit fullscreen mode

Flow — Job Dependencies 

import { FlowProducer } from 'bullmq';

const flow = new FlowProducer({ connection });

// Run jobs in order: resize → watermark → upload
await flow.add({
  name: 'upload',
  queueName: 'images',
  children: [{
    name: 'watermark',
    queueName: 'images',
    children: [{
      name: 'resize',
      queueName: 'images',
      data: { imageId: '123', targetWidth: 800 }
    }]
  }]
});
Enter fullscreen mode Exit fullscreen mode

Rate Limiting Workers 

// Process max 100 jobs per minute (e.g. external API rate limit)
const worker = new Worker('api-calls', processor, {
  connection,
  limiter: { max: 100, duration: 60000 },
});
Enter fullscreen mode Exit fullscreen mode

Monitoring with Bull Board 

import { createBullBoard } from '@bull-board/api';
import { BullMQAdapter } from '@bull-board/api/bullMQAdapter';
import { ExpressAdapter } from '@bull-board/express';

const serverAdapter = new ExpressAdapter();
createBullBoard({
  queues: [new BullMQAdapter(emailQueue), new BullMQAdapter(imageQueue)],
  serverAdapter,
});

app.use('/admin/queues', serverAdapter.getRouter());
// Dashboard at /admin/queues — view jobs, retry failures, clear queues
Enter fullscreen mode Exit fullscreen mode

BullMQ job queues are included in the AI SaaS Starter Kit — email queue, webhook processing queue, and Bull Board dashboard pre-configured. $99 at whoffagents.com.

Top comments (0)