Scaling Background Jobs With BullMQ and Redis in Node.js

In Node.js applications, handling time-consuming tasks like sending emails, processing images, or interacting with third-party APIs can be offloaded to background jobs. BullMQ, backed by Redis, is a modern and performant queueing solution perfect for production-scale applications.

Why Use BullMQ?

BullMQ is a Node.js job queue built on top of Redis. It's the successor to Bull, with a modular architecture and support for robust job management, repeatable jobs, rate-limiting, and more.

Step 1: Install Dependencies

First, add BullMQ and Redis to your project:

npm install bullmq ioredis

Step 2: Set Up the Queue

Create a queue instance using BullMQ:

// queue.js
const { Queue } = require('bullmq');
const connection = { host: 'localhost', port: 6379 };

const emailQueue = new Queue('emailQueue', { connection });

module.exports = emailQueue;

Step 3: Add Jobs to the Queue

You can enqueue jobs like this:

// addJob.js
const emailQueue = require('./queue');

emailQueue.add('sendWelcomeEmail', {
  to: '[e-mail]',
  subject: 'Welcome!',
  body: 'Thanks for joining us!'
});

Step 4: Process Jobs

Now create a worker to handle job processing:

// worker.js
const { Worker } = require('bullmq');
const connection = { host: 'localhost', port: 6379 };

const worker = new Worker('emailQueue', async job => {
  const { to, subject, body } = job.data;
  console.log(`Sending email to ${to} with subject "${subject}"`);
  // simulate email sending
}, { connection });

worker.on('completed', job => {
  console.log(`Job ${job.id} completed`);
});

Step 5: Monitor With Bull Board (Optional)

Use bull-board to monitor and manage your queues via a web UI:

npm install @bull-board/api @bull-board/express

// monitor.js
const express = require('express');
const { createBullBoard } = require('@bull-board/api');
const { BullMQAdapter } = require('@bull-board/api/bullMQAdapter');
const { ExpressAdapter } = require('@bull-board/express');
const emailQueue = require('./queue');

const serverAdapter = new ExpressAdapter();
const app = express();

createBullBoard({
  queues: [new BullMQAdapter(emailQueue)],
  serverAdapter,
});

serverAdapter.setBasePath('/admin/queues');
app.use('/admin/queues', serverAdapter.getRouter());
app.listen(3000, () => console.log('Dashboard running at localhost:3000'));

Conclusion

With BullMQ and Redis, you can scale job processing efficiently and reliably in Node.js. It’s a modern solution for asynchronous workflows, built for performance and flexibility.

If this post helped you, consider supporting me: buymeacoffee.com/hexshift