🚀 Handling Cron Jobs in NestJS with Multiple Instances using Bull

Introduction

Cron jobs are essential when you need to execute scheduled tasks in your application. But what happens when you have multiple instances running? 🤯 Without proper handling, each instance might execute the job simultaneously, causing duplicate processing and potential data inconsistencies.

A solid solution is using Bull (a Node.js queue library based on Redis) to ensure that only one instance executes the cron job at a time. In this article, we'll explore how to achieve this using NestJS and Bull.

---

🛠 Setting Up Bull in NestJS

1️⃣ Install Dependencies

First, install Bull and Redis client:

npm install --save @nestjs/bull bull ioredis

2️⃣ Configure BullModule

In your app.module.ts, configure Bull to use Redis:

import { Module } from '@nestjs/common';
import { BullModule } from '@nestjs/bull';
import { MyCronJobProcessor } from './cron.processor';
import { MyCronJobService } from './cron.service';

@Module({
  imports: [
    BullModule.forRoot({
      redis: {
        host: 'localhost', // Use the Redis host
        port: 6379, // Default Redis port
      },
    }),
    BullModule.registerQueue({
      name: 'cronQueue',
    }),
  ],
  providers: [MyCronJobProcessor, MyCronJobService],
})
export class AppModule {}

Note: Ensure Redis is running locally or use a cloud-hosted Redis service.

---

🎯 Implementing the Cron Job

3️⃣ Creating the Cron Job Service

We’ll create a service that adds jobs to the queue at scheduled intervals.

import { Injectable } from '@nestjs/common';
import { InjectQueue } from '@nestjs/bull';
import { Queue } from 'bull';
import { Cron } from '@nestjs/schedule';

@Injectable()
export class MyCronJobService {
  constructor(@InjectQueue('cronQueue') private cronQueue: Queue) {}

  @Cron('*/5 * * * * *') // Runs every 5 seconds
  async scheduleJob() {
    await this.cronQueue.add('processData', {});
    console.log('Cron job added to the queue ✅');
  }
}

The @Cron decorator schedules the job at a fixed interval, ensuring that the task is queued rather than executed by every instance.

---

4️⃣ Processing the Cron Job

Only one instance should process the job at a time. Bull takes care of that for us! 🎉

import { Process, Processor } from '@nestjs/bull';
import { Job } from 'bull';

@Processor('cronQueue')
export class MyCronJobProcessor {
  @Process('processData')
  async handleCronJob(job: Job) {
    console.log('Processing cron job... 🤖', job.id);
    // Your task logic here (e.g., database cleanup, report generation)
  }
}

---

📦 Create a Docker compose

If you run multiple instances of your NestJS app, only one instance will process the queued job, thanks to Redis locking mechanisms in Bull.

version: '3.8'

services:
  redis:
    image: redis:6.2
    restart: always
    ports:
      - '6379:6379'
    networks:
      - app-network

  worker:
    image: node:18
    working_dir: /app
    volumes:
      - .:/app
    command: ["npm", "run", "start"]
    environment:
      - NODE_ENV=production
      - REDIS_HOST=redis
    networks:
      - app-network
    deploy:
      replicas: 3  # Example: Run 3 instances of the worker
      restart_policy:
        condition: on-failure

networks:
  app-network:
    driver: bridge

Run:

docker-compose up -d

Now, your Redis instance is ready to use! 🚀

---

✅ Conclusion

Using Bull with Redis in NestJS, we’ve ensured that:

✔️ Cron jobs are scheduled only once per interval
✔️ Multiple instances don’t trigger duplicate executions
✔️ Scalability is achieved with a queue-based approach

Now you’re ready to handle scheduled tasks like a pro! 💪🔥

Happy coding! 🚀