DEV Community: Ronak Navadia

Stop Confusing Workers with Concurrency

Ronak Navadia — Tue, 14 Apr 2026 17:35:32 +0000

A conversation with every developer who's ever asked "but aren't they the same thing?"

Let me tell you about a conversation I've had at least a dozen times.

A developer comes to me, frustrated. They've set up Bull for background jobs. They've even set the concurrency option to 5, feeling confident. But their API is still grinding to a halt whenever heavy jobs kick in. And when they try to "scale their workers", they end up with five copies of their entire API running — which wasn't the plan at all.

Sound familiar? Let's fix this, once and for all — not with theory, but with the clear mental model you actually need.

First, what actually happens when you run your app?

When you run node dist/main.js, something very specific happens on your machine:

💡 The truth: You start one Node.js process. One event loop. One chunk of memory. One running instance of your entire application.

That's it. Everything you've built — your controllers, your services, your Bull job processors — all of it lives inside that single process.

This isn't a problem at first. But as your system grows, that single process starts to become a bottleneck. And this is exactly where the confusion about "workers" begins.

So what is a "worker", really?

Here's the thing nobody says clearly enough: a worker is just another Node.js process.

Not a thread. Not a magic Bull setting. Not a special NestJS construct. It's a process — the same kind of process you start when you run node dist/main.js. The only difference is in what it does once it starts.

	API Process	Worker Process
Starts with	`app.listen(3000)`	`createApplicationContext()`
Does	Waits for HTTP requests	Polls a queue for jobs
Handles	Validation, auth, routing	Sending emails, processing files, etc.
Visibility	Has a port the world can reach	No port, invisible from outside

A process becomes an API server only because it calls app.listen(). A process becomes a worker only because it processes background jobs instead.

"API and Worker are not different code — they are different ways of running your app."

Same code. Different entry point. Different role.

Then what does Bull's `concurrency` option actually do?

This is the question that trips everyone up. And it's a completely fair question.

When you write something like this:

email.processor.ts

@Processor('email')
export class EmailProcessor {
  @Process({ concurrency: 5 })
  async handle(job: Job) {
    // send the email
  }
}

You are not creating 5 workers. You are telling the one worker process that's already running to handle up to 5 jobs at the same time — all within its single event loop.

Term	What it means
Concurrency	How many jobs one process handles simultaneously
Multiple Workers	How many separate processes are doing the handling

Think of it this way:

🍳 Concurrency = one chef trying to watch 5 pots at once
👨‍🍳👨‍🍳👨‍🍳 Multiple workers = hiring 5 separate chefs, each at their own stove

For I/O-heavy jobs (like sending emails or making API calls), concurrency works beautifully.
For CPU-heavy jobs (like image processing or video encoding), you want separate processes — because CPU work actually blocks the event loop.

Why mixing API and Worker in one process causes pain

Here's how most NestJS apps start out. Everything in one process:

app.module.ts (the problematic way)

@Module({
  imports: [
    BullModule.forRoot({ ... }),
    BullModule.registerQueue({ name: 'email' }),
    EmailModule, // contains the @Processor
  ],
})
export class AppModule {}

When you start this app with node dist/main.js, you get:

✅ HTTP server listening on port 3000
✅ Bull processor actively picking up jobs

Fine for small scale. But three ugly problems emerge as you grow:

⚠️ Problem 1 · Slow API

A heavy background job — resizing images, crunching data — hogs the event loop. Your API users start seeing timeouts. They didn't change anything. Your background jobs strangled the API.

⚠️ Problem 2 · Wasteful Scaling

You want 3 more job processors, so you spin up 3 more instances of your app. Surprise — you now have 3 extra API servers too, all competing on the same port or behind a load balancer.

You wanted kitchen staff. You hired three more waiters who also cook.

⚠️ Problem 3 · No Isolation

A runaway job crashes the process. Down goes your API too. Two unrelated concerns, one shared fate.

The fix: two entry points, two roles

The solution is elegant. You write two entry point files — one for each role. They both import AppModule, but they start it differently.

main.ts — the API process

async function bootstrap() {
  const app = await NestFactory.create(AppModule);
  await app.listen(3000); // gives it a port — makes it an API
}
bootstrap();

worker.ts — the worker process

async function bootstrap() {
  await NestFactory.createApplicationContext(AppModule);
  // no port — just loads modules and starts processing
}
bootstrap();

Now you can start them independently:

node dist/main.js    # starts the API
node dist/worker.js  # starts a worker
node dist/worker.js  # starts another worker

Three separate processes. One handles HTTP. Two process jobs. They never step on each other.

The hidden gotcha you will hit

Even after splitting your entry points, there's a trap. Both files still import AppModule — and AppModule still registers your Bull processor. So your "API process" will also start picking up jobs. You've separated the files but not the behaviour.

Fix it with an environment variable:

app.module.ts — now with role awareness

@Module({
  imports: [
    // Only load the job processor in worker processes
    ...(process.env.RUN_WORKER === 'true'
      ? [EmailProcessorModule]
      : []),
  ],
})
export class AppModule {}

Then start them like this:

# API — no job processing
RUN_WORKER=false node dist/main.js

# Worker — only job processing
RUN_WORKER=true node dist/worker.js

Now your API process has zero awareness of job processors. It queues jobs and walks away. The worker processes pick those jobs up.

How they talk to each other (they don't, directly)

Your API and your workers never call each other. There are no inter-process function calls. Instead, they communicate through Redis via Bull's queue:

┌────────────┐       ┌─────────────────────────┐       ┌─────────────┐       ┌──────────────────────────┐
│   Client   │  ───► │      API Process        │  ───► │ Redis Queue │  ───► │      Worker Process      │
│            │       │   adds job to queue     │       │             │       │  picks up & executes     │
└────────────┘       └─────────────────────────┘       └─────────────┘       └──────────────────────────┘

In your API controller:

await this.emailQueue.add('sendWelcome', { userId: 42 });
// API's job is done. It forgets about this immediately.

In your worker processor:

@Process('sendWelcome')
async handleWelcome(job: Job) {
  await this.mailer.send(job.data.userId);
  // Worker handles it whenever it's free.
}

The queue is the contract between them. Durable, decoupled, and fault-tolerant.

Managing it all with PM2

Manually running terminal commands gets old fast. PM2 is the standard solution — it manages your processes, restarts them on crash, and lets you scale workers with a single flag.

# Start the API (1 instance)
pm2 start dist/main.js --name api --env RUN_WORKER=false

# Start 3 worker processes
pm2 start dist/worker.js --name worker -i 3 --env RUN_WORKER=true

# See what's running
pm2 list

# Scale workers to 5 without restarting anything
pm2 scale worker 5

This gives you independent control. Need to handle a spike in emails? Scale the workers. Getting more API traffic? Scale the API. They move independently because they are independent.

The restaurant analogy (because it genuinely helps)

Your system is a restaurant. Here's the full cast:

Restaurant Role	Your System Equivalent	Technical Term
Customer walks in	HTTP request arrives	Client request
Waiter takes the order	API receives request, validates, queues a job	API Process
Order ticket goes to kitchen	Job lands in Redis	Bull Queue
Chef cooks the meal	Worker picks up the job and executes it	Worker Process
Chef can multitask on 3 orders	One worker with `concurrency: 3`	Bull concurrency
Hiring a second chef	Starting a second worker process	Process scaling

A great restaurant doesn't make the waiter cook and manage inventory at the same time. Separation of concerns isn't just an architectural principle — it's common sense.

The complete picture, in one place

Let's bring it all together with one final summary. The key ideas, no fluff:

Concept	What it means
A process	A running Node.js app with its own memory and event loop
An API process	A process that called `app.listen()`
A worker process	A process that didn't — it just processes queue jobs
Concurrency	How many jobs one worker handles at once, not separate processes
Separation	Different entry points + env flags to control which modules load
PM2	The thing that manages, monitors, and scales all of it

"Your API handles requests. Your workers handle work.
They share a codebase — but never a process."

Once this clicks, a whole category of "why is my API slow?" and "why are my jobs not running where I expect?" questions just… disappear. You start thinking in processes instead of in files, and that changes how you architect everything.

If you have questions, or if something in here is still fuzzy — that's what the comments are for. I've had this conversation many times, and each version of it has made this explanation a little sharper. Your confusion is probably someone else's confusion too.

The Queue Whisperer: Taming Bull Jobs in NestJS Like a Pro

Ronak Navadia — Tue, 14 Apr 2026 17:32:49 +0000

Picture this: your app is blowing up. Hundreds of appointment confirmations need to go out, 48-hour reminders are piling up, and your server is sweating. You could process everything in one big, synchronous pile — but that's like trying to run a restaurant where one chef cooks, serves, cleans, and takes reservations all at once.

Enter Bull — NestJS's battle-tested job queue library. And today, we're going to talk about the unsung heroes of Bull: consumers and concurrency.

Grab a coffee. Let's fix that restaurant. ☕

The Problem: Jobs Are Piling Up

Every non-trivial backend eventually faces this moment. You need to:

Send an order confirmation the second someone books an appointment
Fire off a reminder 48 hours before that appointment
Not block your main thread doing any of it

Bull solves this beautifully — but how you set up your consumers determines whether your queue hums like a well-oiled machine or groans under pressure.

There are three patterns worth knowing. Each has its moment to shine.

Pattern 1: The One-Chef Kitchen (Single Consumer)

Sometimes, simplicity wins. One processor class. Multiple job handlers. Clean, contained, easy to reason about.

Think of it as a single skilled chef who knows how to make every dish — they just work on a few orders at a time.

@Processor('appointmentQueue')
export class AppointmentProcessor {

  // Chef's special #1: Order Confirmations
  @Process('sendOrderConfirmationMessageJob', { concurrency: 3 })
  async handleOrderConfirmationMessage(job: Job) {
    console.log('Processing order confirmation job:', job.id);
    // Your magic here
  }

  // Chef's special #2: 48-Hour Reminders
  @Process('send48HourReminderForAppointmentJob', { concurrency: 2 })
  async handle48HourReminder(job: Job) {
    console.log('Processing 48-hour reminder job:', job.id);
    // Your magic here
  }
}

What's happening here?

One class, two jobs — AppointmentProcessor owns everything related to appointments.
Concurrency per job — Notice how sendOrderConfirmationMessageJob gets 3 workers while the reminder job gets 2. You're not guessing; you're deliberately allocating processing power based on expected load.
The { concurrency: X } option inside @Process() is your dial — turn it up for high-traffic jobs, keep it lower for jobs that are resource-heavy or rate-limited by external APIs.

When to use this: Small-to-medium queues, tightly related job types, or when you want everything in one place. Great for getting started fast.

Pattern 2: The Brigade Kitchen (Multiple Consumers)

Now imagine your restaurant got a Michelin star. One chef can't cut it anymore. You hire specialists — a pasta chef, a pastry chef, a grill master.

This is the multi-consumer pattern: each job type gets its own dedicated processor class.

// Specialist #1: The Confirmation Pro
@Processor('appointmentQueue')
export class OrderConfirmationProcessor {

  @Process('sendOrderConfirmationMessageJob', { concurrency: 3 })
  async handleOrderConfirmation(job: Job) {
    console.log('Processing order confirmation job:', job.id);
    // Confirmation logic
  }
}

// Specialist #2: The Reminder Guru
@Processor('appointmentQueue')
export class ReminderProcessor {

  @Process('send48HourReminderForAppointmentJob', { concurrency: 2 })
  async handle48HourReminder(job: Job) {
    console.log('Processing 48-hour reminder job:', job.id);
    // Reminder logic
  }
}

Why split them up?

Separation of concerns — Each class does one thing and does it well. When the reminder logic needs to change, you open one small file, not a sprawling processor.
Independent scaling — Need more confirmation workers? Crank up the concurrency on OrderConfirmationProcessor without touching anything else.
Easier testing — Unit testing a focused class is a joy compared to wrestling with a monolith.

When to use this: Growing codebases, jobs with very different logic or dependencies, or teams where different developers own different job types.

Pattern 3: The Ghost Kitchen (Dynamic Processing with `onModuleInit`)

Here's where things get interesting. What if you don't want to use decorators at all? What if you want to wire everything up programmatically, maybe based on config, environment variables, or runtime conditions?

Meet the ghost kitchen — fully functional, no storefront, completely dynamic.

import { InjectQueue } from '@nestjs/bull';
import { Queue } from 'bull';
import { Injectable, OnModuleInit } from '@nestjs/common';

@Injectable()
export class JobProcessingService implements OnModuleInit {
  constructor(
    @InjectQueue('appointmentQueue') private appointmentQueue: Queue
  ) {}

  async onModuleInit() {
    // Wire up confirmation jobs with 3 concurrent workers
    await this.appointmentQueue.process(
      'sendOrderConfirmationMessageJob',
      3,
      async (job) => {
        console.log('Processing order confirmation job:', job.id);
        // Confirmation logic
      }
    );

    // Wire up reminder jobs with 2 concurrent workers
    await this.appointmentQueue.process(
      'send48HourReminderForAppointmentJob',
      2,
      async (job) => {
        console.log('Processing 48-hour reminder job:', job.id);
        // Reminder logic
      }
    );
  }
}

The trick here?

onModuleInit() fires when the NestJS module boots up — the perfect moment to register your workers before any jobs start flowing.
Concurrency as the second argument — In queue.process(jobName, concurrency, handler), that middle number is your power lever. No decorators, just plain code.
Maximum flexibility — You can read concurrency values from a config service, toggle jobs on/off per environment, or even spin up workers conditionally. Decorators can't do that.

When to use this: When you need runtime flexibility, config-driven behaviour, or you're working in a service-first architecture where decorators feel out of place.

The Secret Ingredient: Concurrency

All three patterns orbit the same concept — concurrency. Let's demystify it.

Concurrency in Bull is simply: "how many workers can chew through this job type at the same time?"

Concurrency	What it means
`1`	One job at a time. Safe, slow, sequential.
`3`	Three jobs running simultaneously. Faster, but uses more resources.
`10`	High throughput. Great for I/O-bound tasks; risky for CPU-heavy ones.

A few rules of thumb:

I/O-bound jobs (sending emails, hitting APIs) → higher concurrency is fine; your workers spend most time waiting, not computing.
CPU-bound jobs (image processing, PDF generation) → keep concurrency low; too many parallel workers will starve your CPU.
Rate-limited external APIs → concurrency should respect the API's limits, not just your server's capacity.

Putting It All Together

Here's a quick cheat sheet for picking your pattern:

Scenario	Best Pattern
Small app, few job types	Single Consumer
Separate teams or complex logic	Multiple Consumers
Config-driven or dynamic setup	`onModuleInit()`

The beauty of Bull is that none of these patterns are mutually exclusive. You can use a single consumer for simple jobs and a dedicated processor for a particularly complex one — mix and match as your needs evolve.

Wrapping Up

Job queues aren't glamorous. They live in the background, quietly doing the work that keeps your users happy — sending that confirmation email before they start to wonder, firing that reminder just in time.

But how you architect your consumers? That's where the craft lives.

Whether you go with a one-chef kitchen, a full brigade, or a ghost kitchen running on vibes and onModuleInit — now you know the trade-offs, and you can choose with confidence.

Go build something that scales. 🚀

How does Javascript work under the hood?

Ronak Navadia — Sat, 06 Jul 2024 12:54:45 +0000

Okay, So today we shall learn about How Javascript works.

Firstly, JavaScript is a lightweight, cross-platform, single-threaded, and interpreted compiled programming language.

So, Everything inside Javascript happens inside a place known as execution context.

Now the question arises in your mind what is execution context?

Execution Context:-

The environment that handles the code transformation and code execution by scanning the script file (JS file).

As shown above, the Execution context is divided into 2 parts.

Memory component (variable environment)
This is a place where all the variables and functions are stored in key-value pairs.
Code component (thread of execution)
This is a place where code is executed one line at a time.

Now lets see how execution context works?

consider the below example

var n = 3;

function cube(num) {
  var result = num * num * num;
  return result;
}

var cube3 = cube(n);
var cube5 = cube(5);

Now execution context is created with memory and code components as shown in the above image

The first phase is the memory creation phase and the second phase is the code execution phase.

Memory creation phase

In this phase, JS will allocate the memory to all the variables and functions

allocate memory to n variable and store value as undefined.
allocate memory to the cube function and store the whole function code.
allocate memory to cube3 variable and store value as undefined.
allocate memory to cube5 variable and store value as undefined.

Now after the memory creation let's see how the code is executed in the code execution phase.

Code execution phase

Javascript goes through the whole JS program line by line and executes the code.

assign value 2 to n

now it goes to the line where the square function is invoked. so whenever any function is invoked a new execution context is created same as mentioned in the image.

so for the square(n) a new execution context is created with memory and code components

here again, memory is allocated to num and as undefined and then it runs the code execution phase for the function in which it assigns num a value i.e. passed from square(n) where n = 3. so num will assign 3. After that ans will be assigned num * num * num i.e. 27.

Now whenever anything is returned from the function its value is assigned to the variable that invoked the function. here in our case, cube3 will have 27 as ans is assigned 27.

Note here that whenever an execution context is done with its task it is deleted from the call stack.

The same thing happens for the 2nd function cube 5.

Thank you so much for being with me.