DEV Community

Samcorp
Samcorp

Posted on

Debugging Production Memory Leaks in a Node Monolith

Debugging Production Memory Leaks in a Node Monolith

The graph looked harmless at first.

09:00  → 420 MB
12:00  → 610 MB
15:00  → 790 MB
18:00  → 1.1 GB
Enter fullscreen mode Exit fullscreen mode

Traffic dropped.

Memory didn't.

Eventually:

FATAL ERROR:
JavaScript heap out of memory
Enter fullscreen mode Exit fullscreen mode

When a large Node.js monolith starts behaving like this, the tempting fix is:

node --max-old-space-size=8192 app.js
Enter fullscreen mode Exit fullscreen mode

Sometimes that buys time.

It doesn't tell us why memory keeps growing.

A useful Node memory leak investigation starts by identifying which kind of memory is growing, then finding what continues retaining it.


First: Confirm It's Actually a Leak

Not every growing memory graph means leaked JavaScript objects.

Start with:

const memory = process.memoryUsage();

console.table({
  rss: memory.rss,
  heapTotal: memory.heapTotal,
  heapUsed: memory.heapUsed,
  external: memory.external,
  arrayBuffers: memory.arrayBuffers
});
Enter fullscreen mode Exit fullscreen mode

These numbers describe different things.

heapUsed
→ JavaScript objects currently using V8 heap

heapTotal
→ Heap currently allocated by V8

external
→ Memory associated with native/C++ objects

arrayBuffers
→ ArrayBuffer/SharedArrayBuffer memory,
  including Node Buffers

rss
→ Overall resident process memory
Enter fullscreen mode Exit fullscreen mode

That distinction is important.

If:

heapUsed ↑ continuously
Enter fullscreen mode Exit fullscreen mode

we may have retained JavaScript objects.

If:

RSS ↑
heapUsed ≈ stable
Enter fullscreen mode Exit fullscreen mode

we should also investigate Buffers, native modules, allocator behavior, or other memory outside the normal V8 heap.

Don't diagnose from one metric.


Look at the Shape of Memory

Healthy applications allocate memory constantly.

Garbage collection should recover much of it.

Conceptually:

Memory
  │      /\      /\      /\
  │     /  \    /  \    /  \
  │____/    \__/    \__/    \__
  └─────────────────────────────
Enter fullscreen mode Exit fullscreen mode

The interesting pattern is:

Memory
  │             /\
  │          __/  \
  │       __/
  │    __/
  │___/
  └─────────────────────────────
Enter fullscreen mode Exit fullscreen mode

The baseline keeps climbing after garbage collection.

That's when I start asking:

What is still referencing objects that should already be dead?


In a Monolith, Start With Long-Lived Containers

A large Node monolith usually has plenty of objects designed to live for the entire process lifetime.

That makes them excellent leak candidates.

Look at:

Global Maps
Caches
Module-level arrays
Event emitters
Queues
Session stores
Timers
Request registries
WebSocket connections
Enter fullscreen mode Exit fullscreen mode

Consider this innocent-looking cache:

const userCache = new Map();

async function getUser(id) {
  if (!userCache.has(id)) {
    userCache.set(id, await loadUser(id));
  }

  return userCache.get(id);
}
Enter fullscreen mode Exit fullscreen mode

What's the eviction policy?

There isn't one.

If new IDs continue arriving:

10,000 users
   ↓
50,000 users
   ↓
500,000 users
   ↓
Map keeps everything
Enter fullscreen mode Exit fullscreen mode

Technically the objects are still reachable.

So garbage collection is behaving correctly.

The application is the thing leaking.


Watch Event Listeners

Another common source of retention is listeners that are added repeatedly but never removed.

function handleRequest(req) {
  eventBus.on("user-updated", () => {
    updateSomething(req.user);
  });
}
Enter fullscreen mode Exit fullscreen mode

Now imagine that happens for every request.

The listener retains its closure.

The closure may retain:

Request
User
Metadata
Other objects
Enter fullscreen mode Exit fullscreen mode

Instead, lifecycle-sensitive listeners need lifecycle-sensitive cleanup.

function handler(data) {
  // process event
}

eventBus.on("user-updated", handler);

// Later
eventBus.off("user-updated", handler);
Enter fullscreen mode Exit fullscreen mode

The real question isn't just:

“How many listeners exist?”

It's:

“Why is this listener still alive?”


Timers Can Quietly Retain Large Object Graphs

Consider:

function startJob(customer) {
  setInterval(() => {
    refreshCustomer(customer);
  }, 60_000);
}
Enter fullscreen mode Exit fullscreen mode

That callback retains customer.

If jobs are repeatedly created without clearing old intervals:

Timer
  ↓
Closure
  ↓
Customer
  ↓
Associated objects
Enter fullscreen mode Exit fullscreen mode

remain reachable.

Always know who owns a timer and when it ends.

const timer = setInterval(runJob, 60_000);

// lifecycle ends
clearInterval(timer);
Enter fullscreen mode Exit fullscreen mode

This becomes particularly important in monoliths because dozens of unrelated modules may create their own background jobs.


Take Heap Snapshots — Carefully

Guessing eventually stops being useful.

A better investigation is:

Snapshot A
    ↓
Generate representative traffic
    ↓
Wait / reproduce growth
    ↓
Snapshot B
    ↓
Compare
Enter fullscreen mode Exit fullscreen mode

We're looking for objects whose retained population or retained size keeps increasing.

Typical suspects might include:

Array
Map
Object
String
Closure
Buffer wrappers
Application-specific classes
Enter fullscreen mode Exit fullscreen mode

Then inspect the retaining path.

For example:

Global Cache
   ↓
Map
   ↓
UserSession
   ↓
Orders[]
   ↓
Large response objects
Enter fullscreen mode Exit fullscreen mode

Now we have something actionable.

Heap snapshots can temporarily consume significant CPU and memory, so capturing them blindly on a heavily loaded production instance is risky. Use a safe diagnostic strategy appropriate to your infrastructure.

For long-lived Node applications, this kind of monitoring and troubleshooting belongs alongside ordinary Node.js application development rather than being treated only as emergency work after an out-of-memory crash.


Don't Ignore Buffers

Suppose:

heapUsed = relatively stable
RSS      = climbing
Enter fullscreen mode Exit fullscreen mode

The JavaScript heap may not be the primary problem.

Node applications handling:

File uploads
Image processing
Streams
Compression
Large API payloads
Sockets
Binary protocols
Enter fullscreen mode Exit fullscreen mode

often work heavily with Buffer.

That's why I monitor:

const {
  rss,
  heapUsed,
  external,
  arrayBuffers
} = process.memoryUsage();
Enter fullscreen mode Exit fullscreen mode

If external or arrayBuffers grows alongside RSS, the investigation changes direction.

This is one reason simply increasing the V8 heap limit can completely miss the actual problem.


Reproduce One Workload at a Time

A monolith might handle:

REST API
Background jobs
Uploads
Reports
WebSockets
Emails
Scheduled tasks
Integrations
Enter fullscreen mode Exit fullscreen mode

If everything runs simultaneously, identifying the leak becomes difficult.

Isolate workloads.

Baseline
   ↓
Run API traffic
   ↓
Measure

Baseline
   ↓
Run report generation
   ↓
Measure

Baseline
   ↓
Run background jobs
   ↓
Measure
Enter fullscreen mode Exit fullscreen mode

You might discover:

Normal API traffic
→ stable

WebSocket traffic
→ stable

Report generation
→ memory never returns
Enter fullscreen mode Exit fullscreen mode

Now the search space is dramatically smaller.


Add Memory Observability Before the Next Incident

At minimum, track:

RSS
Heap used
Heap total
External memory
ArrayBuffer memory
Process restarts
OOM events
Request volume
Job volume
Enter fullscreen mode Exit fullscreen mode

Then correlate memory growth with application behavior.

Memory spike
     +
Report jobs increased
     +
Large Buffers increased
Enter fullscreen mode Exit fullscreen mode

is much more useful than:

Server uses 1.8 GB.
Enter fullscreen mode Exit fullscreen mode

Production performance should be treated as part of ongoing web application maintenance and optimization, especially when a monolith has accumulated years of APIs, integrations, jobs, and shared application state.


The Debugging Order I Prefer

When investigating a Node memory leak, I use roughly this sequence:

1. Confirm sustained growth
        ↓
2. Compare heapUsed vs RSS
        ↓
3. Check external / Buffer memory
        ↓
4. Correlate growth with workload
        ↓
5. Reproduce the suspicious path
        ↓
6. Capture safe heap snapshots
        ↓
7. Compare retained objects
        ↓
8. Follow retaining paths
        ↓
9. Fix ownership / cleanup
        ↓
10. Repeat the workload
Enter fullscreen mode Exit fullscreen mode

Then ask one final question:

Does memory return to a stable baseline?
Enter fullscreen mode Exit fullscreen mode

If not, keep investigating.


Common Places I'd Check First

For a production Node monolith:

□ Unbounded Map/Set caches
□ Global arrays
□ Event listeners
□ setInterval/setTimeout
□ Request objects retained by closures
□ WebSocket connection state
□ Unfinished promises
□ Large Buffers
□ File-processing pipelines
□ In-memory sessions
□ Background queues
□ ORM/database result retention
□ Application metrics with unbounded labels
Enter fullscreen mode Exit fullscreen mode

Notice that most leaks aren't mysterious V8 bugs.

They're usually object ownership problems.

Something was created.

Something should have released it.

Something didn't.


Final Takeaway

The worst way to debug a Node memory problem is:

Memory high
   ↓
Increase heap
   ↓
Restart server
   ↓
Wait
   ↓
Memory high again
Enter fullscreen mode Exit fullscreen mode

A better approach is:

MEASURE
   ↓
CLASSIFY
   ↓
REPRODUCE
   ↓
SNAPSHOT
   ↓
FIND RETAINER
   ↓
FIX
   ↓
MEASURE AGAIN
Enter fullscreen mode Exit fullscreen mode

The key lesson is that a Node memory leak isn't simply:

“Node is using too much RAM.”

The useful question is:

“Which memory is growing, and what is keeping it alive?”

Once you can answer that, debugging stops being guesswork.

And in a large monolith, reducing the search space is often half the fix.

Top comments (0)