DEV Community

Cover image for Autopsy #02: Anatomy of a Memory Leak on an 8-Hour Dashboard
frontend.rocks
frontend.rocks

Posted on Originally published at frontend.rocks

Autopsy #02: Anatomy of a Memory Leak on an 8-Hour Dashboard

Incident Report

A routine PR added an auto-resizing MutationObserver to the NOC telemetry table widget so column widths adjusted when live alerts streamed in. Every 10-minute QA smoke test passed with flat memory graphs.

Six hours into the Monday shift, dedicated wallboard monitors running the dispatch SPA began crashing with Chrome's Aw, Snap! (Out of Memory) error.

  • T+00:00: Initial JS heap settles at 120 MB after dashboard boot.
  • T+02:00: Operators switch between regional tabs; heap climbs at 280 MB/hour to 680 MB.
  • T+05:30: V8 Mark-Compact GC pauses hit 900 ms per frame; DevTools heap snapshots show 18,400 Detached HTMLDivElement nodes and 28 retained FiberNode trees.
  • T+07:45: Renderer process crosses 2.4 GB and terminates with Aw, Snap!.

The Red Herring

The first Slack reflex was to add telemetryRows = null and host = null inside the useEffect cleanup function. Local testing showed a brief drop in the DevTools memory timeline after manual GC, so the team shipped the hotfix.

Production crashes continued on the exact same 6-to-8-hour cadence. Nulling local bindings inside a cleanup closure does not sever the native C++ retainer edge held by an active MutationObserver or an uncancelled fetch stream.

Causal Chain

The leak compounds across five runtime layers. Uncollected allocations promote into V8 Old Space, native DOM observers act as immortal GC roots, stale hook closures pull entire component trees into those roots, un-disconnected MutationObserver instances persist across route unmounts, and absent AbortController wiring leaves no unified teardown path.

Failure Simulator

Interactive Failure Simulator — Click to step through the live model

Layer Dissection

01 → 02: Surviving Allocations Promote to Immortal Retainers

Under JavaScript Generational Garbage Collection, short-lived objects die in V8's Young Generation during fast Scavenge cycles. Allocations that remain reachable across two scavenges promote into Old Space, where reclamation requires expensive Mark-Compact sweeps.

As shown in Memory Leaks, Retainers, and Detached DOM Trees, a single reachable JS reference to an unmounted <div class="chart-host"> pins the entire detached DOM subtree in Old Space.

02 → 03: Retainer Paths Originate in Stale Hook Closures

The retainer edge rarely comes from a global window.leak = node assignment. Instead, Stale Closures in Hooks & Callbacks capture the lexical environment of the render in which an effect was created:

  • Captured state: A 40 MB telemetryRows array from t = 0h 15m stays bound inside the callback scope.
  • Captured ref: widgetRef.current points to the detached DOM container after the user navigates to another route.

03 → 04: Native MutationObserver Keeps the Closure Reachable

V8 fails to collect the stale closure after unmount because MutationObserver and Batched Microtask Delivery registers the callback inside Blink's C++ DOM mutation agent.

  • Native GC root: Until observer.disconnect() runs, Blink holds a strong edge to the JS callback.
  • Retained graph: Every variable and DOM node inside that callback closure remains reachable across route swaps.

04 → 05: Absent AbortController Prevents Unified Teardown

In the incident codebase, the effect also opened a fetch stream and attached a window resize listener. Without AbortController and Cooperative Cancellation, early returns skipped observer.disconnect() whenever host changed mid-flight, leaving partial subscriptions alive.

Architectural Fix & Invariants

useEffect(() => {
  const host = widgetRef.current;
  if (!host) return;
  const controller = new AbortController();
  const { signal } = controller;

  const observer = new MutationObserver(() => syncLayout(host));
  observer.observe(host, { childList: true, subtree: true });
  signal.addEventListener('abort', () => observer.disconnect(), { once: true });

  window.addEventListener('resize', () => syncLayout(host), { signal });
  fetch('/api/telemetry', { signal })
    .then((r) => r.json())
    .then((rows) => renderChart(host, rows))
    .catch((err) => { if (err.name !== 'AbortError') reportError(err); });

  return () => controller.abort();
}, []);
Enter fullscreen mode Exit fullscreen mode

Architectural Invariants:

  • Single-token teardown: Every useEffect that allocates a DOM observer, window listener, or network stream creates one AbortController and returns () => controller.abort().
  • Paired observer disconnect: Every observer.observe() call registers { once: true } on signal to invoke observer.disconnect() atomically.
  • Thin callback scopes: Observers read only the DOM node passed to their callback or a mutable ref, never large historical state arrays from outer renders.
  • Soak-test verification: Navigation stress tests swap routes 50 times in Playwright and assert Detached node count returns to baseline after gc().

Golden Rule: Nulling variables never collects memory held by native C++ DOM roots; bind every observer, listener, and fetch to one AbortSignal and abort on unmount.

Top comments (0)