DEV Community

Cover image for Running headless Chrome in production is a part-time job
Stevie g
Stevie g

Posted on • Originally published at shotpipe.io

Running headless Chrome in production is a part-time job

It works on your laptop. You wrote fifteen lines of Puppeteer, pointed it at a
URL, got a PNG back. You wrapped it in a route, deployed it, and for about a day
it was the easiest feature you ever shipped. Then the container got OOM-killed
at 2am, the queue backed up behind it, and you spent the next week learning that
"just screenshot the page" is a systems problem wearing a fifteen-line disguise.

Here is the list of what breaks, in roughly the order it breaks, and the process
shape that stops it. It's the same list whether you're on Puppeteer or
Playwright, and whether you're rendering screenshots, PDFs, or OG images — it's
Chrome's lifecycle that's hard, not the API in front of it.

1. Memory: the first wall, and the loudest

A single headless Chrome rendering one page sits in the low hundreds of
megabytes of RSS — call it 150–300 MB depending on the page. That's per render,
and it's the good case. Chrome also leaks: keep one instance alive long enough
and RSS climbs and doesn't come back, because a browser was built to be closed
by a human at the end of the day, not kept resident for a month. On a box with a
fixed memory limit the ending is always the same — the kernel's OOM killer picks
the fattest process and kills it mid-render, and your logs show a bare SIGKILL
with no stack trace.

If you've searched puppeteer out of memory and found forty issues with no
accepted answer, this is why: there isn't a line to fix, there's a lifecycle to
manage. Two things follow. Cap how many renders share one instance and
recycle it — close the whole browser and launch a fresh one every N pages,
so leaked memory is reclaimed by process death, not by hope. And size
concurrency to RAM, not CPU: if one render is 250 MB, eight concurrent
renders is 2 GB before you've counted the OS, and "eight" is a small number.

2. Zombies: the processes that don't die

browser.close() is supposed to clean up, and usually does. But Chrome isn't
one process — it's a tree: a main process, a zygote, a renderer per page, plus
GPU and utility helpers. When Chrome crashes, or your Node process is killed
while a browser is open, or a navigation wedges the renderer, close() never
runs or never finishes, and you're left with <defunct> chrome processes
reparented to init. They hold memory and file descriptors. Do that a few
thousand times and you exhaust PIDs or FDs, and the box stops accepting work for
reasons that have nothing to do with your code.

The fix is unglamorous: reap the tree yourself. Track the browser's PID and, on
any abnormal exit, kill the whole process group instead of trusting the
library's close(). In a container, run a real init that reaps orphans
(--init, tini, or dumb-init) so PID 1 isn't your app pretending to be an init
system it isn't.

3. Cold starts: the 800ms tax

Launching Chrome costs roughly 800 milliseconds before it renders a single pixel
— process spawn, sandbox setup, the first blank page. Launch a fresh browser per
request and you pay that tax every request, where it dwarfs the actual render for
anything simple. So the instinct is to launch one browser and reuse it forever —
which walks you straight back into problem #1.

The resolution is the distinction most tutorials skip: reuse the instance,
isolate per context. A browser context is a clean, cookieless, cacheless
session inside an already-running Chrome — cheap to create, cheap to destroy,
isolated from every other render.

// not this — 800ms of startup tax on every request
const browser = await puppeteer.launch()
const page = await browser.newPage()

// this — warm instance, throwaway context per render
const context = await browser.createBrowserContext()  // cheap, isolated
const page = await context.newPage()
// …render…
await context.close()  // nothing leaks into the next render
Enter fullscreen mode Exit fullscreen mode

Keep a small pool of warm instances, give each render its own fresh context,
throw the context away after, and recycle the whole instance every N renders
(problem #1). You pay the 800ms once per instance lifetime, not once per request,
and renders don't bleed into each other.

4. Pages that fight back

Your renderer navigates to a URL and waits. Sometimes the wait never ends: an
infinite redirect, a page that never fires load, a websocket that keeps the
network "busy" forever, a while(true) in someone's analytics. Without a hard
ceiling, one bad URL parks a browser until it's killed — and if you're reusing
instances, one hostile page can wedge a slot in your pool for good.

Put hard timeouts on both navigation and capture, and when one trips, kill the
instance and launch a new one — don't try to nurse it back.
A browser that
hung once is not a browser you can reason about; it's carrying whatever state
caused the hang. The correct response to a sick Chrome is a fresh Chrome. It
feels wasteful and it's the single most reliability-improving rule in the whole
system.

5. Concurrency is a queue problem, not a loop

The naive version renders inline: request arrives, you launch or borrow a
browser, render, respond. Under load this is exactly how you die — a traffic
spike becomes N simultaneous browsers becomes an OOM kill becomes every in-flight
render failing at once. Rendering is expensive and bursty, which is the precise
profile a queue exists for.

Put a queue between the request and the render. The API accepts the job and
returns immediately; a pool of workers pulls jobs at a rate their RAM can
survive. A spike becomes queue depth — a number you can watch and autoscale
on — instead of a memory graph that falls off a cliff. Queue depth is your
capacity early-warning signal; RSS-at-the-OOM-line is the alternative, and it
warns you by paging you.

6. The font stack nobody mentions

Your laptop has fonts. A minimal Linux container does not. So the page that
looked right locally renders with tofu boxes where the CJK text was, blank
rectangles where the emoji were, and the wrong fallback for everything else — and
you find out from a customer's screenshot, not a test. Real page rendering means
you now own a font pipeline: a base font set, CJK coverage, an emoji font, and
the standing knowledge that a Chrome upgrade can shift glyph rendering and
quietly change your output out from under a cache.

7. Serverless doesn't make this go away

The reflex is "put it on Lambda and let someone else scale it." That moves the
problems, it doesn't remove them. You ship a special slimmed Chromium build to
fit the unzipped size limit; you eat a multi-second cold start on every
scale-up, because a warm pool is exactly what serverless won't give you; you cap
out at the function's memory and time limits, which is where playwright lambda
timeout
comes from; and you still own the fonts. It's a legitimate deployment
target, but it's a different set of sharp edges, not fewer of them.

The shape that survives

Put it together and the design that works isn't exotic — it's this list, applied
consistently:

![Architecture diagram: bursty traffic enters a queue; a pool of stateless, disposable workers each run a warm Chrome instance with one context per render, recycled every N renders and killed on hang; results land in an edge cache served in milliseconds.]

  • A queue between the request and the render — capacity is a worker count, not a prayer.
  • Stateless, disposable workers, each holding a small pool of warm Chrome instances.
  • A fresh context per render, discarded after.
  • Recycle each instance every N renders, so leaks die with the process.
  • Hard timeouts on navigation and capture; kill-and-respawn on any hang — never nurse a sick browser.
  • Concurrency sized to RAM, with queue depth as the capacity signal.
  • A maintained font and emoji stack, plus a cache version you can bump when Chrome moves.

That's the whole trick. It's also, to be blunt, most of what our workers do —
because there isn't a cleverer answer, only this list, monitored.

Disclosure before the turn: we run this as a service
(Shotpipe), so read the next two sections as the author
pointing at their own tool. The list above is true whether you build it or buy
it — I'm claiming the list is the work, not that you need us to do it.

The part the memory threads leave out

Every puppeteer out of memory thread is about pages you control — your own
dashboard, your own invoice, your own marketing page. The moment the URL comes
from your users — a link preview, an unfurl, a "screenshot my site" button —
you've added a second problem that has nothing to do with memory: the URL
might point back at you.
Someone submits
http://169.254.169.254/latest/meta-data/ and your obliging headless browser
reads your cloud credentials and hands them back as a PNG. That's SSRF, and a
browser is a near-perfect engine for it, because it follows redirects and
resolves DNS for you — the two exact places the attack hides.

Fixing it properly means resolving DNS yourself, checking every resolved IP
against private, loopback, link-local, and cloud-metadata ranges, pinning the IP
you validated and connecting to that one, and re-checking on every redirect
hop. It's a module with its own test suite, not an if-statement — and it's the
part no memory-leak tutorial mentions, because those authors are rendering their
own pages. We wrote it up in screenshotting URLs you don't
control
: if your renderer ever touches a
URL a stranger typed, read that before you ship.

When to run it yourself anyway

Honestly: if you render a handful of pages you control, on a schedule,
self-hosting is fine. Launch Chrome, render, close, move on — none of the above
bites at that volume, and you shouldn't pay anyone to avoid a problem you don't
have. The list starts mattering when renders get frequent, bursty, or pointed at
URLs you don't own. That's the crossover where "just screenshot the page" stops
being fifteen lines and becomes a service — ours, or the one you'll end up
building.

Top comments (0)