Picture a ticket shop. A customer buys two tickets, and the receipt that comes back shows someone else's name and someone else's total. No exception, no alarm, all unit tests green. The Lambda function behind it has been running without a hitch for years. The only thing that changed: it now runs on Lambda Managed Instances (LMI).
That scenario is the reason I wrote Lambda multi-concurrency: safe Node.js handlers on LMI. Here is the short version of why it happens and what to do about it.
One environment, many invocations
Classic Lambda gave us a comfortable guarantee: one execution environment handles exactly one invocation at a time. A global variable was effectively per invocation, it just happened to survive into the next one. Many of us built habits on that guarantee without noticing.
On LMI, an environment runs on an EC2 instance in your account and handles many invocations in parallel. For Node.js, the runtime starts worker threads (by default one per vCPU), and inside each thread several invocations interleave via async/await. By default that is up to 64 concurrent invocations per vCPU, capped by PerExecutionEnvironmentMaxConcurrency.
The mental model that helped me most has three levels:
-
Per invocation: everything you create inside the handler, plus
eventandcontext. - Per worker thread: the module scope, so global variables, caches, clients and connections.
-
Per environment:
/tmpand the process itself, shared across all workers.
JavaScript inside a worker is still single-threaded, but two invocations can swap at every await. Invocation A writes its order into a module variable and waits for the database. B runs and overwrites the variable. A resumes with B's data. That's the wrong receipt.
The habits that turn into traps
The patterns I keep finding in Lambda code are harmless on classic Lambda and risky on LMI: request data such as the current user in module scope, lazy initialization like if (!prices) prices = await load(), a single database connection shared by everything, fixed file names in /tmp, and process-wide helpers such as _X_AMZN_TRACE_ID or process.on('SIGTERM'), which don't work there at all.
The fixes are what anyone running Express or Fastify already knows: keep request data local or in AsyncLocalStorage, use a connection pool (and remember its size applies per worker), give temp files unique names and clean them up. For lazy initialization, memoize the promise rather than the result. Here is a simplified version of the idea:
export function once<T>(init: () => Promise<T>): () => Promise<T> {
let pending: Promise<T> | undefined;
return () => {
if (!pending) {
// forget a failed attempt so the next call can retry
pending = init().catch((err) => { pending = undefined; throw err; });
}
return pending;
};
}
Ten simultaneous invocations now share one load instead of triggering ten.
Finding and testing it without EC2
Hunting for these spots by hand doesn't scale, so I wrote a small script on top of the TypeScript compiler API that lists mutable module variables, writes to them from inside functions, fixed /tmp paths and the unsupported process helpers. It is a checklist, not a verdict: not every hit is a bug, and it can't see state hidden inside libraries.
The part I like best: you can reproduce the bug locally. A worker thread on LMI behaves like a normal Node.js process handling several requests, so a test that fires 50 invocations with Promise.all and checks that every response matches its own order is enough. The unsafe handler usually returns dozens of mixed-up responses, the fixed one none.
My take: the LMI model is sound. What worries me is old code moved over "just like that" because it's cheaper. I would always start such a migration with the scan and a concurrency test.
What the full article covers
- The broken handler and its safe counterpart side by side, with
AsyncLocalStoragerequest context - The complete check script built on the TypeScript compiler API
- The local concurrency test plus a Serverless Framework setup to fire 100 parallel invocations at the real function
- A ten-point checklist before migrating, including minimum versions for the AWS SDK and Powertools
All the details and code are here: Lambda multi-concurrency: safe Node.js handlers on LMI
๐ This is the condensed version. The full article on sascha-ms.de goes deeper โ with all code samples, diagrams and a short knowledge check.
๐ Next step: Curious what your own Lambda functions cost? Try my Lambda cost calculator โ it runs right in your browser.
๐ฌ Stay updated: New articles on serverless, AI and leading tech teams via RSS, browser push or the weekly newsletter (in German).
Originally published on sascha-ms.de. More articles on serverless, AI in everyday development and leading tech teams: sascha-ms.de/en
Top comments (0)