This is a really good distinction: synchronization is not scheduling. π
I think thatβs where Promise.all(items.map(...)) becomes deceptive. The code looks nicely controlled because everything is wrapped in one await, but the actual work may already be flooding the downstream service.
I also really like the comparison with fixed-size chunks. A worker pool keeping the available slots busy is much nicer than making 9 fast tasks wait for one slow one. πΈ
Freelance fullstack developer creating fast, modern, and refined applications. I focus on simplicity, performance, and real value for users. I also write tech-related articles here.
Thanks so much, nyaomaru! That single await really does create a false sense of safety. It looks tidy on the page even while all those requests are already firing at once. Glad the chunking comparison resonated too, since eliminating that tail-latency bottleneck is usually where the biggest real-world speedup comes from.
One thing worth adding to the numbers: swapping in Promise.allSettled fixes nothing here. It changes which rejections you see, not how many sockets you open, and one 429 still poisons the other 1,199 writes. The unbounded part is map() itself, so bounding the pool (p-limit, or an index-based worker loop over a fixed array of runners) is the actual fix; allSettled sits on top of that if you want per-invoice results instead of fail-fast.
Second: fetch won't queue for you either. undici's default Agent has no per-origin connection cap, so an app-level limit of ~8 is the only thing between the billing job and the payment gateway β same reason keep-alive alone doesn't save you. And even a bounded pool leaves partial commits when the gateway rejects mid-run, so the durable part is an idempotency key per invoice so a re-run doesn't double-charge. Did you reconcile the 1,199 orphaned writes by hand, or add a resume cursor keyed on the last committed id?
Freelance fullstack developer creating fast, modern, and refined applications. I focus on simplicity, performance, and real value for users. I also write tech-related articles here.
Great points across the board, especially calling out undiciβs default Agent behavior, since a lot of folks assume keep-alive automatically queues excess requests per origin. For that initial incident, I reconciled the completed writes against the gateway's settlement logs and immediately paired the bounded pool with deterministic per-invoice idempotency keys and a persisted sync status so retries safely skip committed records. Appreciate you adding that production depth to the thread.
Small trap in mapWithConcurrency. If limit comes from config and ends up 0 or NaN (Number(process.env.SYNC_CONCURRENCY) when the var isn't set), Array.from makes zero workers, workerFn never runs, and it resolves with 5 empty slots like the job succeeded. I'd throw unless limit is an integer >= 1.
Freelance fullstack developer creating fast, modern, and refined applications. I focus on simplicity, performance, and real value for users. I also write tech-related articles here.
Great catch. That is a nasty silent-failure trap, especially when Number(process.env.SYNC_CONCURRENCY) evaluates to NaN on an unset env var. Since Math.min(NaN, items.length) resolves to NaN, Array.from quietly creates zero workers and returns sparse empty slots as if the whole batch succeeded. Adding an explicit Number.isInteger(limit) && limit >= 1 guard at the top is definitely worth the extra line in production code.
A concurrency limit is one of those fixes that looks boring and prevents a whole class of outages. A small p-limit around the batch usually costs a few lines and saves a rate-limit storm.
Freelance fullstack developer creating fast, modern, and refined applications. I focus on simplicity, performance, and real value for users. I also write tech-related articles here.
Spot on. Those unglamorous 5-line guardrails are usually the difference between a quiet on-call rotation and a 2 AM rate-limit cascade, and p-limit is a great lightweight default for that exact reason. Thanks for reading.
I measure AI coding claims in public. LGTM β Let's Get the Measurements: real runs, real numbers, receipts. Every article is an experiment I actually ran. Books & packs: aheadkit.gumroad.com
The 429 storm gets all the attention, but the orphaned side effects are the part that scared me. Promise.all rejects on the first failure while the other 999 calls keep mutating state in the background with nothing awaiting them. I've hit the same shape with webhook retries, where the first attempt already landed and the retry has no idea. In your worker pool, when the AbortController fires mid-batch, what happens to the result slots for work that never ran? Do they stay empty, and can the caller tell "never ran" apart from "failed"?
Freelance fullstack developer creating fast, modern, and refined applications. I focus on simplicity, performance, and real value for users. I also write tech-related articles here.
Orphaned writes on webhook retries are brutal to untangle after the fact. In the fail-fast mapWithConcurrency snippet, rethrowing inside the worker causes Promise.all(workers) to reject directly, so the results array is never returned and any unstarted slots simply remain uninitialized holes. When I need full per-item visibility after an early abort, I pre-fill results with { status: "skipped" } before starting the workers so the caller can cleanly distinguish tasks that failed from ones that never ran. Thanks for bringing up that edge case.
The worker pool is the right fix for the flood, but for a billing job I would pair it with an idempotency key per invoice. The pool stops the 429 storm, yet a retry after an abort can still double-charge, and only an idempotent sync makes the retry safe.
Freelance fullstack developer creating fast, modern, and refined applications. I focus on simplicity, performance, and real value for users. I also write tech-related articles here.
Completely agree. Bounding concurrency protects the upstream API from getting flooded, while deterministic idempotency keys per invoice are what actually make retries safe after a mid-batch abort without risking double charges.
I research AI tools, APIs & agents β comparing specs, pricing and real user reports β then publish honest rankings so you don't waste money. Research-first, never sponsored fluff.
The worker pool is the part I'd actually copy, but one thing β aborting the controller stops the queue, it doesn't cancel the in-flight calls unless workerFn listens to the signal, so for that billing job you'd still have a handful of orphaned writes, just bounded instead of 1,199. Do you pass the signal into the actual gateway call in practice, or is stopping the queue good enough because the pool is small?
Freelance fullstack developer creating fast, modern, and refined applications. I focus on simplicity, performance, and real value for users. I also write tech-related articles here.
Great catch. In practice, I always pass signal directly into fetch or the HTTP client to release sockets immediately, while keeping in mind that a client-side abort won't roll back a mutating gateway request the server has already received. Bounding the pool shrinks that blast radius to just a handful of records, and pairing it with per-invoice idempotency keys and persisted sync statuses handles those few in-flight writes cleanly on retry. Thanks for highlighting that nuance.
I write about Next.js + TypeScript + AI engineering, with a focus on making sites discoverable to LLMs and AI search. Long-form notes at mudassirkhan.me
the fail fast orphaned side effects bit is the one that burns people most. we had a nightly sync where one rejection left 900 writes still running with nothing awaiting them, and the retry then double wrote half of them.
the worker pool with the abort signal fixes the scheduling, but the abort only stops queued work, not requests already in flight. do you make the writes idempotent as well, or lean on the abort alone?
We're a place where coders share, stay up-to-date and grow their careers.
This is a really good distinction: synchronization is not scheduling. π
I think thatβs where
Promise.all(items.map(...))becomes deceptive. The code looks nicely controlled because everything is wrapped in oneawait, but the actual work may already be flooding the downstream service.I also really like the comparison with fixed-size chunks. A worker pool keeping the available slots busy is much nicer than making 9 fast tasks wait for one slow one. πΈ
Great article, John! π
Thanks so much, nyaomaru! That single
awaitreally does create a false sense of safety. It looks tidy on the page even while all those requests are already firing at once. Glad the chunking comparison resonated too, since eliminating that tail-latency bottleneck is usually where the biggest real-world speedup comes from.One thing worth adding to the numbers: swapping in Promise.allSettled fixes nothing here. It changes which rejections you see, not how many sockets you open, and one 429 still poisons the other 1,199 writes. The unbounded part is map() itself, so bounding the pool (p-limit, or an index-based worker loop over a fixed array of runners) is the actual fix; allSettled sits on top of that if you want per-invoice results instead of fail-fast.
Second: fetch won't queue for you either. undici's default Agent has no per-origin connection cap, so an app-level limit of ~8 is the only thing between the billing job and the payment gateway β same reason keep-alive alone doesn't save you. And even a bounded pool leaves partial commits when the gateway rejects mid-run, so the durable part is an idempotency key per invoice so a re-run doesn't double-charge. Did you reconcile the 1,199 orphaned writes by hand, or add a resume cursor keyed on the last committed id?
Great points across the board, especially calling out
undiciβs defaultAgentbehavior, since a lot of folks assume keep-alive automatically queues excess requests per origin. For that initial incident, I reconciled the completed writes against the gateway's settlement logs and immediately paired the bounded pool with deterministic per-invoice idempotency keys and a persisted sync status so retries safely skip committed records. Appreciate you adding that production depth to the thread.Small trap in mapWithConcurrency. If limit comes from config and ends up 0 or NaN (
Number(process.env.SYNC_CONCURRENCY)when the var isn't set), Array.from makes zero workers, workerFn never runs, and it resolves with 5 empty slots like the job succeeded. I'd throw unless limit is an integer >= 1.Great catch. That is a nasty silent-failure trap, especially when
Number(process.env.SYNC_CONCURRENCY)evaluates toNaNon an unset env var. SinceMath.min(NaN, items.length)resolves toNaN,Array.fromquietly creates zero workers and returns sparse empty slots as if the whole batch succeeded. Adding an explicitNumber.isInteger(limit) && limit >= 1guard at the top is definitely worth the extra line in production code.A concurrency limit is one of those fixes that looks boring and prevents a whole class of outages. A small p-limit around the batch usually costs a few lines and saves a rate-limit storm.
Spot on. Those unglamorous 5-line guardrails are usually the difference between a quiet on-call rotation and a 2 AM rate-limit cascade, and
p-limitis a great lightweight default for that exact reason. Thanks for reading.The 429 storm gets all the attention, but the orphaned side effects are the part that scared me. Promise.all rejects on the first failure while the other 999 calls keep mutating state in the background with nothing awaiting them. I've hit the same shape with webhook retries, where the first attempt already landed and the retry has no idea. In your worker pool, when the AbortController fires mid-batch, what happens to the result slots for work that never ran? Do they stay empty, and can the caller tell "never ran" apart from "failed"?
Orphaned writes on webhook retries are brutal to untangle after the fact. In the fail-fast
mapWithConcurrencysnippet, rethrowing inside the worker causesPromise.all(workers)to reject directly, so theresultsarray is never returned and any unstarted slots simply remain uninitialized holes. When I need full per-item visibility after an early abort, I pre-fillresultswith{ status: "skipped" }before starting the workers so the caller can cleanly distinguish tasks that failed from ones that never ran. Thanks for bringing up that edge case.The worker pool is the right fix for the flood, but for a billing job I would pair it with an idempotency key per invoice. The pool stops the 429 storm, yet a retry after an abort can still double-charge, and only an idempotent sync makes the retry safe.
Completely agree. Bounding concurrency protects the upstream API from getting flooded, while deterministic idempotency keys per invoice are what actually make retries safe after a mid-batch abort without risking double charges.
I think I have done the Promise.all thing on a big list before and learned it the hard way too.
The worker pool is the part I'd actually copy, but one thing β aborting the controller stops the queue, it doesn't cancel the in-flight calls unless workerFn listens to the signal, so for that billing job you'd still have a handful of orphaned writes, just bounded instead of 1,199. Do you pass the signal into the actual gateway call in practice, or is stopping the queue good enough because the pool is small?
Great catch. In practice, I always pass
signaldirectly intofetchor the HTTP client to release sockets immediately, while keeping in mind that a client-side abort won't roll back a mutating gateway request the server has already received. Bounding the pool shrinks that blast radius to just a handful of records, and pairing it with per-invoice idempotency keys and persisted sync statuses handles those few in-flight writes cleanly on retry. Thanks for highlighting that nuance.the fail fast orphaned side effects bit is the one that burns people most. we had a nightly sync where one rejection left 900 writes still running with nothing awaiting them, and the retry then double wrote half of them.
the worker pool with the abort signal fixes the scheduling, but the abort only stops queued work, not requests already in flight. do you make the writes idempotent as well, or lean on the abort alone?