A shared upstream quota, rather than the size of a batch, decides this design. Short answer: use a durable queue with one active worker for a single shared rate limit, publish batches as individually recorded jobs, and retry each logical operation with a stable idempotency key. Node.js can enforce worker concurrency 1 inside one process; it cannot make that promise across a deployment by itself.
This is an architecture decision record, not a library recipe. The useful invariants are durable intent, one scoped rate gate, and a completion record that survives the awkward interval between a downstream side effect and a queue acknowledgment. Lose any one of them and a restart turns an ordinary 429 response or timeout into either duplicated work or missing work.
Decision record: invariants, scope, and failure boundaries
The first invariant is durable intent. Once an API accepts a batch, every accepted item needs a durable row or broker record before a worker can claim it. A batch identifier is useful for tracing, but it does not replace a per-item key: an item may be accepted, already present, or rejected independently of its neighbors. Record that outcome explicitly, along with payload version, attempt count, next eligible time, lease expiry, and a correlation ID. A process-local array gives none of those facts after a deployment.
The second invariant is that “concurrency 1” must name its scope. One awaited loop limits one runtime to one in-flight operation. Three replicas each running that loop make three calls. If the remote quota is global, use one active consumer, a shared rate gate, or a deployment topology that allocates the quota deliberately. If quotas are per tenant or credential, partition by that boundary and give each partition its own gate. Don't quietly assume that a local setting describes a fleet.
The third is completion evidence. A queue acknowledgment says the consumer received a message; an HTTP success says the endpoint accepted a request under its contract. Neither statement automatically proves a business effect in another data store. For an externally visible write, keep a receiver-side uniqueness constraint, a durable completion receipt, or a reconciliation process that can compare accepted intent with observable state. This is especially important around object and data layers, where a successful transport metric can look healthy while the record needed by a later reader is absent.
The uncomfortable interval is short but real: the remote system may apply the effect just before the worker loses its lease or exits, leaving the next delivery unable to tell whether it should repeat the call. Idempotency makes that ambiguity tractable. The receiver must recognize the same key as the same logical request, and the worker must use that key on every attempt. A local done flag written before the call loses work after a crash; written only after the call, it still cannot stop a duplicate remote effect when the response is lost. The domain boundary has to participate.
| Queue shape | Appropriate boundary | Operational cost | Reason to reject it |
|---|---|---|---|
| One durable FIFO lane | A single global quota and required ordering | Head-of-line blocking | Independent tenants wait behind a slow item |
| Partitioned ordered lanes | Tenant, account, or credential is the quota boundary | Hot partitions need attention | Key assignment must remain stable |
| Competing consumers | Work is independent and redelivery is acceptable | Ordering weakens as consumers increase | Shared rate accounting becomes separate work |
| Durable workflow state | Multi-step jobs need timers or compensation | More state and deployment machinery | A short, serial operation does not need it |
Ordering has a scope.
The queue is the clock.
How should a Node.js rate-limited job processing queue publish batches with worker concurrency 1 and idempotent retry?
Treat publishing and processing as different transactions. The publisher validates each input, derives or accepts a deterministic idempotency key, and stores the accepted jobs durably. It returns a result per item rather than pretending the whole batch is atomic when one entry is malformed. The worker then leases one eligible job, waits until its scheduled retry time and rate slot, calls the dependency, records confirmed completion, and only then acknowledges or deletes the queue record.
HTTP 429 means the server is asking the client to reduce its request rate. MDN documents that a response may include Retry-After; respect it when present rather than layering a guessed delay on top. Other retryable transport outcomes can use capped exponential backoff plus jitter, while invalid input belongs in a terminal state or a dead-letter path. Repeating a request that cannot become valid consumes the exact capacity the queue is meant to protect.
The following Python reference keeps the control flow visible. It models the order a Node.js async worker should preserve: one leased item, one rate slot, one durable idempotency key. The in-memory structures are deliberately a teaching device, not production storage; a real adapter needs atomic persistence and lease recovery.
import asyncio
import random
import time
from dataclasses import dataclass, field
class RateLimited(Exception):
def __init__(self, retry_after_seconds: float) -> None:
self.retry_after_seconds = retry_after_seconds
@dataclass(order=True)
class Job:
ready_at: float
key: str = field(compare=False)
payload: dict = field(compare=False)
attempts: int = field(default=0, compare=False)
async def process_one(job: Job, next_slot: float) -> tuple[Job | None, float]:
await asyncio.sleep(max(0.0, job.ready_at - time.monotonic()))
await asyncio.sleep(max(0.0, next_slot - time.monotonic()))
job.attempts += 1
try:
await apply_idempotent_effect(job.key, job.payload)
return None, time.monotonic() + 0.2
except RateLimited as error:
job.ready_at = time.monotonic() + error.retry_after_seconds
except (TimeoutError, ConnectionError):
if job.attempts >= 5:
await mark_terminal(job.key)
return None, time.monotonic() + 0.2
delay = min(30.0, 2 ** (job.attempts - 1))
job.ready_at = time.monotonic() + delay + random.uniform(0, 0.25)
return job, time.monotonic() + 0.2
apply_idempotent_effect represents the receiver-side operation: it uses job.key to return the existing completion or apply the effect once. mark_terminal must be durable too, because a terminal outcome is an operational fact, not a log line. The 0.2 seconds in the example is only a placeholder for a documented quota; don't deploy a number copied from an example. If the dependency returns Retry-After, that signal outranks the local interval for the next attempt.
Batch dispatch needs its own failure test. Consider a publisher that writes jobs 1 through 99 and exits before job 100: callers need a durable batch status that says which entries are present, not an optimistic “accepted” response for all 100. A useful test fixture begins with a preexisting job for one key, submits that key again beside new entries, interrupts the transaction after a chosen write, and then asks the publisher for the batch result after recovery. The expected answer must distinguish the preserved duplicate from the records that never committed; a retrying client should not have to reconstruct that distinction from logs. Run the same fixture with the queue table temporarily unavailable to workers, so publication and consumption cannot accidentally conceal one another during the test.
Conversely, a worker may complete an effect and exit before it persists its own acknowledgment. Redelivery is correct in that case, provided the receiver recognizes the key. The test should assert both sides: the queue item becomes eligible again after its lease, and the receiver returns the original completion instead of creating a second domain effect. This is where a shallow fake often lies. A fake that merely records a Boolean done cannot model the request arriving twice with the same key, a response being lost after the first call, and a later reconciliation reading the durable receipt. These tests should run with a simulated process stop at each state transition, because happy-path tests don't exercise the boundary that matters.
Rejected design: parallel promises and process-local schedules
Reject Promise.all(batch.map(process_job)) when all items draw from one rate limit. It translates batch size directly into instantaneous concurrency, and it makes a burst of retries likely to arrive together. A local setTimeout is also the wrong source of truth for retry eligibility: it disappears on restart and it does not coordinate replicas. The queue record, with its next eligible timestamp, is the schedule.
The catch is throughput. One worker intentionally creates head-of-line blocking, so it is not suitable when independent customers have separate limits or when a single slow operation should not delay unrelated work. Partitioned lanes are the better fit when the quota boundary is demonstrably independent; use a stable partition key and maintain a separate rate budget for each lane. Stick with one lane when ordering and the downstream limit are truly shared. I'm not sure a partition is safe until the upstream service documents that independence, and your mileage may vary where quotas change by endpoint.
There is no universal “exactly once” switch here. FIFO ordering and broker deduplication can narrow delivery behavior within their documented scope, but a remote database mutation, email, or object write still crosses a separate failure boundary. AWS describes FIFO queues as ordered and designed to avoid duplicate messages in the queue; that is valuable, but it does not certify a side effect beyond the queue. Design for redelivery, then make the effect idempotent.
Operationally, measure oldest eligible-job age, time waiting for a rate slot, lease expirations, attempts by outcome class, terminal counts, and reconciled completions. Alert on growing age rather than process uptime. A running worker can be making no useful progress.
Deployment checks that keep the decision honest
Before production, prove the scope of the quota with a load test that includes the expected replica count. Exercise a restart after durable publish, during a lease, after the remote call, and before acknowledgment. Verify that a repeated batch returns stable per-item answers and that the same idempotency key reaches the receiving operation on every attempt. Then test a 429 with Retry-After, a timeout with unknown completion, invalid input, and a lease that expires while a worker is unavailable.
The decision remains modest: a single durable lane is a good default for one shared dependency limit, not a universal queue architecture. Add partitions only when the ordering and quota boundaries support them. Keep the durable state inspectable. That is what makes a delayed job debuggable instead of merely invisible.
Top comments (0)