Short answer: use cron for small, reconstructable job processing; use a durable queue when rate-limited API work must survive retries, bursts, and reconciliation. The deciding constraint is not the timer. It is the evidence needed after a process restart or an ambiguous external result.
A per-minute allowance turns scheduling into accounting. The system needs a durable statement of which work was accepted, which attempt consumed a dispatch slot, and which effects remain unresolved. A clock can wake a process, but it cannot by itself establish that a particular item was claimed once, retried deliberately, or reconciled before it is sent again.
How should a Node.js job processing queue enforce a per-minute API rate limit?
Put the rate decision immediately before the outbound request and make its scope explicit. If an upstream API permits L requests in a minute for one credential, route, or tenant, every worker sharing that scope must draw from the same budget. A local counter in each process is valid only after the allowance has been partitioned among those processes; otherwise two healthy workers can both conclude that capacity remains and jointly exceed the limit.
The useful unit of work is a state transition, rather than a timer callback. A durable record might move from ready to leased, then to succeeded or quarantined, while an append-only attempt record captures the lease owner, dispatch time, response classification, and retry decision. The lease expires because a worker can stop after claiming work. Completion must reject a stale lease owner. Those two rules make recovery explainable during a reconciliation review.
HTTP 429 Too Many Requests is a scheduling signal, not proof that the job is malformed. MDN specifies that a 429 response may include Retry-After; when present, the dispatcher should honor that value and record the resulting next-eligible time. A malformed input, an authorization decision, and an ambiguous timeout require different policies. Treating every failure as a generic exponential retry spends quota while concealing the cause.
Short queues often fail at the minute boundary. Consider a backlog of 600 items and an allowance of 60 dispatches per minute: a worker that reloads at 12:00:59 and another that starts at 12:01:00 must agree on the same window definition, or a harmless deployment change becomes an unexpected burst. A serialized dispatcher, a transactional lease over a counter row, or another shared atomic budget can enforce that agreement. The choice depends on the durability and contention guarantees of the storage already operated by the team, but the invariant stays constant: no request begins until a recorded budget decision permits it.
This is the narrow part. The recovery model is larger.
Stop there.
Derive cron or queued delivery from the loss model
Cron is a wake-up mechanism. It works well when a run can scan an authoritative table by a stable watermark, claim a bounded batch, and safely repeat the scan after interruption. For example, a nightly reconciler can re-derive candidate rows from settlement status, provided its claims and effects are idempotent. The database remains the work ledger; cron merely asks it what is due.
Queued delivery represents a different promise: an accepted item remains visible until a terminal disposition is recorded. That is the better fit for an ingestion burst, a user-facing deadline, independently retryable work, or a request that must be accounted for after the producer returns. Neither mechanism creates exactly-once external effects. Exactly-once language is defensible only around a concrete local transaction; beyond that boundary, the design needs a stable idempotency key where the receiving API supports one and a reconciliation path for outcomes that cannot be known after a timeout.
The catch is that a queue is not suitable as a substitute for the system of record. Keep the authoritative business state separate from delivery state, especially for payments, ledger entries, or compliance-sensitive changes. A queue can preserve a message while the underlying business decision has been superseded. Conversely, cron is not suitable when polling delay, overlapping scans, or backlog buffering violates the service objective. The correct choice follows the tolerated loss, duplication, and delay, not the fewest initial components.
The distinction becomes concrete at the boundary between a local transaction and an external effect. Suppose an application accepts an instruction, stores its business record, and creates a delivery record. If those writes are in one transactional database, their relationship can be audited together. A worker later leases the delivery record, writes an attempt entry, and calls the recipient with the stable idempotency key. If the connection ends after the recipient accepted the request but before the worker records success, the local system has no right to infer either success or failure from its own timeout. It has an unresolved outcome. The next action must follow the recipient's documented idempotency and status-query contract: query for the prior effect where that is supported, record the reconciliation result, then either complete the delivery or schedule a replay. Don't turn an unknown into a retry merely because the queue has redelivery. This sequence costs more design effort than a callback loop, yet it gives an auditor a coherent answer to a simple question: which attempt caused this external effect, and what evidence established that conclusion? For financial instructions, that answer is often more valuable than the worker's throughput graph.
| Mechanism | Appropriate constraint | Boundary to design explicitly | Evidence to retain |
|---|---|---|---|
| Cron plus authoritative storage | Rebuildable, bounded work with tolerable polling delay | Watermark, overlap policy, and claim transaction | Run ID, query range, claimed IDs, completion count |
| Durable job queue | Accepted work must survive bursts and independent retries | Lease expiry, redelivery, and terminal ownership | Job ID, enqueue receipt, attempt history, disposition |
| Shared rate dispatcher | Many workers consume one external allowance | Tenant, credential, route, and time-window scope | Budget key, decision time, dispatch count, retry time |
Make the dispatcher idempotent and observable
The worker should persist its intent before it performs an external side effect, then persist the observed outcome under the same job identity. When the downstream contract supports an idempotency key, send the job ID or a separately generated stable key. When it does not, mark an interrupted call as ambiguous and reconcile it through the downstream system before replaying. A retry without this distinction can duplicate an irreversible effect.
The following Go sketch separates leasing from dispatch and records the budget decision before Send. Its interfaces deliberately leave storage and transport open, because the important contract is atomic lease ownership and auditable state, not a particular library.
package dispatch
import (
"context"
"time"
)
type Job struct {
ID string
Payload []byte
}
type Store interface {
LeaseReady(ctx context.Context, owner string, until time.Time) (Job, error)
ReserveDispatch(ctx context.Context, scope string, at time.Time) (bool, error)
RecordAttempt(ctx context.Context, jobID, owner string, at time.Time) error
Complete(ctx context.Context, jobID, owner string, at time.Time) error
RetryAt(ctx context.Context, jobID, owner string, readyAt time.Time) error
}
type Sender interface {
Send(ctx context.Context, idempotencyKey string, payload []byte) (retryAt *time.Time, err error)
}
func DispatchOne(ctx context.Context, store Store, sender Sender, owner, scope string, now time.Time) error {
allowed, err := store.ReserveDispatch(ctx, scope, now)
if err != nil || !allowed {
return err
}
job, err := store.LeaseReady(ctx, owner, now.Add(2*time.Minute))
if err != nil {
return err
}
if err := store.RecordAttempt(ctx, job.ID, owner, now); err != nil {
return err
}
retryAt, err := sender.Send(ctx, job.ID, job.Payload)
if err != nil || retryAt != nil {
when := now.Add(time.Minute)
if retryAt != nil {
when = *retryAt
}
return store.RetryAt(ctx, job.ID, owner, when)
}
return store.Complete(ctx, job.ID, owner, time.Now())
}
In production, reserve capacity only after there is confirmed ready work, or make an unused reservation release safely; the implementation also needs a documented policy for storage failures before and after the request. Track oldest-ready age, active leases, terminal dispositions, attempts by error class, and the difference between local success records and the downstream acknowledgment or settlement feed. Raw queue depth is useful, but age and reconciliation difference reveal whether a deadline or financial control is actually being missed.
Treat dead-letter handling as a controlled exception
A dead-letter destination is a quarantine record, not a deletion policy. AWS describes dead-letter queues as a way to isolate messages that fail processing repeatedly, and documents that moving messages there can affect ordering. For an ordered sequence whose meaning depends on predecessor state, automatic isolation needs an explicit review path that protects the required ordering and identifies which business state must be repaired before replay.
Define the maximum receive or attempt count, the owner of review, the retention period, and the evidence required to replay. This should include the original payload reference, the job identity, every prior attempt, the error classification, and the currently authoritative business state. A dead-letter item without a named disposition owner is deferred loss.
There is no universal retry interval. The documented API contract, the scope of the quota, and the business deadline decide it. Where evidence is incomplete, verify the recipient's idempotency and status-query semantics before allowing automated replay; assumptions about either are a compliance risk in systems that move money or regulated records.
Roll out the scheduling policy in small partitions
Begin by measuring the existing workload: arrivals per minute, oldest-ready age, completion latency, retry distribution, and the number of outcomes requiring reconciliation. Run the new dispatcher against copied metadata or a non-effecting partition, then compare its budget ledger with observed outbound requests at minute boundaries. Enable one small partition with idempotency protections, reconcile accepted, attempted, succeeded, and quarantined counts, and increase scope only after the numbers agree.
Rollback should stop new leases while preserving accepted work and the attempt trail. Keep cron for recovery scans and reconciliation where it fits; use queued delivery where the system must retain an individual promise of later work. This division keeps a timer from carrying obligations that belong to durable state.
Top comments (0)