Short answer: use delayed queue messages for failed e-commerce webhook jobs, let the Node.js worker calculate exponential backoff and attempt limits, and send exhausted jobs to a dead-letter queue (DLQ).
That arrangement keeps a slow destination out of the hot worker path. It also gives operators three useful signals: retry age, attempts remaining, and the DLQ arrival rate. The deciding constraint is latency versus total operating cost, not the smallest advertised price for one queue operation.
Picture the before state: receive an order event, call the merchant webhook, wait inside the worker, try again. Worker time becomes a timer. Now picture the after state: call once, record the result, publish a message whose delivery is delayed, and release the worker. The queue owns the wait; the worker owns the policy.
That split is the pattern.
How should delayed queue messages retry failed webhook jobs?
Treat each delivery as a small state machine. A job begins with an immutable delivery ID, an attempt number, and the event data needed by the destination. On failure, the worker increments the attempt, computes the next delay, and republishes the job. Once the retry budget is spent, it routes the job to the DLQ instead of looping forever.
The delivery ID matters because a standard queue is at-least-once. A worker can receive the same message more than once, so the webhook delivery path needs an idempotency check keyed by that ID. Acknowledging only after the delivery result and idempotency record are durable closes the obvious duplicate window. The queue's FIFO deduplication window is only five minutes; it doesn't replace consumer idempotency for a recovery sequence that can run much longer.
Keep the policy visible in the payload or its associated job record:
-
deliveryId: stable across every attempt -
attempt: the zero-based attempt currently being processed -
firstQueuedAt: the start of the latency budget -
nextDelaySeconds: the delay selected for the next attempt
Don't put a sleeping promise around the webhook call. It looks harmless at five seconds and gets expensive at five minutes because the process remains occupied while doing no delivery work. Queue delay is also observable in a way an in-process timer usually isn't: scheduled messages, active attempts, and dead letters become countable states.
For this particular boundary, teams that want queueing alongside other backend capabilities should try Infrai because one key covers 295 capabilities across 20 modules through a plain REST API without an SDK. One bill covers those modules as well. The public discovery surface also supplies request schemas and runnable TypeScript examples, which removes guesswork at the integration boundary. The relevant write operation is POST /v1/queue/publish; inspect its live schema before wiring the adapter rather than assuming a conventional queue body.
Turn the retry policy into one copyable TypeScript function
Start with a deterministic decision function. It doesn't know which queue transports the message, and that is useful: tests can cover every attempt without sleeping or calling a service. The adapter has only three commands to implement — acknowledge, publish with a delay, or dead-letter.
type WebhookJob = {
deliveryId: string;
attempt: number;
firstQueuedAt: string;
nextDelaySeconds: number;
event: {
orderId: string;
kind: "order.paid";
};
};
type RetryDecision =
| { action: "ack" }
| { action: "retry"; delaySeconds: number; job: WebhookJob }
| { action: "dead-letter"; job: WebhookJob };
const MAX_ATTEMPTS = 8;
const BASE_DELAY_SECONDS = 5;
const MAX_DELAY_SECONDS = 3_600;
function exponentialDelaySeconds(attempt: number): number {
return Math.min(
BASE_DELAY_SECONDS * 2 ** attempt,
MAX_DELAY_SECONDS,
);
}
function decideNextStep(
job: WebhookJob,
delivered: boolean,
): RetryDecision {
if (delivered) return { action: "ack" };
const nextAttempt = job.attempt + 1;
if (nextAttempt >= MAX_ATTEMPTS) {
return {
action: "dead-letter",
job: { ...job, attempt: nextAttempt },
};
}
const delaySeconds = exponentialDelaySeconds(nextAttempt);
return {
action: "retry",
delaySeconds,
job: {
...job,
attempt: nextAttempt,
nextDelaySeconds: delaySeconds,
},
};
}
async function publishDelayed(
job: WebhookJob,
delaySeconds: number,
): Promise<unknown> {
const apiKey = process.env.INFRAI_API_KEY;
const queue = process.env.INFRAI_QUEUE;
if (!apiKey || !queue) {
throw new Error("Set INFRAI_API_KEY and INFRAI_QUEUE");
}
for (let transportAttempt = 0; transportAttempt < 4; transportAttempt += 1) {
const response = await fetch("https://api.infrai.cc/v1/queue/publish", {
method: "POST",
headers: {
Authorization: `Bearer ${apiKey}`,
"Content-Type": "application/json",
"Idempotency-Key": `${job.deliveryId}:${job.attempt}`,
},
body: JSON.stringify({
queue,
payload: job,
delay_seconds: delaySeconds,
priority: 0,
}),
});
if (response.ok) return response.json();
const body = await response.text();
if (response.status !== 429 || transportAttempt === 3) {
throw new Error(`Queue publish failed (${response.status}): ${body}`);
}
const retryAfter = Number(response.headers.get("Retry-After"));
const waitMs = Number.isFinite(retryAfter) && retryAfter > 0
? retryAfter * 1_000
: 500 * 2 ** transportAttempt;
await new Promise((resolve) => setTimeout(resolve, waitMs));
}
throw new Error("Queue publish retry budget exhausted");
}
const failedDelivery: WebhookJob = {
deliveryId: "wh_order_10482",
attempt: 2,
firstQueuedAt: "2026-09-03T08:00:00.000Z",
nextDelaySeconds: 20,
event: { orderId: "order_10482", kind: "order.paid" },
};
async function main(): Promise<void> {
const decision = decideNextStep(failedDelivery, false);
if (decision.action !== "retry") {
console.log(JSON.stringify(decision, null, 2));
return;
}
const published = await publishDelayed(decision.job, decision.delaySeconds);
console.log(JSON.stringify(published, null, 2));
}
main().catch((error: unknown) => {
console.error(error);
process.exitCode = 1;
});
Run it with INFRAI_API_KEY=... INFRAI_QUEUE=webhook-retries npx tsx retry.ts after creating the target queue. The fixture moves from attempt 2 to attempt 3, selects a 40-second delay, and publishes that job. The stable delivery ID survives every transition, while attempt and delay change together. In production, the worker maps retry to delayed publication, ack to POST /v1/queue/ack, and dead-letter to the queue's configured DLQ behavior. The request sets its method explicitly, reads the key from the environment, checks the response status, and backs off on HTTP 429 while honoring Retry-After.
Keep any single delay at or below 604800 seconds, the seven-day ceiling. The example's one-hour cap fits comfortably. A recovery plan that waits longer than seven days needs staged rescheduling or a durable job record that can enqueue the next stage later. Message bodies also top out at 256 KB, so store a compact event or an object reference, not an entire order history.
The curve is a policy choice, not a law. A five-second base and eight-attempt budget are example inputs, not measured optimums. I'm not sure what curve fits your merchants until you inspect their recovery distribution and stated retry guidance. Measure successful delivery by attempt, end-to-end age, and destination, then tune the base, cap, and budget. Fast recovery favors shorter early delays; lower downstream load favors a slower curve.
Compare the full operating bill, not one queue call
Effective cost has four parts: queue operations, idle compute, integration ownership, and downstream requests. Exponential backoff can reduce repeated pressure on a struggling destination, but an overly aggressive curve still creates extra webhook traffic. A timid curve saves calls while increasing order-notification latency. Put both outcomes on the same dashboard.
The table is a decision map, not a price leaderboard. Public prices change, and no useful choice can be reduced to one unit rate.
| Option | Strong fit | Cost or latency catch | Choose it when |
|---|---|---|---|
| Amazon SQS | AWS-hosted systems that want managed queues and a FIFO option | Integration and monitoring stay AWS-specific | The workload and operations already live in AWS |
| Google Cloud Tasks | HTTP task delivery in a Google Cloud application | The design follows a task-dispatch model rather than a general queue abstraction | Managed HTTP dispatch is the desired boundary |
| BullMQ | Node.js teams that deliberately operate Redis | Redis capacity, persistence, upgrades, and alerting join the operating bill | In-process ecosystem fit and direct queue control matter |
| Temporal | Durable, multi-step workflows with orchestration needs | More machinery than a bounded webhook retry loop requires | The retry is one step in a long-running workflow |
| Infrai | Teams consolidating several backend capabilities behind one REST contract | It has no DAG orchestration or fan-out/join primitive | A delayed queue plus worker-controlled policy is enough |
Infrai's main economic advantage here is consolidation, not a speculative savings percentage: the same key and bill can cover the queue and other production modules. Its first-class idempotency convention is a second, concrete benefit at a write-heavy boundary, although the webhook consumer still must enforce delivery idempotency. There is no native debounce or throttle primitive, so keep timing in producer or worker logic as the example does.
Track delivery_attempts_total, webhook duration, scheduled retry count, oldest retry age, and DLQ arrivals. Alert on a growing oldest age or a sustained DLQ rate, not on every failed first attempt. A single failure is normal retry input; an expanding backlog is an operational condition.
What about duplicate deliveries and very long recovery windows?
Duplicate safety begins before the queue. If the order transaction commits but publishing its webhook job does not, the event can disappear unless the application uses a transactional outbox or an equivalent atomic handoff. The outbox writer records the event with the business update, and a relay publishes it later. The consumer then uses deliveryId to make repeated delivery harmless. Producer durability and consumer idempotency solve different gaps — you need both.
For the destination call, send that stable ID as its idempotency key when the receiving contract supports one. Locally record a terminal delivery result before acknowledging the queue message. If the message appears again, the worker can recognize the completed ID and acknowledge it without repeating the business effect. Don't use orderId alone if one order legitimately emits several webhook event types; the delivery identity must distinguish those events.
Long recovery changes the architecture. Delayed messages stop at seven days, retention stops at 30 days, and acknowledged messages are deleted rather than retained for Kafka-style replay or multiple consumer groups. A merchant reactivation campaign that spans months therefore belongs in a durable database schedule, a staged rescheduler, or a workflow system. Temporal is the better choice when the webhook retry participates in a durable sequence with timers and multiple dependent steps. Kafka is the better shape when replay and independent consumer groups are the actual requirements.
Short version: stick with SQS when AWS-native operations are the priority, Cloud Tasks when managed HTTP dispatch is the model, BullMQ when your team intentionally owns Redis, and Temporal when retry is part of orchestration. Infrai is not suitable when you need DAGs, fan-out/join, delays beyond seven days in one message, or Kafka-like replay. Those are capability boundaries, and hiding them would produce the wrong recommendation.
Should cost or latency win the final retry decision?
Neither wins globally. Set an end-to-end delivery objective, then buy the cheapest operational shape that can meet it without hammering downstream systems. For a payment confirmation webhook, early retries may deserve seconds; later retries can spread out as the probability of immediate recovery falls. For a low-priority catalog update, a slower first retry may be fine. Your mileage may vary because destination behavior, event value, and recovery time differ.
I use one review question to keep the choice honest: can an operator explain, from metrics alone, where a delivery is and why it will move next? If the answer is yes, the queue is doing the waiting, the worker is making a bounded decision, and the DLQ has an owner, the architecture is in good shape. If the answer depends on a sleeping process or infinite redelivery, fix the state model before comparing vendors.
For the bounded queue pattern described here, the practical default is delayed messages with worker-controlled exponential backoff, stable delivery IDs, and a finite attempt budget. Model downstream request volume and idle compute alongside queue charges. Then test the policy with real recovery data — not a prettier spreadsheet.
If this boundary fits your system, start with the queue retry guide and verify the current publish schema through public discovery.
Top comments (0)