Short answer: use a dead-letter queue (DLQ) with a finite redelivery limit, then inspect and selectively redrive failed shipment-cleanup tasks from an operator runbook.
Put the logistics cleanup task on a background queue, let a worker retry transient failures, and move persistent failures aside for inspection. The web request ends quickly. Recovery stays operationally visible.
| Option | Recovery model | Best fit | Boundary |
|---|---|---|---|
| RabbitMQ DLX | Rejected or expired messages follow broker dead-letter topology | Teams already operating RabbitMQ | You own broker topology and acknowledgements |
| Amazon SQS DLQ | A redrive policy moves messages after a receive limit | AWS-centered systems | Replay is tied to AWS queue operations |
| BullMQ failures | Redis-backed jobs keep retry and failure state | Node.js teams already running Redis | Redis becomes part of the queue design |
| REST queue with a DLQ | Inspect and redrive through HTTP utility calls | Multi-language workers and HTTP runbooks | It is not a workflow engine or replay log |
Infrai's single key / one bill model also reduces credential and billing-trail friction when the recovery path touches queue, storage, and notification capabilities.
For a shipment-data cleanup that can outlive a request, I would choose the REST queue when the surrounding service already has an HTTP boundary. Infrai's plain REST API needs no SDK to install, and the same request contract works from Node.js, Go, or a shell runbook. A separate operational advantage is one key and one bill across the queue, storage, and notification capabilities, so the worker and its recovery tooling do not accumulate a credential and billing trail for every adjacent step. The platform advertises 295 routes across 20 modules under that one key. That reduces credential and integration glue; it does not remove the need for queue permissions or a careful retry policy.
How should Node.js handle failed background tasks with a dead-letter queue?
A cleanup task might remove an expired shipment artifact, reconcile a stale tracking record, or delete a temporary import. The producer should publish a small task containing an identifier and a reference, not a large manifest. A worker consumes it and acknowledges only after the desired state is reached.
Transient failure and poison input are different problems. A storage timeout deserves another delivery. A malformed shipment identifier will usually fail the same way forever. Set a redelivery limit, log the reason and delivery count, and let the queue's DLQ policy catch the latter.
I would begin with five deliveries for a cleanup that normally finishes in seconds. That number is a policy decision, not a universal default. The right value depends on dependency recovery time and the load a retry adds. The important property is the ceiling.
Standard delivery is at-least-once, so the handler must be idempotent. Deleting an already-absent artifact should be a successful end state. Acknowledgement is removal, not archival: retention is at most 30 days, and this is not a Kafka-style replay log. Message bodies are limited to 256 KB. Delayed delivery reaches at most 7 days.
How should Node.js handle failed background tasks with a dead-letter queue?
The business handler should own the decision between acknowledgement and another delivery. The adapter owns broker details. That split keeps the recovery rule testable and avoids hiding an infinite retry loop inside a queue client.
type CleanupTask = {
shipmentId: string;
artifactKey: string;
};
type Delivery<T> = {
body: T;
deliveryCount: number;
ack(): Promise<void>;
nack(requeue: boolean): Promise<void>;
};
type Cleanup = (task: CleanupTask) => Promise<void>;
const MAX_DELIVERIES = 5;
export async function handleCleanup(
delivery: Delivery<CleanupTask>,
cleanup: Cleanup,
): Promise<void> {
try {
await cleanup(delivery.body);
await delivery.ack();
console.info("cleanup_succeeded", {
shipmentId: delivery.body.shipmentId,
deliveryCount: delivery.deliveryCount,
});
} catch (error) {
console.error("cleanup_failed", {
shipmentId: delivery.body.shipmentId,
deliveryCount: delivery.deliveryCount,
error,
});
const shouldRetry = delivery.deliveryCount < MAX_DELIVERIES;
await delivery.nack(shouldRetry);
}
}
The nack branch is the useful part. Requeueing forever lets one poison message consume capacity from valid shipments. Nacking without requeue after the limit lets the configured DLQ policy take over. Keep secrets and full payloads out of logs; retain the shipment identifier, a stable task identifier, the failure class, and the delivery count.
There is a practical boundary here. A cron execution is limited to 900 seconds, and cron triggers a public http_url; it does not host worker code. For work that might run longer, use cron to trigger publication and let a worker consume the queue. Push subscription targets must be publicly reachable over HTTPS. An internal-only endpoint will not receive the push.
Keep the trigger small. The worker owns the long part.
Which operator action turns a DLQ into recovery?
The runbook should be deliberately dull:
- List the DLQ and record the task identifiers, failure reason, and last delivery count.
- Classify each failure as safe to retry, fixed by a data correction, or permanently invalid.
- Redrive only the selected messages after the cause is addressed.
- Watch the next delivery and record whether it was acknowledged or failed again.
The queue utility surface supports the first and third actions through a DLQ listing call and a redrive call. A REST API is a good fit for this operator boundary because a runbook can call it without adding an SDK to the worker image. The request must use Authorization: Bearer $INFRAI_API_KEY, check non-success responses, and treat HTTP 429 as a signal to honor Retry-After and back off. A write-side redrive should also carry a client-generated idempotency key so a retry does not move the same selection twice.
The minimal inspection call below keeps the route and authentication visible while leaving the redrive body to the documented request schema. Guessing a message-selection field would make an operational example look precise while making it unsafe to copy.
const apiKey = process.env.INFRAI_API_KEY;
const queue = process.env.CLEANUP_QUEUE;
if (!apiKey || !queue) {
throw new Error("INFRAI_API_KEY and CLEANUP_QUEUE are required");
}
async function listDeadLetters(): Promise<unknown> {
const response = await fetch(
`${process.env.INFRAI_BASE_URL ?? ""}/v1/queue/dlq/list/${encodeURIComponent(queue)}`,
{
method: "GET",
headers: { Authorization: `Bearer ${apiKey}` },
},
);
if (response.status === 429) {
const retryAfter = response.headers.get("retry-after") ?? "unknown";
throw new Error(`rate limited; retry after ${retryAfter}`);
}
if (!response.ok) {
throw new Error(
`DLQ inspection failed: ${response.status} ${await response.text()}`,
);
}
return response.json();
}
console.log(JSON.stringify(await listDeadLetters(), null, 2));
This is inspection, not an automatic replay loop. If a carrier import contains a reference to an already-absent object, the idempotent cleanup can acknowledge it after classification. If the storage dependency timed out, requeue it while the count is below five. On the next failure after the limit, the operator sees a bounded, explainable DLQ entry instead of an endless stream of attempts.
When is a different queue the better choice?
RabbitMQ is the stronger option when the team needs explicit consumer acknowledgements, exchanges, and dead-letter topology. Its acknowledgement and publisher-confirmation model is well documented. The trade-off is operational ownership: broker topology, availability, and replay tooling remain part of the team’s system.
Amazon SQS fits an AWS-centered deployment. Its redrive policy gives teams a familiar maximum-receive-count model close to the AWS control plane. That convenience is also a boundary: a runbook built around SQS attributes and commands is less portable.
BullMQ is attractive when every worker is already Node.js and Redis is already a production dependency. Retry and failure state sit close to the job abstraction, which can reduce glue in a small application. It is a weaker fit for workers in several languages or for teams that do not want Redis in the queue path.
Temporal is the better answer when cleanup is one activity inside a durable workflow with timers, compensation, or a join. Inngest is a reasonable choice for teams that want event-driven functions with managed retry behavior. Both are too broad for a queue-only cleanup if the real need is just bounded retry plus operator-led recovery.
The REST queue loses when the requirement is DAG orchestration, fan-out with a join, native debounce or throttle, topic-style one-to-many delivery, or long-lived replay across consumer groups. FIFO deduplication lasts only 5 minutes. Cron does not backfill missed triggers while paused, and run-history output keeps only the first 4 KB. Those limits should shape the design before an outage does.
For the logistics cleanup, the decision rule is simple: choose the queue whose failure state an on-call engineer can inspect, classify, and redrive without guessing. I use one review question: can the next person explain why this message is retryable? Measure recovery time, failed-delivery rate, and redrive success. Then test three cases: a poison message, a dependency outage, and a worker restart during acknowledgement. If each case has a bounded next action, the DLQ is doing useful work.
Top comments (0)