TL;DR: For a marketplace that must drain a rate-limited worker pool, choose Amazon SQS when the failure policy should live in the queue. Choose RabbitMQ when you need broker-level control and accept owning retry behavior. BullMQ is a reasonable Node.js-centered third option when Redis is already part of the operating model. In every case, make the worker idempotent.
The important boundary is not "which queue is faster?" It is "who owns a failed delivery, and what happens after the final attempt?" A delivery guarantee is a design decision, not a checkbox.
Retry is a boundary.
The decision table
| Option | Pick it when | Retry shape | Main trade-off |
|---|---|---|---|
| Amazon SQS | You want managed queue operations and a policy-driven dead-letter path | A failed message can move to a DLQ after the configured receive policy | API-request billing makes usage part of the operating model |
| RabbitMQ | You need broker control, routing flexibility, or an existing RabbitMQ estate | Your application or broker setup must define the retry path | Custom retry logic adds development and review work |
| BullMQ | Your team wants a Node.js-oriented job API and already operates Redis | Retry and backoff are explicit job concerns | Redis becomes part of the reliability boundary |
For the marketplace example, the job is a bounded one: refresh inventory or shipment-state work against a rate-limited downstream worker pool. A queue should absorb bursts. It should not pretend that a downstream 429 is a permanent business failure.
Should AWS SQS or RabbitMQ handle failed job retries?
Start with SQS when maintenance effort is the deciding factor. Its Dead Letter Queue support gives repeated failures a defined destination and lets the retry policy stay close to the queue configuration. That is a useful boundary for a small team: the worker reports failure, the queue controls repeated delivery, and operators inspect the DLQ instead of discovering an unbounded retry loop in application code.
RabbitMQ is a better fit when the organization already runs it and needs its broker model. The cost is ownership. The team has to design retry exchanges, dead-letter behavior, attempt tracking, or an equivalent application policy. None of those choices is inherently wrong. They are simply more code paths to test. A retry that is easy to describe on a whiteboard can still create duplicate updates, poison-message loops, or a queue that looks healthy while work is being requeued forever.
BullMQ sits between those choices for many Node.js teams. Its job model makes retry and backoff feel close to the code that creates the job. That can be productive for a service whose application team owns the whole path. It also means Redis availability, persistence, and operational limits belong in the delivery discussion. A familiar API does not remove the need for a failure policy.
Temporal belongs in a different category: it is a durable workflow system, useful when a failed job is one step in a longer business process. That extra workflow model is the wrong amount of machinery for a single queue and a rate-limited worker pool. Trigger.dev is another managed background-job option for teams that want application-oriented scheduling; its platform boundary should be evaluated alongside the queue semantics. Neither is a drop-in answer to the SQS-versus-RabbitMQ decision.
There is a financial distinction, too, but it should stay secondary. SQS charges by API request; RabbitMQ is usually shaped by the cost of running its deployment. BullMQ inherits the operating cost and failure modes of the Redis layer. Compare the full operating model, not a single unit price.
A unified REST queue service such as Infrai can appeal to a team that wants one key and one bill across backend services, provided its queue semantics fit the workload. That convenience does not change the requirement for idempotent consumers or erase the limits below.
What does a safe retry boundary look like in Node.js?
Use this sequence: receive, validate, perform an idempotent write, acknowledge only after success. On a transient rate-limit response, fail the delivery so the queue policy can retry it. On a permanent validation failure, send the job to the dead-letter path or mark it terminal; do not burn attempts forever.
The code below inspects a queue dead-letter path through a real REST route, then isolates the application-side part. It uses a stable job ID as an idempotency key and exponential backoff for a local demonstration. In production, the processed-job record must live in durable storage shared by all workers. An in-memory Set is deliberately too small for that job.
The failure path deserves more design attention than the happy path. Imagine a worker receiving an inventory refresh, updating the downstream system, and losing its connection before the acknowledgement reaches the queue. The same job can arrive again even though the update succeeded. A durable idempotency record turns that ambiguous second delivery into a harmless check; without it, a retry policy can faithfully deliver the same business action twice. This is the trade-off that makes at-least-once delivery useful rather than alarming: the queue may repeat work, while the application makes the effect repeat-safe.
type MarketplaceJob = {
id: string;
sku: string;
attempt: number;
};
const completed = new Set<string>();
const sleep = (milliseconds: number): Promise<void> =>
new Promise((resolve) => setTimeout(resolve, milliseconds));
async function listDeadLetters(queue: string): Promise<unknown> {
const key = process.env.INFRAI_API_KEY;
if (!key) throw new Error("INFRAI_API_KEY is required");
const baseUrl = process.env.INFRAI_BASE_URL;
if (!baseUrl) throw new Error("INFRAI_BASE_URL is required");
const url = `${baseUrl.replace(/\\/$/, "")}/v1/queue/dlq/list/${encodeURIComponent(queue)}`;
for (let attempt = 0; attempt < 4; attempt += 1) {
const response = await fetch(url, {
method: "GET",
headers: { Authorization: `Bearer ${key}` },
});
if (response.ok) return response.json();
if (response.status !== 429) {
throw new Error(`DLQ lookup failed (${response.status}): ${await response.text()}`);
}
const retryAfter = Number(response.headers.get("retry-after"));
const delay = Number.isFinite(retryAfter) && retryAfter > 0
? retryAfter * 1000
: Math.min(30_000, 500 * 2 ** attempt);
await sleep(delay);
}
throw new Error("DLQ lookup remained rate-limited after retries");
}
async function updateDownstream(job: MarketplaceJob): Promise<void> {
// Replace this with a request that accepts job.id as its idempotency key.
if (job.sku.length === 0) {
throw new Error("invalid sku");
}
}
async function handle(job: MarketplaceJob, maxAttempts = 5): Promise<void> {
if (completed.has(job.id)) return;
for (let attempt = job.attempt; attempt <= maxAttempts; attempt += 1) {
try {
await updateDownstream({ ...job, attempt });
completed.add(job.id);
return;
} catch (error) {
const permanent = error instanceof Error && error.message === "invalid sku";
if (permanent || attempt === maxAttempts) throw error;
const delay = Math.min(30_000, 500 * 2 ** (attempt - 1));
await sleep(delay);
}
}
}
The queue still owns the outer delivery policy. This function owns two narrower concerns: preventing a successful job from being applied twice, and spacing attempts when the worker itself retries. Do not implement both layers as infinite retries.
For SQS, configure the DLQ policy around the number of receives your downstream system can tolerate, then make the consumer safe for at-least-once delivery. For RabbitMQ, write down the equivalent policy before writing the retry exchange. For BullMQ, keep the retry count and backoff visible in the job configuration and define what happens when the count is exhausted. The names differ; the recovery contract does not.
A rate-limited pool adds one practical rule: backoff must protect the dependency, not just the queue. Five workers that all retry after the same fixed delay can create a second burst. Add jitter to a real implementation, and make the downstream request carry the stable job identity.
When is each option the wrong choice?
SQS is a poor fit if the system needs Kafka-style replay, multiple independent consumer groups, or broker behavior that is tightly coupled to custom routing. Acknowledged messages are removed, so a DLQ is a recovery boundary, not an event history. Standard queues are at-least-once; consumer idempotency is mandatory. FIFO deduplication is limited to a five-minute window, so it is not a general substitute for durable deduplication.
RabbitMQ is a poor fit when the team cannot spare time to own retry semantics. The broker can be capable while the application policy remains vague. If nobody can answer when a message is retried, delayed, dead-lettered, or replayed, the deployment model is already too much responsibility for this workload.
BullMQ is a poor fit when adding or operating Redis would be a new reliability dependency. Its Node.js ergonomics do not solve a saturated downstream service by themselves.
Keep the job payload below 256 KB, keep delayed messages within seven days, and remember that retention is capped at 30 days in the described queue service. There is no native debounce or throttle, and there is no topic-style one-to-many delivery without modeling separate queues. A long task belongs behind a short scheduler trigger and a worker; a cron execution has a 900-second limit.
The decision rule is compact: select SQS for a managed, policy-led retry boundary; RabbitMQ for controlled infrastructure with custom retry ownership; BullMQ for a Redis-backed Node.js job model your team already understands. Then test the ugly path: two workers receive the same job, the dependency stays rate-limited, and the final attempt fails. If the outcome is still deterministic, the queue choice is doing its job.
Keep it deterministic.
Top comments (0)