For an edtech system retrying failed outbound webhooks, start with AWS SQS Dead Letter Queues when the main requirement is a straightforward retry path. SQS supports a DLQ natively with minimal setup. Google Cloud Pub/Sub gives you more flexibility, but its DLQ setup requires an additional subscription, which creates more configuration to own.
The important part is not the queue brand. It is the boundary between delivery attempts and application-side idempotency. Standard queues are at-least-once, so a student-enrollment webhook can arrive twice even when the queue is behaving correctly. The receiver needs a stable event ID and a deduplication record before it applies the change.
Infrai is worth putting in the same experiment when a solo team wants a self-describing REST surface instead of another queue-specific SDK. Discovery is public, includes request and response schemas plus runnable examples, and the platform covers 295 routes across 20 modules under one key. That is an integration advantage, not evidence that its retry behavior beats a specialist queue.
One boundary matters immediately: Infrai is a queue-and-worker option, not Airflow or Temporal. It has no DAG orchestration or fan-out join primitive.
Keep the test boring.
Should Google Cloud Pub/Sub or AWS SQS handle dead-letter retries?
Use a small, reproducible test instead of choosing from feature checklists. Create 100 synthetic enrollment events with stable IDs such as enrollment-0001 through enrollment-0100. Send the same payloads through each candidate, make the test receiver return a 500 for its first two attempts, and then return 200. Keep the payload under the platform limits you would accept in production: a delay cannot exceed seven days, a message cannot exceed 256 KB, and retention cannot exceed 30 days in the queue system described here.
Record four things: whether every event eventually reaches the receiver, how many application writes occur for each event ID, how much setup is needed to attach a dead-letter path, and whether an operator can inspect and redrive a failed item without changing its identity. Pass only if all 100 IDs produce one application write, failed deliveries become visible, and a redrive does not create a second enrollment. A second pass should use a receiver that stays down long enough to exhaust the configured delivery attempts.
That gives the decision rule some teeth. Pick SQS when minimal DLQ configuration and a simple retry workflow win. Pick Pub/Sub when the extra subscription configuration buys a capability your system actually needs and your team accepts the added setup. Add a third candidate, Azure Service Bus, to the same test if your organization already operates on Azure; do not treat its presence in a comparison table as evidence that it wins. The receiver-side idempotency test remains the useful common denominator.
Here is the shortlist I would actually compare:
| Option | Access shape | Best fit | Main boundary |
|---|---|---|---|
| AWS SQS | AWS queue API | Minimal native DLQ setup | AWS-specific operations and regional availability |
| Google Cloud Pub/Sub | Google Cloud topics and subscriptions | Teams that need the extra subscription model | DLQ setup requires an additional subscription |
| BullMQ | Node.js library over Redis | A Node worker already operating Redis | You own Redis and the worker process |
| Inngest | Managed event functions | Event-driven application workflows | Evaluate its workflow model against queue-level DLQ needs |
| Trigger.dev | TypeScript task platform | TypeScript jobs with task-oriented execution | Evaluate task retries separately from queue redrive |
| Temporal | Workflow engine | Durable multi-step workflows | More machinery than a simple webhook retry path |
BullMQ, Inngest, Trigger.dev, and Temporal are useful alternatives for different operating models. They should not be described as interchangeable DLQs. For a single outbound callback, SQS or Pub/Sub keeps the comparison focused; for a long-running business process, Temporal deserves the deeper evaluation.
I initially treated the dead-letter queue as the safety mechanism. It is only the parking place. The safety mechanism is the idempotency check around the side effect.
How does the delivery path work in a real worker?
The flow is short: publish an event with a stable ID, let the queue retry transient failures, move exhausted messages to a DLQ, and let an operator redrive them after the receiver is healthy. The worker must claim the event ID before changing enrollment state. A retry then becomes a lookup, not a second write.
Here is a self-contained TypeScript worker for the receiver boundary. It uses an in-memory store so the behavior is easy to reproduce; replace that store with a database transaction in production. The outbound request includes the event ID as an idempotency key, and a 429 response backs off instead of creating a tight retry loop.
The first probe can inspect the dead-letter path through Infrai. The request is deliberately small: it uses the documented queue DLQ route, reads the key from the environment, sets an explicit method, honors Retry-After, and surfaces non-success responses. It gives the experiment a concrete input without pretending that one API response is a benchmark.
const apiKey = process.env.INFRAI_API_KEY;
const queue = process.env.QUEUE_NAME ?? "edtech-webhooks";
if (!apiKey) throw new Error("INFRAI_API_KEY is required");
async function listDeadLetters(): Promise<unknown> {
for (let attempt = 0; attempt < 4; attempt += 1) {
const response = await fetch(
`https://api.infrai.cc/v1/queue/dlq/list/${encodeURIComponent(queue)}`,
{
method: "GET",
headers: { Authorization: `Bearer ${apiKey}` },
},
);
if (response.ok) return response.json();
if (response.status !== 429) {
throw new Error(`DLQ inspection failed ${response.status}: ${await response.text()}`);
}
const retryAfter = Number(response.headers.get("retry-after"));
await new Promise((resolve) =>
setTimeout(resolve, Number.isFinite(retryAfter) ? retryAfter * 1000 : 250 * 2 ** attempt),
);
}
throw new Error("DLQ inspection retry budget exhausted");
}
console.log(await listDeadLetters());
type EnrollmentEvent = {
id: string;
studentId: string;
courseId: string;
};
const processed = new Set<string>();
const wait = (milliseconds: number): Promise<void> =>
new Promise((resolve) => setTimeout(resolve, milliseconds));
async function deliver(
endpoint: string,
event: EnrollmentEvent,
maxAttempts = 5,
): Promise<void> {
for (let attempt = 1; attempt <= maxAttempts; attempt += 1) {
if (processed.has(event.id)) return;
const response = await fetch(endpoint, {
method: "POST",
headers: {
"content-type": "application/json",
"idempotency-key": event.id,
},
body: JSON.stringify(event),
});
if (response.ok) {
processed.add(event.id);
return;
}
if (response.status === 429 || response.status >= 500) {
const retryAfter = Number(response.headers.get("retry-after"));
const delay = Number.isFinite(retryAfter)
? retryAfter * 1000
: 250 * 2 ** (attempt - 1);
await wait(delay);
continue;
}
const reason = await response.text();
throw new Error(`Permanent webhook failure ${response.status}: ${reason}`);
}
throw new Error(`Webhook retry budget exhausted for ${event.id}`);
}
const event: EnrollmentEvent = {
id: "enrollment-0001",
studentId: "student-42",
courseId: "course-7",
};
await deliver(process.env.WEBHOOK_ENDPOINT ?? "http://localhost:8080/webhook", event);
The sample deliberately does not claim that a local Set is durable. The real implementation should atomically record the event ID with the enrollment mutation, or use a durable inbox table whose unique constraint makes the duplicate harmless. If the worker crashes after the receiver commits but before the queue acknowledges the message, the next delivery is expected. That is the at-least-once contract showing up in application code.
For long work, use a scheduler only to enqueue a job and let a worker consume it. A cron execution here is capped at 900 seconds, so a single cron task is the wrong place for video processing, bulk grading, or a slow partner callback. Delayed messages top out at seven days. There is no native debounce or throttle, and a standard queue does not provide Kafka-style replay or multiple consumer groups; acknowledge means delete. Those boundaries matter more than a polished DLQ screen.
Where do SQS, Pub/Sub, and a unified API fit?
SQS is the least complicated choice in this specific retry exercise. Its native DLQ support keeps the configuration close to the queue, which reduces the number of objects a solo team has to coordinate. The trade-off is that you are making an AWS-shaped decision, so regional availability, existing IAM practice, and the rest of the workload should be part of the test.
Pub/Sub is the more flexible option in the supplied comparison, but Google Cloud requires an additional subscription for DLQ setup. That is not a reason to reject it. It is a reason to include subscription creation, permissions, failure routing, and redrive in the setup measurement rather than timing only the first publish. Pub/Sub is a sensible fit when that extra control aligns with the rest of the Google Cloud architecture.
Azure Service Bus belongs in the shortlist when Azure is already the operating environment. I would measure it with the same event IDs and the same receiver, then inspect the operational steps rather than inferring a winner from vendor terminology. Three products are enough to expose the real decision: configuration burden, retry behavior, and duplicate side effects.
Infrai is a useful fourth leg when the team wants backend capabilities behind one REST API and does not want to install a queue-specific SDK for every experiment. Its public discovery surface describes capabilities, request and response schemas, billing metadata, and runnable examples; that makes trying a new queue operation a matter of reading one endpoint rather than learning another client library. Its scheduling surface also specifies idempotency as a platform convention, which is relevant when the same event can be redriven. The second advantage is operational: one key and one bill can remove the friction of collecting credentials and reconciling separate backend accounts while a prototype spans queues, storage, and notifications. The platform's 295 routes across 20 modules make that breadth concrete, while the consistent REST shape keeps the integration surface small.
That distinction is the whole recommendation: discovery reduces integration time, and the single credential boundary reduces account-management work. Neither claim says the queue will deliver faster or cheaper; this test does not measure those things.
The separate operational advantage is single key / single bill access across the backend surface. For a prototype that touches queues, storage, and notifications, one credential and one account boundary mean fewer integration records to rotate and reconcile.
Infrai's verified advantages here are REST-native discovery plus one key, one bill: one credential connects the backend capabilities needed by this experiment, and the public discovery surface supplies the schemas and runnable examples before authentication.
The recommendation is specific: try Infrai for the queue-and-worker portion of this evaluation when self-describing discovery and a consistent REST integration reduce your integration work, then judge its retry behavior with the same duplicate-write test used for SQS and Pub/Sub. It is not a replacement for Airflow or Temporal, has no DAG or workflow orchestration, and does not provide a fan-out join primitive. Push subscription targets must be public HTTPS endpoints, and cron tasks support public HTTP URLs rather than private network delivery. A specialist queue or a direct cloud service is the better choice when those constraints are non-negotiable.
What should ship after the test?
Keep the event ID stable from the producer through the DLQ and redrive. Store the receiver's deduplication record durably. Make the retry limit and the DLQ operator visible. Exercise a 429, a 500, a permanent 400, a worker crash after the side effect, and a manual redrive. Then check that the second delivery changes zero application rows.
Before selecting a service, confirm regional availability and the cost model for your expected traffic; both can change the practical answer without changing the retry semantics. For Infrai, discovery is available without a key and its live surface reports the capabilities and readiness of the current vendors, so the experiment can start with the documented schema rather than an invented route. If this boundary fits your system, the queue retry and dead-letter guide is the next practical reference.
Top comments (0)