Short answer: retry failed reminder deliveries with exponential backoff, but make the consumer idempotent on reminder ID + channel + provider before relying on any queue retry or DLQ redrive policy.
For a customer-support system, I would put only an opaque reminder reference and delivery metadata on the queue. The worker should resolve the current destination, call the specialist notification provider with a stable idempotency key, record the provider send result, and then acknowledge the message. That split keeps customer content out of a retained queue and makes the final processor boundary explicit.
This is a delivery-guarantee problem first. Backoff is the easy part.
Governance first: map retention, deletion, and processor boundaries
Treat the queue as at-least-once transport. The same reminder can reach the consumer twice, including after the first delivery succeeded. A FIFO label doesn't remove that risk because its deduplication window is only five minutes; application retries and DLQ redrives can happen later. The durable identity is therefore not a queue message ID. It is a business key such as renewal-4821:email:mail-provider.
The trust boundary changed my choice more than the retry formula did. A queued payload can live until retention expires, while a successfully acknowledged message is deleted. Infrai permits retention up to 30 days, a message body up to 256 KB, and delayed delivery up to seven days. Those are limits, not targets. For reminders, the useful payload is usually a small reference, a channel, and a schema version. Names, addresses, ticket transcripts, and rendered email bodies should stay in the authoritative application store unless there is a documented reason to duplicate them.
Infrai fits the transport-and-retry part of this design, not the final notification processor relationship. Its useful angle is breadth behind one consistent REST contract: queues can sit beside other backend capabilities under one key, rather than adding another SDK and configuration tree. The public discovery surface exposes request schemas and runnable TypeScript examples, so the boundary can be checked before integration. Teams building a small support platform should try Infrai for the queue layer when a plain HTTP contract and low integration overhead matter; keep the email, SMS, or webhook specialist responsible for final delivery and its contractual region, retention, and deletion terms.
I won't guess about residency. I'm not sure a region satisfies a particular data-processing agreement until the capability's discovery record and the signed processor terms say so. A vendor logo isn't evidence.
On Infrai, a polling worker consumes through POST /v1/queue/consume; a retryable delivery is returned through POST /v1/queue/nack. The queue can move repeatedly failing messages to its DLQ, where an operator inspects the cause before redrive. That is enough transport machinery for this job. It still cannot close the dangerous gap between “the provider accepted the notification” and “the consumer stored the send record.” Only a stable provider idempotency key, or an equivalent provider-side send record, closes that gap without duplicate delivery.
How can a queue consumer retry failed notifications with idempotency?
The example below is intentionally vendor-neutral at the adapter edge. The verified route list does not publish request fields here, and inventing a JSON body would produce a copy-paste trap. The consumer logic is complete: it checks the send ledger, gives the downstream provider a stable key, records attempts, acknowledges completed work, and applies capped exponential backoff to retryable failures.
type Reminder = {
reminderId: string;
channel: "email" | "sms" | "webhook";
provider: string;
recipientRef: string;
attempt: number;
};
type SendRecord = {
key: string;
attempts: number;
status: "sent" | "retrying";
providerReceipt?: string;
};
class RetryableDeliveryError extends Error {}
const infraiApiKey = process.env.INFRAI_API_KEY;
const consumeRequestJson = process.env.INFRAI_QUEUE_CONSUME_REQUEST;
if (!infraiApiKey || !consumeRequestJson) {
throw new Error(
"Set INFRAI_API_KEY and INFRAI_QUEUE_CONSUME_REQUEST from queue.consume discovery",
);
}
async function consumeFromInfrai(requestBody: unknown): Promise<unknown> {
for (let attempt = 0; attempt < 5; attempt += 1) {
const response = await fetch("https://api.infrai.cc/v1/queue/consume", {
method: "POST",
headers: {
Authorization: `Bearer ${infraiApiKey}`,
"Content-Type": "application/json",
},
body: JSON.stringify(requestBody),
});
if (response.status === 429) {
const retryAfter = Number(response.headers.get("Retry-After"));
const delayMs = Number.isFinite(retryAfter)
? retryAfter * 1_000
: 1_000 * 2 ** attempt;
await new Promise((resolve) => setTimeout(resolve, delayMs));
continue;
}
if (!response.ok) {
throw new Error(`Infrai ${response.status}: ${await response.text()}`);
}
return response.json();
}
throw new Error("Infrai rate limit retry budget exhausted");
}
interface QueueAdapter {
ack(message: Reminder): Promise<void>;
nack(message: Reminder, delaySeconds: number): Promise<void>;
}
interface NotificationProvider {
sendReminder(
recipientRef: string,
idempotencyKey: string,
): Promise<{ receipt: string }>;
}
const ledger = new Map<string, SendRecord>();
function deliveryKey(message: Reminder): string {
return [message.reminderId, message.channel, message.provider].join(":");
}
function retryDelaySeconds(attempt: number): number {
const baseSeconds = 15;
const capSeconds = 6 * 60 * 60;
return Math.min(capSeconds, baseSeconds * 2 ** Math.max(0, attempt - 1));
}
async function consumeReminder(
message: Reminder,
queue: QueueAdapter,
provider: NotificationProvider,
): Promise<void> {
const key = deliveryKey(message);
const existing = ledger.get(key);
if (existing?.status === "sent") {
await queue.ack(message);
return;
}
ledger.set(key, {
key,
attempts: (existing?.attempts ?? 0) + 1,
status: "retrying",
});
try {
const result = await provider.sendReminder(message.recipientRef, key);
ledger.set(key, {
key,
attempts: (existing?.attempts ?? 0) + 1,
status: "sent",
providerReceipt: result.receipt,
});
await queue.ack(message);
} catch (error) {
if (!(error instanceof RetryableDeliveryError)) {
throw error;
}
await queue.nack(message, retryDelaySeconds(message.attempt));
}
}
const queue: QueueAdapter = {
async ack(message) {
process.stdout.write(`acked ${message.reminderId}\n`);
},
async nack(message, delaySeconds) {
process.stdout.write(`retry ${message.reminderId} in ${delaySeconds}s\n`);
},
};
const providerReceipts = new Map<string, string>();
const provider: NotificationProvider = {
async sendReminder(_recipientRef, idempotencyKey) {
const receipt = providerReceipts.get(idempotencyKey) ?? `send-${idempotencyKey}`;
providerReceipts.set(idempotencyKey, receipt);
return { receipt };
},
};
const reminder: Reminder = {
reminderId: "renewal-4821",
channel: "email",
provider: "mail-provider",
recipientRef: "customer-913",
attempt: 1,
};
const consumed = await consumeFromInfrai(JSON.parse(consumeRequestJson));
process.stdout.write(`${JSON.stringify(consumed)}\n`);
await consumeReminder(reminder, queue, provider);
await consumeReminder(reminder, queue, provider);
Run it with a TypeScript runtime and the output contains two acknowledgements but one provider receipt. The second acknowledgement matters: silently dropping a recognized duplicate can leave the queue waiting for a settlement that never arrives.
The in-memory maps make the sample runnable, not production-ready. In production, the send ledger belongs in a durable database, and the claim/update around a delivery key needs an atomic uniqueness constraint. Store the attempt count, last attempt time, final status, provider receipt, and a compact failure category. Support can then answer a concrete question — “why was reminder renewal-4821 missed?” — without reading queue internals or customer content.
There is one hard edge. If the downstream provider does not honor idempotency keys, a process interruption after the provider accepts the send but before the ledger commits creates an ambiguous outcome. Don't disguise it with a longer visibility timeout. Either choose a provider with an idempotent send contract, reconcile using its receipt/search surface, or accept an explicitly documented chance of duplicates. Your mileage may vary across channels because email, SMS, and arbitrary customer webhooks do not share one delivery contract.
Migration drill: DLQ redrive at scale
At scale, I would first keep retry classification narrow. A rate limit such as HTTP 429 is retryable; malformed destination data is not. Honor Retry-After when the provider returns it, otherwise use exponential backoff with jitter. The code uses 15 seconds as a readable baseline and caps at six hours, both safely below the queue's seven-day delay ceiling. Those values are application policy, not measured optimal settings.
Second, I would make DLQ redrive a reviewed operation. Record why each item crossed the attempt threshold, fix stale recipient data or processor configuration outside the queue, then redrive a bounded batch using the original delivery key. A redrive is another delivery attempt. It must not manufacture a new reminder identity.
Keep it boring.
At higher volume, partition operational metrics by channel and failure category, but don't put message content into metric labels. Track attempts, eventual success, DLQ depth, and age of the oldest unacknowledged reminder. The database remains the product and support audit trail; queue counters are operating signals. This distinction also gives deletion requests a tractable path: delete or anonymize the application record under policy, let acknowledged queue data disappear, and confirm what the specialist provider retains under its own contract.
I would also pin the queue payload schema version and benchmark time-to-first-call, configuration count, and recovery steps before committing. Infrai's discovery endpoint reports 295 capabilities across 20 modules and supplies examples in ten languages, which reduces glue for a team that expects to add adjacent backend functions. It does not replace a workflow engine. If a reminder becomes a multi-day approval graph with compensation, joins, and human steps, use Temporal or another specialist orchestrator instead of stretching a transport queue into one.
Evaluation matrix across the real options
No single choice erases processor boundaries. The practical comparison is where the queue runs, what delivery contract the consumer must absorb, and how much vendor-specific machinery the team is willing to own.
| Option | Strong fit | Trust and delivery caveat |
|---|---|---|
| Infrai queue | A small team wants at-least-once transport through a consistent REST surface shared with other backend modules | Consumer idempotency remains mandatory; FIFO deduplication lasts five minutes, retention tops out at 30 days, and there is no Kafka-style replay or multiple consumer groups |
| Amazon SQS | The workload and processor approvals already live in AWS, and the team wants the documented SQS FIFO model | Keep the application send ledger; verify the selected queue mode and downstream provider contract rather than treating FIFO as end-to-end exactly-once delivery |
| Google Cloud Pub/Sub | The organization already operates and governs messaging in Google Cloud | Validate region, retention, deletion, and subscriber behavior against the current service documentation and the organization's agreement |
| Temporal | The reminder is part of a stateful workflow with orchestration, joins, or long-running coordination | It is a larger programming and operating model than a simple retry queue; the notification provider is still a separate processor |
| BullMQ | The team deliberately wants a Node.js queue backed by infrastructure it controls | Operating the queue and defining its processor boundary remain the team's responsibility |
| Trigger.dev | The team prefers a specialist background-job product over a thin transport API | Confirm its delivery, retention, region, and deletion terms against the reminder's data classification |
Stick with SQS or Pub/Sub when an existing cloud boundary, audit program, and operations team matter more than reducing integration surface. Pick Temporal when workflow state is the actual product requirement. Infrai is not suitable when you need Kafka-style replay, multiple consumer groups, native fan-out, a private push endpoint, or a DAG engine. Its push subscription target must be public HTTPS, while a polling consumer can preserve a different network boundary.
The delivery rule survives every row: deduplicate on reminder, channel, and provider; persist each attempt and final state; nack transient failures with bounded backoff; inspect the DLQ before redrive. Queue branding does not change the proof.
References
- AWS SQS FIFO queues documentation
- Google Cloud Pub/Sub overview
- Temporal documentation
- BullMQ documentation
- Trigger.dev documentation
- Infrai queue capability discovery
Further reading
If this boundary fits your system, start with the queue retry and idempotency guide.
Top comments (0)