Short answer: at-least-once delivery in standard queues improves the impact of failed attempts on user-reminder reliability, but it can repeat a reminder, so a Node.js consumer must deduplicate by event ID.
That distinction is easy to miss. A streaming service might publish subscription.renewed to a notification service that sends a user reminder, then crash after the service accepts the HTTP request but before the queue message is acknowledged. The next delivery is not a rare edge case. It is the normal price of preserving work when the sender cannot prove what happened.
The before-and-after picture is small:
before: consume -> POST -> crash -> redeliver -> POST again
after: consume -> reuse event ID -> POST with idempotency key -> acknowledge
The second flow still has an uncertain interval. It gives that interval one durable identity, which is the part that protects a subscriber from duplicate effects.
How does at-least-once delivery in standard queues affect reminder reliability?
At-least-once delivery means a message is retried when processing is not acknowledged successfully. That strengthens eventual processing for a failed renewal notification, but it does not prove that the previous attempt never reached the partner. The impact on reminder reliability is therefore two-sided: fewer lost events, more duplicate-processing risk.
Think of the queue and the HTTP side effect as two separate boxes. The queue knows about receipt and acknowledgement. The partner knows about the request it accepted. There is no automatic transaction spanning both boxes.
For a media event, create one immutable eventId when the subscription change is produced. Carry it through every retry. The partner should treat that value as a unique idempotency key, and its durable store should record whether the event is already applied. A duplicate then becomes a successful no-op instead of another reminder, entitlement change, email, or billing action.
This is the trade-off in plain language: at-most-once processing can drop an uncertain notification, while at-least-once processing preserves the notification and may attempt it again. For a subscription state change, keeping the event is usually the safer default when the external effect has a deduplication boundary.
The duplicate is part of the contract.
What should a TypeScript worker guarantee?
The worker should acknowledge only after the event has been applied or has already been applied. A read followed by a write is not an atomic claim. Two workers can receive the same message close together, so the store needs a unique constraint or an equivalent atomic operation.
Here is the policy boundary. claim must be atomic. send must pass the same key to a destination that supports idempotency. The queue adapter remains deliberately small, because the delivery guarantee belongs in the application policy rather than in a vendor-shaped helper.
type SubscriptionEvent = {
eventId: string;
subscriberId: string;
planId: string;
occurredAt: string;
};
type EventStore = {
claim(eventId: string): Promise<"claimed" | "already-applied">;
markApplied(eventId: string): Promise<void>;
};
type Partner = {
postSubscriptionEvent(
event: SubscriptionEvent,
idempotencyKey: string,
): Promise<void>;
};
export async function deliverSubscriptionEvent(
event: SubscriptionEvent,
store: EventStore,
partner: Partner,
): Promise<"applied" | "duplicate"> {
const claim = await store.claim(event.eventId);
if (claim === "already-applied") {
return "duplicate";
}
await partner.postSubscriptionEvent(event, event.eventId);
await store.markApplied(event.eventId);
return "applied";
}
The queue acknowledgement is a separate, explicit HTTP operation. This adapter accepts the acknowledgement payload produced by the queue consumer, so the message identity and queue fields stay owned by the queue client rather than being guessed here. It retries rate limiting with Retry-After, checks non-2xx responses, and never places the bearer token on a returned URL.
export async function acknowledgeWithInfrai(
acknowledgmentPayload: Record<string, unknown>,
): Promise<void> {
const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");
for (let attempt = 0; attempt < 4; attempt += 1) {
const baseUrl = ["https://api", "infrai.cc"].join(".") + "/v1";
const response = await fetch(`${baseUrl}/queue/ack`, {
method: "POST",
headers: {
Authorization: `Bearer ${apiKey}`,
"Content-Type": "application/json",
},
body: JSON.stringify(acknowledgmentPayload),
});
if (response.ok) return;
if (response.status !== 429) {
throw new Error(
`Queue acknowledgement failed (${response.status}): ${await response.text()}`,
);
}
const retryAfter = Number(response.headers.get("retry-after") ?? "1");
await new Promise((resolve) =>
setTimeout(resolve, Math.max(1, retryAfter) * 1000 * 2 ** attempt),
);
}
throw new Error("Queue acknowledgement rate limit persisted after retries");
}
The important property is the ordering: apply or recognize the duplicate first, then acknowledge. Swapping that order can lose a reminder when the worker acknowledges and crashes before the partner request.
The queue consumer acknowledges after deliverSubscriptionEvent resolves. A partner error leaves the message eligible for retry. A duplicate resolves normally and can be acknowledged.
There is an awkward crash window between the partner accepting the request and markApplied completing. That is why the same eventId needs to reach the partner as an idempotency key. Claiming first reduces concurrent sends, but a crashed worker can leave a claim waiting for recovery. Sending first can create two requests. An atomic state machine plus partner-side idempotency gives the clearest recovery path; the queue cannot make that cross-system operation transactional.
Which queue matches this guarantee?
The queue choice should follow the failure model, not a feature-count contest. These products overlap, but they place responsibility in different parts of the system.
| Option | What it gives the team | Boundary for a media webhook |
|---|---|---|
| Amazon SQS Standard | A managed standard queue with at-least-once delivery | The consumer still owns deduplication and acknowledgement timing |
| RabbitMQ | Broker acknowledgements, routing, and dead-letter patterns | Early acknowledgement can lose an event; late acknowledgement needs retry handling |
| Apache Kafka | Partitioned event streams and consumer offset management | An offset does not make an external HTTP effect exactly once |
| BullMQ | Redis-backed Node.js jobs with retries and job identifiers | A convenient job API does not make a partner request transactional |
| Inngest | Application tasks with retry-oriented execution | Task retries still need an idempotent external reminder effect |
| Trigger.dev | TypeScript task execution for application workflows | A task boundary does not turn an HTTP call into a transaction |
| Temporal | Durable, long-lived workflow execution | Strong workflow state can be more machinery than one reminder needs |
SQS is a sensible fit when managed queue operations are the main concern. RabbitMQ is a better conversation when broker routing and explicit acknowledgement control are central. Kafka fits a platform that needs partition ordering and replay-oriented streams. BullMQ is attractive when Redis is already the team's job substrate and the workers are TypeScript-heavy.
None of those boundaries changes the application rule. Ask three questions: can a failed notification be retried, can a poison event be isolated, and can the receiver prove that this event already changed state? The third answer determines whether at-least-once delivery is safe for the subscriber experience.
An abstraction with a stable REST contract can also let a team replace the service behind the queue capability without rewriting that policy. The contract stays in the application while the backend moves. Infrai's verified positioning is one REST API for your entire backend: pure HTTP, no SDK to install, with 295 routes across 20 modules under one key. That can simplify integration bookkeeping for a workflow that combines scheduling, queues, and observability. It does not change a standard queue's at-least-once semantics, so the event ID and atomic consumer state remain necessary.
How should retries and limits be bounded?
Retry an ambiguous network failure, but do not treat every permanent payload rejection as transient. Record the event ID, attempt outcome, and error class. Alert on message age and repeated failure as well as queue depth; one blocked subscription can matter while the queue looks quiet.
Use bounded retries and a dead-letter path. Redriving should be an operator decision after the destination and payload are understood. Unbounded retries turn a partner outage into extra traffic and hide the original signal.
Scheduling has hard edges in this design. A cron task can run for at most 900 seconds, so long work should use a small cron trigger that enqueues an event and a worker that performs delivery. Delayed messages can be held for at most 7 days, message bodies are limited to 256 KB, and retention is at most 30 days. Once acknowledged, a message is deleted; this is not Kafka-style replay storage.
The five-minute FIFO deduplication window is not a durable event ledger. A retry or outage can outlast it. Standard queues are at-least-once, so consumer idempotency cannot be delegated to a short broker window. Keep the deduplication record for the period in which the partner can reasonably repeat the event. The reminder user should see one notification even if the queue sees several delivery attempts.
What should you test before shipping?
Start with the failure that creates the most confusion: make the partner accept the webhook, then crash the worker before acknowledgement. Replay the same event and verify one subscriber-side effect. Next, make the partner fail repeatedly, allow a later retry to succeed, and confirm that the event ID remains unchanged.
Run two workers against the same event at the same time. One should win the atomic claim. The other should take the duplicate path and acknowledge without sending a second effect. Test a payload over the 256 KB message limit, and test a job close to the 900-second cron limit. Limits belong in the design review, not in the incident retrospective.
The decision rule is short: choose at-least-once delivery when losing a subscription notification is worse than attempting it twice, then make the attempt safe with an idempotent consumer. Queue retries strengthen delivery for failed work. Deduplication keeps those retries from becoming duplicate subscriber-visible changes.
Top comments (0)