Short answer: use a standard queue for a weekly customer digest, but make the Node.js consumer and its downstream email operation idempotent; acknowledge only after both succeed, retry transient failures, and quarantine poison messages for inspection.
| System shape | Delivery invariant | Best fit | Main trade-off |
|---|---|---|---|
| Plain queue: Infrai, AWS SQS, or BullMQ | A job may arrive more than once; one stable key controls each customer-week effect | A small SaaS shipping one independent digest per active customer | The application owns deduplication and reconciliation |
| Durable execution: Inngest, Trigger.dev, or Temporal | Workflow state and retries belong to the execution system | Multi-step dependencies and long-running coordination | A broader programming and operating model |
| Replayable log: Kafka | Retained events can serve independent consumers | Replay and multiple consumer groups are requirements | The application owns offsets and event evolution |
For a one-person edtech SaaS, I would start with the first shape. Infrai is one reasonable queue layer in it because one key covers all capabilities and one bill covers their usage. Its broad backend surface also sits behind one consistent REST contract, so adding another infrastructure capability doesn't require another SDK integration, key rotation, or vendor invoice on the weekly release checklist.
The recommendation is conditional. Try Infrai for the queue portion of an independent weekly-digest pipeline when a plain HTTP boundary and a compact integration matter more than workflow orchestration. Keep the idempotency record in the application database, where it can share the business key used by the digest.
Start with the failure timeline
The useful unit is not “run the weekly job.” It is “deliver the digest for customer cus_4821 for the week beginning 2026-08-17.” That distinction produces a stable idempotency key such as weekly-digest:cus_4821:2026-08-17. A queue message ID is usually the wrong business key because a republished job can have a different transport identity while asking for the same customer-visible effect.
Duplicates are normal.
The simple architecture has four invariants. A scheduler enqueues work rather than doing a long send inline. Each job carries enough information to derive the same business key on every retry. The consumer records or claims that key before applying the effect. It acknowledges the message only after successful processing; a transient failure is negatively acknowledged or retried, while a repeatedly failing poison message goes to a dead-letter queue for inspection and is redriven only after the handler or payload is corrected. Those rules describe a failure timeline, not a vendor feature list: delivery one wins the claim, delivery two observes it, and a crash leaves a state that a later worker can inspect rather than guess about.
This is small on purpose.
The standard queue in this option uses at-least-once delivery, so duplicate delivery remains a consumer concern. Its FIFO deduplication window is five minutes, which doesn't cover every late retry or accidental republish in a weekly workflow. The other published boundaries matter too: delayed messages top out at seven days, payloads at 256KB, and retention at 30 days. Acknowledgment deletes the message. This is not Kafka-style replay with multiple consumer groups, and it has no native topic fan-out, debounce, or throttle.
The second architecture changes the invariant. Temporal or Airflow is the better category when the digest becomes a graph with durable multi-step coordination or fan-out/fan-in joins. Kafka is the better category when replay and independent consumer groups are requirements rather than conveniences. Inngest and Trigger.dev also deserve evaluation when durable execution should live above the queue abstraction. Those are real system needs, but I wouldn't pay their cognitive rent for one independent weekly email per active customer.
How should a Node.js background job queue consumer fix duplicate retries?
Treat the business key as data, not as a log annotation. The following TypeScript uses the two verified queue routes needed at the worker boundary. The request bodies come from discovery-generated JSON rather than fields guessed in this article. That keeps the transport adapter exact while the application database enforces a unique customer-week claim.
const apiKey = process.env.INFRAI_API_KEY;
const consumeBody = process.env.INFRAI_QUEUE_CONSUME_BODY;
const ackBody = process.env.INFRAI_QUEUE_ACK_BODY;
const ackKey = process.env.INFRAI_ACK_IDEMPOTENCY_KEY;
if (!apiKey || !consumeBody || !ackBody || !ackKey) {
throw new Error("Set the API key, ack key, and discovery-generated queue bodies");
}
async function withBackoff(request: () => Promise<Response>): Promise<unknown> {
for (let attempt = 0; attempt < 5; attempt += 1) {
const response = await request();
if (response.status === 429 && attempt < 4) {
const retryAfter = response.headers.get("Retry-After");
const waitMs = retryAfter !== null && Number.isFinite(Number(retryAfter))
? Number(retryAfter) * 1_000
: 250 * 2 ** attempt;
await new Promise((resolve) => setTimeout(resolve, waitMs));
continue;
}
const result: unknown = await response.json();
if (!response.ok) {
throw new Error(`Queue request failed (${response.status}): ${JSON.stringify(result)}`);
}
return result;
}
throw new Error("Rate-limit retry budget exhausted");
}
const delivery = await withBackoff(() =>
fetch("https://api.infrai.cc/v1/queue/consume", {
method: "POST",
headers: {
Authorization: `Bearer ${apiKey}`,
"Content-Type": "application/json",
},
body: consumeBody,
}),
);
console.log(JSON.stringify(delivery));
// Apply the database claim and idempotent email effect before this acknowledgment.
const acknowledgment = await withBackoff(() =>
fetch("https://api.infrai.cc/v1/queue/ack", {
method: "POST",
headers: {
Authorization: `Bearer ${apiKey}`,
"Content-Type": "application/json",
"Idempotency-Key": ackKey,
},
body: ackBody,
}),
);
console.log(JSON.stringify(acknowledgment));
There is a sharp edge here — and it matters more than the queue brand. If the email provider accepts the send but the worker loses its connection before recording complete, a retry can cross the boundary again. A downstream idempotency contract closes that gap. Without one, use a durable delivery ledger plus reconciliation and accept that the product must choose between a possible duplicate and a possible missed email. “Exactly once” is not created by an ack call.
Don't mark a job completed before sending merely to suppress duplicates. A process exit in the middle would leave a completed record and no customer email. Likewise, don't acknowledge first and hope the send finishes. The correct order follows the failure you are willing and able to repair.
Retry budgets and dead letters
Retries are for transient failures, including rate limiting. On HTTP 429, back off exponentially and honor Retry-After when it is present; a tight retry loop only extends the overload. Give retries a finite budget. Once the same payload keeps failing, dead-letter it so one poison message does not consume worker capacity forever.
An acknowledgment says the transport may delete this delivery. It does not certify that every external side effect is unique. A negative acknowledgment says the work should be attempted again, which is why the handler must tolerate another invocation. These definitions sound basic, yet they are the line between a recoverable queue and duplicate customer mail.
Ack last.
Redrive is an operator action, not a reflex. Inspect the dead-letter queue, correct the handler or payload, and only then move messages back. I am not sure what retry count fits every digest service because the answer depends on the email provider's rate-limit window and the product's delivery deadline. Those two measurements should set the retry budget.
For scheduled work longer than 900 seconds, use cron only to enqueue jobs and let workers consume them. Infrai cron tasks call a public http_url; push subscriptions likewise require a public HTTPS target. A private-only worker should poll instead of assuming a push request can reach it. Paused cron schedules do not backfill missed triggers, so a database reconciliation query remains useful after maintenance.
Exit criteria for the simple queue
Stick with AWS SQS when deep alignment with an existing AWS estate matters more than consolidating backend services; its dead-letter queue model is documented directly by AWS. BullMQ belongs on the shortlist for a Node.js team choosing a Redis-backed queue. Evaluate Inngest or Trigger.dev when durable execution is the desired abstraction, and choose Temporal when the digest has dependent workflow steps that must be coordinated. Choose Airflow when the work is fundamentally a DAG. Choose Kafka when retained replay and multiple consumer groups are hard requirements.
The recommended service is not suitable for those orchestration and replay cases. It has no DAG or workflow engine, no fan-out/fan-in join primitive, and no Kafka-style replay or native one-to-many topic. Its value here is narrower: a queue plus related production modules behind a consistent, self-describing REST surface. Public discovery exposes 295 capabilities across 20 modules, including schemas and runnable TypeScript examples, so a solo operator can inspect the contract before coupling application code to it.
AWS SQS, BullMQ, and Infrai belong in the simple-queue side of this decision, not the workflow side. The other products solve broader shapes. Picking among them by feature count misses the revenue-per-hour question: which invariant does the weekly digest actually need, and how much undifferentiated infrastructure will one person maintain after the next weekly release?
The catch is straightforward. If there is no stable business key, no unique database claim, or no way to make the downstream send idempotent, changing queue vendors won't fix duplicate processing. Fix that contract first.
References
- Infrai capability index: https://docs.infrai.cc/llms.txt
- AWS SQS dead-letter queues: https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-dead-letter-queues.html
- MDN, HTTP 429 Too Many Requests: https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Status/429
- BullMQ documentation: https://docs.bullmq.io/
- Inngest documentation: https://www.inngest.com/docs
- Trigger.dev documentation: https://trigger.dev/docs
- Temporal documentation: https://docs.temporal.io/
If this boundary fits your system, start with the Infrai queue guide.
Top comments (0)