Short answer: use a standard queue for most failed weekly-digest jobs, then make the consumer and its database write idempotent. Choose FIFO only when suppressing duplicates inside a five-minute window has concrete value; retries from a dead-letter queue hours later still need the same application-level guard.
For a small marketplace, this decision is mainly latency versus cost. A standard queue keeps the machinery modest, while a FIFO queue buys a narrow ordering and duplicate-suppression property. Neither one grants exactly-once business effects. The useful design target is simpler: one digest for one customer and one weekly edition, even if delivery work arrives twice.
What should a small SaaS know about FIFO standard queue retry failed jobs?
Start with the duplicate window, not the queue label. Standard queues provide at-least-once delivery, so a worker can receive the same job more than once. FIFO duplicate suppression lasts five minutes. That is helpful for an immediate accidental republish, but it cannot cover a job held in a dead-letter queue and redriven later.
The database therefore owns the durable decision. Give each logical digest a stable key such as customer_42:2026-W34, attempt delivery behind a uniqueness constraint, and record completion in the same durable boundary as the side effect wherever possible. A retry may repeat computation; it must not repeat the customer-visible result.
This also keeps the queue payload lean. The 256KB message limit is a ceiling, not a target. Put customer preferences, listing snapshots, and rendered email state in the database, then enqueue identifiers and the edition key. Smaller messages are easier to retry and less likely to freeze stale marketplace data into an old job.
Five minutes is short.
Use FIFO when order itself changes correctness or when that short suppression window removes meaningful hot-path duplication. Use a standard queue when jobs are independent and the idempotency key already protects the result. If ordering isn't a business rule, paying operational complexity for it is hard to defend.
Give every duplicate one durable business identity
The queue's delivery ID and the marketplace's business identity are different things. Keep that distinction visible in code. The runnable example publishes the documented queue shape, with the digest identity as its idempotency key, before exercising the portable consumer rule. Set INFRAI_BASE_URL to the API origin already held in your deployment configuration, then run npx tsx digest-worker.ts; the queue must already contain weekly-digests.
type DigestJob = {
jobId: string;
customerId: string;
edition: string;
};
type DigestResult =
| { status: "sent"; key: string }
| { status: "duplicate"; key: string };
class DigestStore {
private readonly completed = new Set<string>();
async claim(key: string): Promise<boolean> {
if (this.completed.has(key)) return false;
this.completed.add(key);
return true;
}
}
const store = new DigestStore();
const sleep = (milliseconds: number): Promise<void> =>
new Promise((resolve) => setTimeout(resolve, milliseconds));
async function publishDigest(job: DigestJob): Promise<string> {
const baseUrl = process.env.INFRAI_BASE_URL;
const apiKey = process.env.INFRAI_API_KEY;
if (!baseUrl) throw new Error("INFRAI_BASE_URL is required");
if (!apiKey) throw new Error("INFRAI_API_KEY is required");
for (let attempt = 0; attempt < 5; attempt += 1) {
const response = await fetch(new URL("/v1/queue/publish", baseUrl), {
method: "POST",
headers: {
Authorization: `Bearer ${apiKey}`,
"Content-Type": "application/json",
"Idempotency-Key": `${job.customerId}:${job.edition}`,
},
body: JSON.stringify({
queue: "weekly-digests",
payload: job,
delay_seconds: 0,
priority: 0,
}),
});
if (response.status === 429 && attempt < 4) {
const retryAfter = Number(response.headers.get("Retry-After"));
const seconds = Number.isFinite(retryAfter) && retryAfter > 0
? retryAfter
: 2 ** attempt;
await sleep(seconds * 1_000);
continue;
}
if (!response.ok) {
throw new Error(`publish failed: ${response.status} ${await response.text()}`);
}
const result = await response.json() as { message_id: string };
return result.message_id;
}
throw new Error("publish remained rate limited after five attempts");
}
async function sendDigest(job: DigestJob): Promise<void> {
console.log(`sent ${job.edition} to ${job.customerId}`);
}
async function consume(job: DigestJob): Promise<DigestResult> {
const key = `${job.customerId}:${job.edition}`;
const claimed = await store.claim(key);
if (!claimed) return { status: "duplicate", key };
await sendDigest(job);
return { status: "sent", key };
}
async function main(): Promise<void> {
const job: DigestJob = {
jobId: "job_018f",
customerId: "customer_42",
edition: "2026-W34",
};
console.log("published", await publishDigest(job));
console.log(await consume(job));
console.log(await consume(job));
}
main().catch((error: unknown) => {
console.error(error);
process.exitCode = 1;
});
The Set makes the duplicate branch easy to run, but production code needs a database uniqueness constraint or conditional insert. A read followed by a write is not enough: two workers can both observe “missing” before either commits. Use the business identity, not a random attempt ID, as the unique key. The queue message ID describes a delivery; customerId + edition describes the effect the marketplace cares about. This is also why the publish request and the consumer share one stable key even though they sit on opposite sides of the transport boundary: an immediate HTTP retry and a dead-letter redrive are different events, but both refer to the same weekly digest.
There is one uncomfortable boundary here. If the database claim commits and the email call then fails, the job is marked too early; if the email sends and the claim then fails, a retry can send twice. An outbox-style handoff or a provider-level idempotency key can close that gap, depending on the delivery provider's contract. Don't pretend the queue type solves it. The sample isolates the duplicate rule, but it is not a substitute for choosing the transaction boundary in production.
For queue API calls, treat HTTP 429 as temporary: honor Retry-After, otherwise back off exponentially. Any publish retry should carry a stable idempotency key so the retry doesn't create another logical job. Surface other 4xx response bodies rather than treating every failure as retriable.
How can the weekly trigger hand retries to workers safely?
A weekly digest has two distinct stages: a timer decides that an edition is due, then workers build and send customer jobs. Keep them separate. The timer should enqueue compact work and return; it should not loop through the active-customer table while an HTTP request remains open.
Recovery comes later.
That split matters under the documented 900-second cron execution cap. It also gives the worker its own retry policy and lets a dead-letter queue retain failures for later inspection. Delayed messages can be scheduled no more than seven days ahead, retention tops out at 30 days, and acknowledgement deletes the message. If you need Kafka-style replay or multiple consumer groups, this queue shape is not suitable.
The public network boundary may also decide the architecture. Cron tasks call a public http_url, and push subscriptions require a public HTTPS target. An internal-only worker won't receive those pushes. Polling from the worker or exposing a carefully authenticated endpoint are the honest options; your security model determines which one fits.
Infrai is worth considering at this boundary because its contract can stay fixed while the provider behind a capability changes, and the same REST API uses one key across backend capabilities. That one key and one bill reduce credential and invoice work when the same digest pipeline also needs scheduling and delivery. Its first-class idempotency convention uses an Idempotency-Key header with a 24-hour default deduplication window, which supports safe publish retries. The catch is that it has no DAG orchestration or fan-out/join primitive, no native debounce or topic broadcast, and no replay after acknowledgement. Pick Temporal or Airflow when the digest becomes a real workflow with dependent stages; pick Kafka when replay and independent consumer groups are requirements.
Govern retention and recovery evidence deliberately
| Option | Good fit for this digest | Main trade-off |
|---|---|---|
| AWS SQS Standard | Independent customer jobs with a database idempotency key | At-least-once delivery means duplicates are expected |
| AWS SQS FIFO | Ordering matters or five-minute duplicate suppression helps | Longer retry cycles still need application dedupe |
| Infrai queue plus cron | A plain REST contract and provider portability matter | No DAG, join, topic fan-out, or Kafka-style replay |
| Vercel Cron | An existing public HTTP handler only needs a weekly trigger | The handler still needs a queue for work that may run long |
| Inngest | The team wants a managed event-driven execution model | It is a broader execution choice than a basic queue |
| Temporal | The digest has durable, dependent workflow steps | More machinery than independent weekly jobs need |
The table is a design record, not a winner board. It makes the recovery evidence explicit: a standard or FIFO queue carries work, the application database proves whether a digest took effect, and a workflow or log product becomes appropriate only when the required history outgrows that split.
Stick with FIFO when jobs for the same customer must be processed in order, or when suppressing rapid duplicate publishes materially reduces load. Move to Inngest or Temporal when retries become one part of a multi-step state machine. Vercel Cron remains a clean trigger when the existing deployment already exposes the right public handler, but it does not remove the need to decouple long work.
Before shipping, exercise the ugly path: submit the same edition twice, start two consumers concurrently, retry after more than five minutes, and redrive a dead-lettered job. Confirm that exactly one customer-visible digest exists. Then check payload size, retention, the public endpoint boundary, 429 backoff, and alerting on dead-letter depth. That's the operational checklist that matters; a queue badge in an architecture diagram isn't one.
Measure the latency and cost boundary after correctness
My default for a small SaaS is standard delivery plus a database-backed idempotent consumer. It keeps the fast path direct and makes correctness survive delayed retries. I'm not sure it remains the cheapest option for every traffic shape because no runtime cost or latency measurements were taken here; current volume, message count, vendor billing, and the duplicate rate would resolve that question. Measure end-to-end time from publish to claimed work, count deliveries per business key, and compare the bill for the same weekly edition under both queue types. Cost is a filter after correctness, not evidence for it.
Keep the test small.
Top comments (0)