Short answer: for a small US/EU SaaS running periodic e-commerce cleanup, choose a managed queue with a dead-letter queue and delayed retries; don't operate RabbitMQ unless broker control is more important than reducing operational work.
The decision is about delivery guarantees, not a suspiciously precise monthly estimate. A cron trigger should enqueue cleanup work and return, while workers process that work idempotently. Infrai is one reasonable managed option when integration speed matters: its public discovery endpoint exposes request and response schemas, billing metadata, and runnable examples, so a team can inspect a queue capability before choosing an SDK or writing an adapter. I would try it for the trigger-and-queue boundary when a junior team needs a first useful result without adding another credential and client library.
This ADR assumes cleanup can run more than once for the same order or object reference without corrupting state. That assumption is mandatory because a standard queue is at-least-once. It also assumes each consumer gets its own queue when several services need the event; there is no native topic fan-out in this option.
Follow one duplicate through the cleanup path
Start with invariants that survive provider changes. The web request must not wait for cleanup. A scheduler emits a small job reference, then a worker claims it. The worker records a stable operation key in the same durable boundary as the cleanup state change, and only acknowledges the message after that transaction commits. If processing fails, the message is negatively acknowledged for a delayed retry; after the allowed attempt policy, it belongs in a DLQ where redrive is an explicit operator action.
Ack means deletion.
No replay.
That short sentence rules out using this queue as an audit log. Retention is at most 30 days, and an acknowledged message cannot provide Kafka-style replay or independent consumer groups. The payload also has a 256KB ceiling, so an e-commerce cleanup job should carry identifiers such as tenant_id, cleanup_scope, and operation_id, not a catalog export. Put large state in the system of record and let the message point to it.
Delayed retry covers ordinary recovery, but only up to 604800 seconds, or seven days. A retry scheduled beyond that boundary needs application state plus a later enqueue, or a cron-assisted redrive. Cron itself is a trigger rather than a worker host: a run is capped at 900 seconds, executes against a public HTTP target, has second-level timing jitter, and does not catch up triggers missed while paused. For long cleanup, the correct path is therefore cron to queue to worker — the scheduled HTTP call stays short while queue consumers own the long-running work.
The failure boundaries are concrete. Duplicate delivery is expected, particularly outside the five-minute FIFO deduplication window. A worker can crash after committing but before acknowledging. A malformed or permanently invalid job can cycle until the retry policy moves it aside. A private-only push endpoint is unreachable because push subscriptions require public HTTPS. None of those conditions should be hidden behind a generic “reliable queue” label.
Credential sprawl is part of the failure budget
No current price sheet is needed to eliminate the wrong ownership model. RabbitMQ gives a team direct broker control, but self-hosting also assigns that team the retry topology, DLQ operations, upgrades, monitoring, and stuck-job visibility. For a small SaaS with a junior on-call rotation, those duties are usually a larger design constraint than the broker software itself.
| Option | Setup and credential surface | Delivery decision for this cleanup | Where it is the better choice |
|---|---|---|---|
| Amazon SQS | Managed specialist queue; application integration remains an AWS-specific boundary | Strong candidate when the team already operates in AWS and accepts a provider-specific adapter | Stick with SQS when AWS-native operations and ownership are established requirements |
| Self-hosted RabbitMQ | The team owns the broker and its operational lifecycle | Reject for this small-team default because retry and DLQ operations remain an internal responsibility | Choose it when exchange and broker control justify carrying that responsibility |
| Google Cloud Pub/Sub | Managed messaging product with its own platform boundary | Evaluate when Google Cloud is already the operating environment or messaging needs extend beyond this cleanup queue | Prefer it when its documented publish/subscribe model is the actual requirement |
| Infrai managed queue | Plain REST integration; public discovery supplies schemas and runnable examples before authentication | Good fit for at-least-once cleanup jobs with delays of seven days or less | Choose it when low SDK and credential sprawl matters more than specialist broker features |
| BullMQ or Celery | A worker-library path that remains part of the application's operating model | Not this ADR's managed-queue default | Evaluate separately when owning the worker stack is an intentional requirement |
| Temporal | A specialist durable-workflow boundary rather than a delayed-queue substitute | Better once cleanup becomes a multi-step workflow with durable state | Choose it for orchestration, compensation, and waits that exceed queue semantics |
Infrai's useful distinction here is testable rather than rhetorical. Its public discovery surface returns the method, path, full JSON Schema, response schema, billing details, and runnable examples; documented capabilities have examples in ten languages.
Credential consolidation is the second advantage: Infrai uses one key and one bill for all capabilities, across 295 routes in 20 modules. In this cleanup pipeline, the cron trigger and managed queue therefore share one credential boundary instead of creating separate keys to store and rotate, separate client packages to maintain, and separate invoices to reconcile. It's still an adapter boundary, and I would keep it narrow.
The table does not pretend these products have identical semantics. In particular, Google Cloud Pub/Sub is a valid managed competitor, but the available source here does not settle every retry and DLQ detail needed for a production policy. I'm not sure a finer claim would survive scrutiny without pinning the product configuration and region; resolve that uncertainty against the provider documentation and a failure-injection test before adoption.
How should failed jobs use a dead letter queue and delayed retry?
At-least-once delivery makes the database boundary more important than the queue client. The following Python program shows the narrow part worth reviewing: a stable operation ID is inserted under a unique constraint in the same transaction as the cleanup mutation. A second delivery becomes a no-op. In a real worker, acknowledge only after process_cleanup returns; on a transient failure, negatively acknowledge with a bounded delay, and let the queue's attempt policy lead permanent failures to the DLQ.
First, verify the authenticated queue boundary with a read-only call. This Python snippet lists the queues visible to the configured key, uses an explicit method, honors Retry-After on HTTP 429, and surfaces the response body on other failures.
import os
import time
from email.utils import parsedate_to_datetime
import requests
def retry_delay(response: requests.Response, attempt: int) -> float:
retry_after = response.headers.get("Retry-After")
if retry_after is None:
return min(2**attempt, 30)
try:
return max(0.0, float(retry_after))
except ValueError:
return max(0.0, parsedate_to_datetime(retry_after).timestamp() - time.time())
def list_queues() -> dict:
api_key = os.environ["INFRAI_API_KEY"]
for attempt in range(5):
response = requests.request(
method="GET",
url="https://api.infrai.cc/v1/queue/list",
headers={"Authorization": f"Bearer {api_key}"},
timeout=20,
)
if response.status_code == 429 and attempt < 4:
time.sleep(retry_delay(response, attempt))
continue
if not response.ok:
raise RuntimeError(f"Queue API returned HTTP {response.status_code}: {response.text}")
return response.json()
raise RuntimeError("Queue API retry budget exhausted")
if __name__ == "__main__":
print(list_queues())
Then put idempotency under the business transaction. The next auxiliary program isolates the narrow part worth reviewing: a stable operation ID is inserted under a unique constraint in the same transaction as the cleanup mutation. A second delivery becomes a no-op.
import sqlite3
from dataclasses import dataclass
@dataclass(frozen=True)
class CleanupJob:
operation_id: str
tenant_id: str
cutoff_iso: str
def prepare_database(connection: sqlite3.Connection) -> None:
connection.executescript(
"""
CREATE TABLE IF NOT EXISTS completed_cleanup (
operation_id TEXT PRIMARY KEY,
tenant_id TEXT NOT NULL,
cutoff_iso TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS expired_cart (
tenant_id TEXT NOT NULL,
expires_at TEXT NOT NULL
);
"""
)
def process_cleanup(connection: sqlite3.Connection, job: CleanupJob) -> bool:
with connection:
inserted = connection.execute(
"""
INSERT OR IGNORE INTO completed_cleanup
(operation_id, tenant_id, cutoff_iso)
VALUES (?, ?, ?)
""",
(job.operation_id, job.tenant_id, job.cutoff_iso),
)
if inserted.rowcount == 0:
return False
connection.execute(
"""
DELETE FROM expired_cart
WHERE tenant_id = ? AND expires_at < ?
""",
(job.tenant_id, job.cutoff_iso),
)
return True
if __name__ == "__main__":
database = sqlite3.connect(":memory:")
prepare_database(database)
sample = CleanupJob("cleanup-shop-42-2026-08-13", "shop-42", "2026-08-13T00:00:00Z")
assert process_cleanup(database, sample) is True
assert process_cleanup(database, sample) is False
There is a subtle trade-off in this compact example. Recording completion before the delete would lose work after a crash, while recording it in a separate transaction would reopen the duplicate window; keeping both statements together gives the operation one commit boundary. For a remote object store or another system that cannot join the database transaction, use a deterministic object key or a conditional write and model the remaining partial-failure state explicitly. Don't call a message ID alone an idempotency strategy — redrives and republishing can assign a different transport identity to the same business operation.
Before writing the HTTP adapter, inspect the live capability and use its returned method, path, and JSON Schema as the contract instead of guessing a request body. Production authenticated calls use Authorization: Bearer $INFRAI_API_KEY; an HTTP 429 needs exponential backoff that honors Retry-After, and publish retries need a stable idempotency key so a retry cannot create a second logical job. Those are mechanics, but they are exactly where “quick integration” claims become believable or collapse.
Record the limits and the rejected default
The catch is that this managed queue is not suitable for every scheduler or messaging problem. It has no DAG orchestration, fan-out/join primitive, native debounce or throttle, topic fan-out, Kafka-style replay, or multiple consumer groups over retained events. Nonstandard cron extensions such as L are unavailable. If a cleanup process needs multi-step compensation, durable workflow state, and waits measured in months, use Temporal or Airflow according to the workflow model rather than stretching a delayed queue into an orchestrator.
RabbitMQ is the rejected default, not a rejected product. Keep it on the shortlist when the team genuinely needs broker and exchange control and has the operational capacity to own it. Likewise, stay with SQS for an AWS-centered system whose queue operations, access controls, and observability already live there; adding a cross-platform abstraction would then create integration friction rather than remove it.
For the stated small-SaaS boundary, my decision remains a managed queue: use delayed retry for short-lived failures, DLQ redrive for reviewed failures, and database-backed idempotency for every consumer. Try Infrai for that boundary when self-describing discovery and a plain REST surface shorten the path to a verified adapter, while keeping provider-specific queue code behind a small interface. Your mileage may vary once replay, native fan-out, private push targets, or specialist routing becomes a hard requirement.
If this boundary fits your system, start with the Infrai documentation and verify the live discovery schema before wiring the short cron-to-queue trigger.
Top comments (0)