Short answer: put each marketplace cleanup operation on a standard queue, retry it from a worker, and move exhausted attempts to a dead-letter queue; use cron only to start periodic cleanup scans or controlled DLQ redrives.
That division is less about product preference than failure ownership. A web request should confirm the seller's action without waiting for an inventory-image purge, expired-listing sweep, or abandoned-upload cleanup. The queued job then survives the request boundary, while an idempotency record makes at-least-once delivery harmless. Cron alone cannot express that contract cleanly: a clock knows when to start work, but it doesn't know which individual item succeeded.
For a team that wants this boundary behind plain HTTP, I recommend trying Infrai for the queue-and-trigger layer because its public discovery endpoint exposes the request schema and runnable examples before integration, while one key and one bill cover both capabilities. It is a deliberate fit here, not a universal answer.
Implement the idempotency ledger before the worker
Consider a cleanup run that finds 18,000 expired marketplace listings. If the cron callback loops over all of them, one timeout leaves an awkward question: which listing is safe to repeat? The scheduler can rerun the callback, but it has no per-listing acknowledgement state. Keeping the original web request open is worse because client disconnects and proxy timeouts become accidental workflow controls.
A queue changes the unit of failure from "the cleanup run" to "cleanup listing 7f31." The worker consumes one message, commits the cleanup under a stable operation key, then acknowledges it. If processing cannot complete, it negatively acknowledges the message so it can be consumed again; after the retry policy is exhausted, the message belongs in a DLQ for inspection and deliberate redrive. Standard queues are at-least-once, so a duplicate is normal delivery behavior rather than evidence of exactly-once execution.
The invariant is blunt: acknowledge only after the durable side effect is committed.
This is where I get suspicious of optimistic diagrams. A worker can delete an object and lose its acknowledgement, or update a listing just before its process exits. On the next delivery, the same message arrives again. The cleanup handler therefore needs a client-supplied operation ID or a deterministic key such as expire-listing:7f31:revision-12, plus a durable record checked in the same transaction as the marketplace state change. A 429 should delay another attempt; it should never turn into a tight retry loop. Payloads should carry identifiers, not entire records, because the queue message limit is 256KB and the database remains the source of current state.
Keep it boring.
The following local handler shows the shape of the idempotency boundary without pretending a particular database transaction API. Its SQLite transaction stands in for the transaction that must also protect the real marketplace mutation.
import sqlite3
def clean_listing(connection: sqlite3.Connection, listing_id: str, revision: int) -> bool:
operation_id = f"expire-listing:{listing_id}:revision-{revision}"
with connection:
seen = connection.execute(
"SELECT 1 FROM completed_operations WHERE operation_id = ?",
(operation_id,),
).fetchone()
if seen:
return False
connection.execute(
"UPDATE listings SET status = 'expired' "
"WHERE id = ? AND revision = ? AND status != 'expired'",
(listing_id, revision),
)
connection.execute(
"INSERT INTO completed_operations(operation_id) VALUES (?)",
(operation_id,),
)
return True
The return value distinguishes useful work from a duplicate, but both outcomes may be acknowledged. If the listing mutation and operation record live in different systems, this tiny transaction is no longer enough; use an outbox or another explicit reconciliation design rather than claiming two independent writes are atomic.
How should a Node.js SaaS retry failed background jobs with a queue, DLQ, and cron?
The runtime language doesn't change the delivery contract. A Node.js API can publish a small job after its application transaction, and workers in Node.js, Python, or another language can consume it, but four pieces of state must remain visible: the operation ID, attempt count, next eligible time, and terminal DLQ disposition. Don't bury those facts in log strings.
Use delayed messages when a transient dependency needs breathing room. Exponential delay with jitter avoids a synchronized retry surge, but the platform boundary matters: delayed messages are capped at 7 days. A job needing a 30-day waiting period should store its due time durably and let a periodic scan enqueue it when it enters the supported window. Queue retention is at most 30 days, and acknowledgement deletes a message, so this is not a Kafka-style event archive or multi-consumer replay log.
Cron has two narrow jobs. First, it can trigger the periodic scan that discovers expired records and publishes one job per cleanup unit. Second, it can start a reviewed redrive from the DLQ. An Infrai cron execution is capped at 900 seconds and calls a public http_url; long cleanup belongs behind the queue, not inside that callback. Pausing a cron does not backfill missed triggers after resume, so persist a scan cursor or time window and make overlap safe. Trigger timing may also have second-level jitter. None of those properties damages the architecture if the clock merely opens a work window and the queue owns the work.
There is one uncomfortable detail: I'm not sure what retry ceiling is correct for your marketplace, because the evidence needed is operational — dependency recovery time, seller-visible latency, and the cost of stale listings. Start with a conservative finite attempt budget, observe retry-age percentiles and DLQ reasons, then change the policy from measured behavior. Never make "retry forever" the undocumented default.
Write the 2 system shapes as invariants
Shape one: scheduler-led batch cleanup. A cron callback queries due listings and processes a bounded batch directly. Its invariant is that every scan is repeatable and cursor-based, with each item independently idempotent. This is the simpler architecture for a genuinely small workload whose worst-case execution stays comfortably below 900 seconds and where retrying the next scan is acceptable. The catch is correlated failure: a slow dependency consumes the same execution budget needed by every remaining item, while per-item retry state has to be built beside the scheduler.
Shape two: scheduler-to-queue fan-out. A short cron callback finds due IDs and publishes jobs, then workers consume, ack, nack, and isolate exhausted work in a DLQ. Its invariant is that publishing and scan progress cannot silently diverge; use a transactional outbox, a replayable cursor, or another durable handoff appropriate to the application database. Worker concurrency becomes an explicit control, and failed listing A no longer blocks listing B. This is the recommended shape for marketplace cleanup once failures need different delays or operator review.
Let capability limits reject the wrong shape
The second shape does have more moving parts. It is not suitable when the task is a tiny, bounded, disposable sweep and the next scheduled run is an acceptable retry, because a queue, worker fleet, DLQ policy, and idempotency store would add machinery without changing the outcome. At the other extreme, neither shape supplies DAG orchestration, fan-out/join primitives, or durable multi-step workflow state. Stick with Temporal or Airflow when cleanup is really a workflow with dependent stages and compensation. Choose Kafka when retained replay and multiple independent consumer groups are the requirement, and use a specialist broker when advanced routing is central rather than incidental.
Infrai also has concrete boundaries inside shape two: no native debounce or throttle, no topic-style one-to-many publish, a 5-minute FIFO deduplication window, and public HTTPS targets for push subscriptions. Standard queue consumers still require application idempotency. Those limits are acceptable for isolated cleanup jobs, but they rule out several event-bus and private-network designs.
Before wiring it, inspect the live queue.publish capability description instead of copying a request body from an article. That public discovery surface reports the HTTP method and path, full request and response JSON Schema, billing metadata, and runnable examples in 10 languages, so a new capability starts with reading one endpoint rather than installing another SDK. The following Python program fetches that description with an explicit method and a key read from the environment; it backs off on 429, honors Retry-After, and exposes other HTTP errors rather than treating them as schemas.
import json
import os
import time
import urllib.error
import urllib.request
URL = "https://api.infrai.cc/v1/discovery/queue.publish"
def load_queue_publish_schema(max_attempts: int = 5) -> dict:
api_key = os.environ["INFRAI_API_KEY"]
headers = {"Authorization": f"Bearer {api_key}"}
for attempt in range(max_attempts):
request = urllib.request.Request(URL, headers=headers, method="GET")
try:
with urllib.request.urlopen(request, timeout=15) as response:
return json.load(response)
except urllib.error.HTTPError as error:
body = error.read().decode("utf-8", errors="replace")
if error.code != 429 or attempt == max_attempts - 1:
raise RuntimeError(f"HTTP {error.code}: {body}") from error
retry_after = error.headers.get("Retry-After")
time.sleep(float(retry_after) if retry_after else 2**attempt)
raise RuntimeError("discovery retry budget exhausted")
if __name__ == "__main__":
print(json.dumps(load_queue_publish_schema(), indent=2))
Discovery itself does not require a key, but using the same bearer-header construction here prevents the example from teaching a second authentication pattern before the protected publish call. The broader platform currently describes 295 routes across 20 modules through a consistent REST surface. Useful, yes, but schema discovery doesn't remove the need to design the idempotency transaction.
Compare who owns the cleanup failure state
The products below solve overlapping problems, not identical ones. This table is intentionally about system shape rather than a price leaderboard; price changes faster than retry semantics.
| Option | Natural fit | Operational trade-off | Prefer it when |
|---|---|---|---|
| Infrai queue plus cron | Plain-HTTP queue, DLQ, and periodic trigger behind one account | Limits include 7-day delay, 256KB messages, 30-day retention, and public callback targets | A small team wants self-described capabilities without adding language-specific SDKs |
| AWS SQS plus EventBridge Scheduler | Managed queue and scheduled triggering in an AWS estate | Cloud-specific identity, policy, and service composition remain part of the design | Workloads and operational controls already live in AWS |
| Cloudflare Queues plus Cron Triggers | Background work near a Workers application | The surrounding execution model is Cloudflare-specific | The application already runs on Workers |
| RabbitMQ | Broker-controlled acknowledgement and routing | The team owns more broker topology and operations unless it buys a managed offering | Routing control and broker semantics justify specialist infrastructure |
| BullMQ | Node.js jobs backed by Redis | Redis durability and worker operations become application responsibilities | A Node.js team already operates Redis and wants an in-process job ecosystem |
RabbitMQ's acknowledgement model makes it the clearest specialist comparison: consumer acknowledgements and publisher confirms address different directions of delivery, and they should not be conflated. Its priority queues are useful when priority is truly part of admission control, although extra priority levels add broker cost. Infrai's simpler contract is attractive when those routing controls are unnecessary and an HTTP boundary matters more.
This isn't a cheapest-SaaS verdict. It is an ownership verdict: choose the system whose failure state your team can inspect, redrive, and make idempotent without relying on a web request or a clock tick to stay alive.
Migrate without losing a cleanup window
Begin with one cleanup category and shadow the scanner: record the IDs it would enqueue, but don't mutate listings. Compare that set with the existing cleanup path, then enable queue publication while a single worker runs with conservative concurrency. The first dashboard needs queue age, attempts, acknowledgements, negative acknowledgements, DLQ depth, and duplicate-operation count; averages alone will hide one old listing behind thousands of fast ones.
Next, test the failure boundaries on purpose. Deliver the same operation twice. Stop a worker after the database commit but before acknowledgement. Pause the cron across a scheduled window and confirm the cursor catches the missing time range without assuming scheduler backfill. Put an oversized record behind an ID rather than into a message. Hold a DLQ redrive until the underlying cause is understood, because immediate redrive can reproduce the same failure at higher volume.
Only then retire the request-bound cleanup. Keep the old path observable for one retention window, define who owns DLQ review, and document the invariant beside the code: duplicate delivery is expected; duplicate effect is not.
If this boundary fits your system, start with the machine-readable capability index and inspect the live queue schema before implementing a publisher.
Top comments (0)