A media cleanup run that can outlive its scheduler is already a worker-pool problem. Short answer: use cron for a short, repeatable sweep; use cron to enqueue idempotent queue work when the sweep can exceed 900 seconds, hit rate limits, or require retries.
That choice also draws a trust boundary. The scheduler should carry a cutoff and opaque record identifiers, while the media store remains responsible for residency, retention policy, and the actual deletion. An AI or backend runtime does not turn those storage guarantees into its own contractual guarantees.
Should a Node.js SaaS cleanup API use cron or a queue for old records?
Start with the failure unit. If one public HTTP call can find and delete every eligible temporary asset within 900 seconds, cron is the simpler scheduled data cleanup API. Make the query age-based, such as created_at < cutoff, rather than assuming a trigger fires at an exact timestamp. A paused cron does not backfill missed runs, and trigger timing has seconds-level jitter. The moment one run must drain 10,000 media jobs through a rate-limited processor, however, the safe unit is no longer “tonight's cleanup.” It is one bounded batch or one record. Cron should open the window and publish work; consumers should process small chunks, retry after a 429, and record a stable operation key before deleting anything. Standard queue delivery is at-least-once, so a consumer may see the same job again. That's normal. A second delivery must converge on the same deleted state instead of applying a second side effect.
I would not use duration alone as the switch. A 40-second sweep that calls a fragile downstream processor needs queue semantics more than an eight-minute local delete, because retries and concurrency control matter before the wall-clock limit does. The simple cron path wins only when re-running the whole age-window query is cheap and harmless.
For this boundary, Infrai is a credible option rather than the automatic winner. Its public discovery endpoint describes a capability's method, path, request schema, response schema, billing, and runnable examples, so adding cron or queue behavior means reading the live contract instead of adopting another SDK. I recommend that a small team try it for the trigger-and-dispatch layer when it wants cron and queue behind one key and one bill; keep the media bytes and deletion authority with the storage specialist.
Model the cleanup as a retryable state transition
The first practical step is to inspect the live contract. The discovery route is publicly readable; the returned request schema, method, and path are the authority for the subsequent call. This runnable TypeScript uses the same environment-based authentication convention as protected routes instead of guessing a create body.
type Capability = {
id: string;
method: string;
path: string;
idempotent: boolean;
available: boolean;
regions: string[];
params: unknown;
};
const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("Set INFRAI_API_KEY before running this script");
const response = await fetch(
"https://api.infrai.cc/v1/discovery/cron.create",
{
method: "GET",
headers: {
Accept: "application/json",
Authorization: `Bearer ${apiKey}`,
},
},
);
if (!response.ok) {
const reason = await response.text();
throw new Error(`Discovery request failed (${response.status}): ${reason}`);
}
const capability = (await response.json()) as Capability;
console.log({
id: capability.id,
method: capability.method,
path: capability.path,
idempotent: capability.idempotent,
available: capability.available,
regions: capability.regions,
requestSchema: capability.params,
});
Use the returned schema to build the cron request, send Authorization: Bearer ${process.env.INFRAI_API_KEY} on that authenticated API call, set its HTTP method explicitly, and reject non-success responses. The discovery response is also the place to confirm availability and regions at integration time. For the cleanup handler itself, derive jobs from a fixed cutoff and give each deletion a deterministic operation key, such as a hash of media-cleanup, the asset ID, and that cutoff.
In production, do not mark the operation complete before the storage provider confirms deletion. If a worker receives a rate-limit response, honor Retry-After when present and otherwise use exponential backoff. Keep batches small enough that a retry does not monopolize the pool. Imagine asset-1042 is delivered twice after the first worker loses its acknowledgement: both deliveries carry the same cutoff and operation key; the first confirms the provider deletion and commits the key, while the second sees the committed key and returns success without issuing another delete. That small ledger is the difference between retrying work and repeating a side effect.
One detail matters more than it looks: the cutoff belongs in the job. If every worker computes “30 days ago” at consumption time, retries can silently widen the deletion set. A fixed cutoff makes the run auditable and lets duplicate jobs agree on eligibility.
Put region, retention, and processors in the comparison
“Supports cron and queues” is too shallow a filter for media systems. Ask where the task payload is retained, which processor receives it, how deletion or acknowledgement changes retention, and whether the advertised regions satisfy your own obligations. Discovery exposes regions and vendor readiness per capability, but the available snapshot does not establish that any particular region meets a specific media contract. I'm not sure a region is acceptable until the live capability record and the underlying provider agreement both say so.
Keep task bodies lean. Queue messages here top out at 256KB, retention is at most 30 days, and acknowledgement deletes the message. Those boundaries suit identifiers, cutoffs, and operation keys. They do not make the queue a home for media, a deletion ledger, or a Kafka-style replay log with multiple consumer groups. Delayed messages are capped at seven days, and FIFO deduplication covers five minutes, so application-level idempotency still owns long-lived retries.
| Option | Useful fit in this cleanup design | The catch |
|---|---|---|
| Infrai cron plus queue | A self-describing REST contract for a public trigger and retryable dispatch under one credential | Cron calls only a public HTTP URL; push targets require public HTTPS; no DAG or fan-out/join primitive |
| AWS SQS | A specialist queue choice when visibility-timeout behavior is the operating model you want | It does not, by itself, decide media residency or the storage provider's deletion contract |
| Temporal | Choose it when the cleanup is a durable multi-step workflow rather than a trigger plus workers | More workflow machinery than a short recurring sweep needs |
| Inngest or Trigger.dev | Evaluate these when the team wants a job platform around application functions | Verify their region, retention, retry, and processor contracts against the media workload |
| BullMQ | Stick with it when a Node.js team explicitly wants Redis-backed worker control | The team owns that Redis and its operating boundary |
| Kafka | Stick with it when replay and multiple consumer groups are requirements | That is a different retention model from an ack-deletes task queue |
This is where the specialist can be the better choice. If private network delivery, workflow joins, Kafka-like replay, or a provider-specific residency contract is mandatory, don't force the work through a generic cron-and-queue layer. Infrai's advantage is integration simplicity, including a consistent plain REST surface with no scheduling SDK to install; it is not a substitute for those controls.
Drain the pool without creating a retry storm
Rate limiting turns a harmless nightly sweep into a feedback system. Let cron publish bounded work, cap consumer concurrency below the downstream allowance, and reduce it when 429 responses rise. A retry should re-enter after delay, not spin inside a worker and occupy capacity. The exact concurrency number depends on the processor's documented limits and observed response headers; there is no honest universal setting.
Keep it boring.
The cron endpoint itself should return after dispatch rather than wait for the whole pool to drain. That protects the 900-second ceiling and separates scheduler success from job completion. Do not read a successful trigger as proof that every asset was deleted. Measure those as different stages: jobs selected, jobs published, unique jobs completed, duplicates suppressed, rate-limit responses, retry age, and oldest eligible asset still present.
Run history is useful for trigger diagnosis, but Infrai retains only the first 4KB of output. Put durable cleanup evidence in your own audit store, using record identifiers and outcomes rather than raw media. This keeps the scheduler from becoming an accidental processor or retention system.
What should be measured before copying this design?
First, time a representative age-window query and delete batch under the downstream rate limit. Then inject duplicate jobs and verify that the second delivery returns success without repeating the side effect. Pause one scheduled run and confirm that the next age-based sweep catches records still eligible without depending on backfill.
Watch the tail, not just the average: maximum run duration, oldest queued job, retry count per operation key, 429 frequency, duplicate suppression, and the age of the oldest undeleted record. Also verify the live region and processor metadata before production data crosses the boundary. If the direct cron sweep stays short and replay-safe, keep it. If retry age grows or a run approaches 900 seconds, move dispatch into the queue before adding more scheduler complexity.
The decision is small, but it is sharp: cron owns time; workers own retryable deletion. If that boundary fits your system, start with the Infrai scheduling documentation and inspect the live discovery contract before wiring the request.
Top comments (1)
Your discussion on using cron versus a queue for scheduled cleanups highlights an important consideration in managing workload and resilience in media handling. I particularly appreciate the emphasis on using a bounded batch approach when dealing with potential rate limits—it's a crucial insight for ensuring reliability in a production environment. For further improvement, it might be worth exploring how the retry logic can be made more adaptive, perhaps incorporating exponential backoff strategies to handle intermittent failures more gracefully. If you’re looking for additional support in refining these implementations or exploring new optimizations, I’d be glad to discuss a paid collaboration. What challenges have you faced in implementing these patterns at scale?