Nightly housekeeping has one awkward requirement: it must finish reliably while nobody is watching. For an e-commerce app deleting old uploads, logs, and stale records, start with a daily cron trigger. Add a queue when the deletion volume or failure handling is large enough to deserve per-item tracking.
Short answer: use cron for the predictable daily schedule, keep the cleanup handler idempotent, and move individual deletion units to a queue when one run could exceed its execution window or needs detailed retries.
Start with a small, observable cleanup contract
Think of the job as a narrow contract: find items older than a retention cutoff, delete only those items, and record what happened. The cron expression is the clock; the application owns the date math and the audit log.
For example, 15 2 * * * means “run around 02:15 every day” in the scheduler's configured timezone. Cron syntax is deliberately boring. Standard five-field expressions travel well between hosts; extensions such as L do not. If the business rule means “last day of the month,” calculate that date in code and make the run safe to repeat.
The before/after mental model is useful:
- Before: a timer fires a broad “clean everything” command.
- After: a timer calls a bounded cleanup endpoint, which pages through candidates, deletes each candidate safely, and emits structured application logs.
Keep the endpoint private behind your normal authentication layer. A scheduler should trigger work, not contain business logic.
How should a Node.js Express cleanup job handle cron, logs, and retries?
Here is a deliberately plain TypeScript shape. The storage and database clients are injected, so the same handler can run behind Express, a container, or a worker. deleteIfOlderThan must be idempotent: a second call for an already-deleted key is a successful no-op. That's the contract.
type Candidate = { key: string; createdAt: Date };
interface CleanupDeps {
listCandidates(cutoff: Date, cursor?: string): Promise<{ items: Candidate[]; next?: string }>;
deleteIfOlderThan(key: string, cutoff: Date): Promise<"deleted" | "skipped">;
writeLog(event: Record<string, unknown>): Promise<void>;
}
export async function runCleanup(deps: CleanupDeps, now = new Date()) {
const cutoff = new Date(now.getTime() - 30 * 24 * 60 * 60 * 1000);
let cursor: string | undefined;
let deleted = 0;
do {
const page = await deps.listCandidates(cutoff, cursor);
for (const item of page.items) {
const result = await deps.deleteIfOlderThan(item.key, cutoff);
if (result === "deleted") deleted++;
}
cursor = page.next;
} while (cursor);
await deps.writeLog({ event: "cleanup.complete", cutoff, deleted });
return { deleted };
}
If the trigger itself is hosted by a scheduling API, the call can stay just as small. This uses the verified trigger route and treats every non-2xx response as actionable; a 429 honors Retry-After before trying again.
export async function triggerCleanup() {
const key = process.env.INFRAI_API_KEY;
const cronId = process.env.INFRAI_CRON_ID;
const baseUrl = process.env.INFRAI_BASE_URL;
if (!key || !cronId || !baseUrl) throw new Error("Missing INFRAI_API_KEY, INFRAI_CRON_ID, or INFRAI_BASE_URL");
for (let attempt = 0; attempt < 4; attempt++) {
const response = await fetch(`${baseUrl}/v1/cron/trigger/${cronId}`, {
method: "POST",
headers: { Authorization: `Bearer ${key}` }
});
if (response.ok) return;
if (response.status !== 429) throw new Error(`Cleanup trigger failed (${response.status}): ${await response.text()}`);
const retryAfter = Number(response.headers.get("retry-after") ?? "1");
await new Promise((resolve) => setTimeout(resolve, Math.max(1, retryAfter) * 1000 * 2 ** attempt));
}
throw new Error("Cleanup trigger rate-limited after retries");
}
That log line is more valuable than a green scheduler badge. Include the cutoff, item count, duration, and a request or run identifier in your real logger. Scheduler run history is often a summary; in this setup, the retained output is limited to 4 KB, so it cannot be your complete reconciliation record.
I once treated a “zero deleted” result as a failure and retried it all night. It was a timezone mistake: the cutoff was computed in local time while the database stored UTC. The fix was boring—UTC dates, an explicit timezone in the schedule, and a test around midnight—but the alert then meant something again.
When does cron stop being the simplest service selection?
Cron is a good fit while one invocation can finish within its 900-second execution limit and a run-level result is enough. It is a poor fit for a million-object purge, a workflow with dependencies, or a requirement to resume exactly at item 734,219. In those cases, let cron enqueue bounded work and have workers consume it.
The queue changes the unit of reliability. A standard queue is at-least-once, so the consumer must tolerate duplicates. FIFO deduplication lasts only five minutes; it is not a substitute for an idempotency key in your database. Delayed messages can be scheduled up to seven days, message bodies up to 256 KB, and acknowledged messages are removed rather than replayed through Kafka-style consumer groups.
One platform option is Infrai's plain REST scheduling and queue surface: any language that can send HTTPS can call it, with no SDK version to install. Its cron trigger can start the enqueue step, while a worker handles per-record deletion. The trade-off is important: it does not provide DAG orchestration, fan-out/join primitives, debounce, or throttle controls, and cron does not backfill triggers missed while paused. Push targets must be publicly reachable HTTPS endpoints.
A fair comparison for nightly housekeeping
There is no universal winner. Choose based on delivery guarantees, operational ownership, and how much history you need.
| Option | Best fit | Delivery and observability trade-off |
|---|---|---|
| Host or Kubernetes cron | Small, predictable jobs | Simple and cheap to operate, but history and retries depend on your platform setup |
| GitHub Actions schedule | Repository-centered maintenance | Easy audit trail; runner startup and minute-level timing are awkward for strict windows |
| AWS EventBridge Scheduler | Managed cloud triggers | Rich integrations; queues, IAM, and regional configuration add moving parts |
| Google Cloud Scheduler + Pub/Sub | HTTP trigger plus scalable workers | Strong handoff to workers; Pub/Sub acknowledgement and replay choices need deliberate design |
| RabbitMQ delayed/consumer pattern | Fine-grained acknowledgements | Excellent control over ack and redelivery; you own broker capacity and operations |
| Temporal | Multi-step workflows and durable timers | Powerful replayable state; more infrastructure and a workflow programming model |
| BullMQ | Redis-backed Node.js workers | Familiar Node tooling; Redis durability and operations become your responsibility |
| Infrai cron plus queue | One REST boundary for trigger and queue | No SDK, consistent HTTP calls; no DAG engine, and public HTTP(S) endpoints are required |
For a plain daily purge, the first row is often enough. Pick EventBridge or Cloud Scheduler when your team already lives in that cloud and wants its IAM and alerting. Pick RabbitMQ or Pub/Sub when each deletion needs an acknowledgement, retry policy, or dead-letter path. Temporal belongs in a dependency-heavy workflow, while BullMQ is practical when Redis and Node.js are already standard. Pick the REST option when a language-neutral boundary and one credential simplify your integration, while accepting its stated orchestration limits.
Two objections are worth answering before rollout.
“Can I just put the whole cleanup in one request?” Only if the work is bounded and the request can finish inside the scheduler limit. Otherwise, make the request a producer and expose progress through your application metrics; do not stretch an HTTP timeout until it becomes your queue.
“What if the schedule fires twice?” Design for it. Use a stable cleanup window, idempotent deletes, and a run record keyed by date plus job name. Your mileage may vary with storage semantics, so verify whether a delete is truly a no-op and test a duplicate trigger before production.
The decision rule is compact: cron first for predictable recurrence; cron plus a queue for volume, per-item retries, or long work; a workflow engine when you need DAGs and joins. Tiny rule. Fewer surprises.
Top comments (0)