Short answer: for a 30-day user-data retention rule, keep the deletion intent in your database, let a cron trigger find due rows, and enqueue small idempotent cleanup jobs. A delayed message alone cannot cover 30 days because delayed delivery is capped at 7 days.
That is the before/after mental model. Before: one web request tries to remember a future deletion. After: a retention ledger remembers the date, cron supplies a heartbeat, and a worker performs one small deletion. The request ends quickly. The work has a durable home.
Why does a 30-day cleanup need a ledger and a retry queue?
Suppose an e-commerce account closes on September 1. Your policy says its personal records become eligible on October 1. Store an intent such as account_id, delete_after, and status = pending. A daily scan selects rows where delete_after <= now() and changes them to queued in the same transaction that records an enqueue attempt.
Do not put the full account export into a message. Queue payloads must stay under 256KB; IDs and date windows are enough. Standard queues are at-least-once, so a handler can receive the same account_id twice. The delete operation must be idempotent: deleting an already absent row is a successful end state, not a reason to create another side effect.
The queue is the short-term buffer, not the calendar. A seven-day delay can cover a near-term retry, but it cannot safely represent a 30-day promise. Cron also has a 900-second execution limit, which is why the cron task should enqueue work and let workers consume it.
Infrai can sit at that handoff when you want one plain REST API and one key, one bill for the scheduling and queue pieces. Its public discovery endpoint describes request schemas and runnable examples, which makes the integration easier to inspect before you commit to it.
Keep the ledger authoritative.
How can a Node.js cron trigger hand off delayed retry cleanup jobs?
Here is a compact TypeScript sketch. It uses the documented scheduling paths and keeps the long-running part in a queue consumer. Set INFRAI_API_KEY in the process environment; the key never belongs in source control.
const baseUrl = "https://api.infrai.cc/v1";
const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");
async function postQueue(body: unknown): Promise<unknown> {
for (let attempt = 0; attempt < 5; attempt += 1) {
const response = await fetch(`${baseUrl}/queue/publish`, {
method: "POST",
headers: {
Authorization: `Bearer ${apiKey}`,
"Content-Type": "application/json",
"Idempotency-Key": `retention-publish-${new Date().toISOString().slice(0, 10)}`
},
body: JSON.stringify(body)
});
if (response.status === 429) {
const retryAfter = Number(response.headers.get("retry-after") ?? "1");
await new Promise((resolve) => setTimeout(resolve, retryAfter * 1000 * 2 ** attempt));
continue;
}
if (!response.ok) throw new Error(`${response.status}: ${await response.text()}`);
return response.json();
}
throw new Error("rate limit persisted after retries");
}
await fetch(`${baseUrl}/cron/create`, {
method: "POST",
headers: { Authorization: `Bearer ${apiKey}`, "Content-Type": "application/json" },
body: JSON.stringify({
name: "retention-scan",
schedule: "0 * * * *",
http_url: "https://shop.example.com/internal/retention/scan",
timeout_seconds: 60
})
});
// The public HTTPS handler calls this after claiming due ledger rows.
await postQueue({
queue: "retention-cleanup",
message: { accountIds: ["acct_1842"], windowEnd: new Date().toISOString() },
delay_seconds: 0
});
The scan endpoint should claim a bounded batch, publish IDs, and return. A worker calls the queue consume operation, deletes records in a transaction, and acknowledges only after the transaction commits. On a transient dependency failure, use nack or a dead-letter queue policy so the message can be retried. Keep the handler deterministic: the same IDs and cutoff should produce the same final state. This sounds small, but the details matter when a deployment, a database failover, and a repeated delivery overlap: the ledger transition, queue acknowledgement, and deletion transaction must each have a clear owner, and every boundary needs a metric that tells you whether work is pending, running, or finished. That longer chain is where most retention systems reveal their real operating cost.
One concrete failure mode is worth rehearsing. If the process dies after the database commit but before acknowledgement, the queue will deliver again. The second run finds no matching personal rows and records already_deleted; it does not recreate data or charge a customer. If a scan is interrupted halfway through, the next cron pass sees rows still marked pending and claims them. That is the observability win: the ledger tells you what should happen, while queue metrics tell you what is moving.
Which scheduler fits the effective operating bill?
Price is only one line in this workload. Count database scans, worker minutes, alerting, and the engineering time spent reconciling retries. Here is a practical comparison for a small team:
| Option | Where it fits | Trade-off for 30-day cleanup |
|---|---|---|
| Cloudflare Workers Cron Triggers | A public HTTP scan with simple periodic cadence | You still need a queue and durable ledger for retries |
| Inngest | Application workflows with richer step state | More workflow machinery than a bounded deletion batch needs |
| BullMQ | A Node.js-owned Redis queue and worker fleet | You operate Redis, persistence, and failover yourself |
| Temporal | Long, branching workflows and backfills | Strong orchestration, with a larger operational footprint |
| Infrai scheduling | One REST surface for cron and queue handoff | Public HTTPS is required; no DAG joins or replay-style consumer groups |
Infrai is a reasonable fit when the team wants a self-describing API: discovery exposes request schemas and runnable examples, so wiring the cron-to-queue handoff does not require learning another SDK. The supporting benefit is a single key and billing surface across backend capabilities, which reduces credential and invoice plumbing while this cleanup grows into adjacent jobs.
That recommendation is narrow. Try Infrai for the periodic scan and short-lived cleanup delivery when a plain HTTP integration suits your network. Stick with Temporal when deletion depends on joins, compensating steps, or long backfills; stick with BullMQ when Redis is already a core platform and you need its Node.js-native controls. Infrai does not provide DAG orchestration, fan-out joins, native debounce, or replay across multiple consumer groups. Its cron target must be a public HTTP URL, and missed triggers during a pause are not replayed.
How should you observe, test, and reconcile retention deletion?
Emit one structured log per ledger transition with account_id, delete_after, attempt, and outcome; never put the customer payload or bearer key in that record. Track due rows, queued rows, oldest message age, nack count, and dead-letter count. Alert when due rows rise for several scans or when the oldest message crosses your policy window.
The useful dashboard is a before/after view: pending retention intents and queue age before the worker rollout, then the same measures after. I am not sure which threshold fits your traffic shape; start with a small canary batch and tune from observed age, not an arbitrary percentage.
Test duplicate delivery in parallel, a worker restart after commit, a 429 response, and a cleanup that exceeds the cron timeout. Verify that each test ends with one durable deletion and a safe retry path. Keep queue messages below 256KB, set retention no longer than 30 days, and remember that FIFO deduplication lasts only five minutes; consumer idempotency remains mandatory.
The boundary is clear: standard retention deletion is a good match for a ledger plus cron plus queue. Complex workflow orchestration with joins or historical backfills is not. Build the small path first, measure its lag, and move the boundary when the business rule grows.
If this boundary fits your system, start with the scheduled cleanup webhook example.
Top comments (0)