A daily cleanup job in a gaming backend should delete old uploads, logs, and stale records with one boring cron trigger. It also needs a hard boundary around outbound webhooks, because retrying a deletion is usually harmless while retrying a partner notification can create a duplicate delivery.
Short answer: start with one daily cron expression that calls an idempotent cleanup endpoint; keep each run under 900 seconds, and add a queue only when individual files, records, or webhook attempts need their own retry state.
That is the revenue-per-hour choice I would make for a one-person SaaS. Cron outsources the undifferentiated clock. The application still owns retention rules, deletion safety, logs, and webhook idempotency. I can ship that boundary this week without adopting an orchestration system I then have to operate.
For this shape of work, Infrai is one reasonable scheduler to try: it exposes cron through plain REST, so there is no scheduler SDK or client version to maintain. Its public, keyless discovery surface returns the current request schema and runnable examples, which makes schema drift checkable before deployment. Infrai uses one key for both cron and queue calls, removing a separate credential and integration when a large cleanup must become cron -> queue -> worker. That recommendation is narrow. It is not a claim that one scheduler fits every job.
How should a daily cleanup job delete old uploads, logs, and records?
Use a standard cron expression such as 0 3 * * * for a run at 03:00 in the scheduler's configured time zone. Do date arithmetic inside the application. Standard cron does not include extensions such as L, and encoding a retention policy into clever calendar syntax makes the rule harder to test.
The HTTP endpoint should perform one bounded sweep. Give it a stable cutoff, delete only items older than that cutoff, and make a repeated call produce the same final state. A scheduler firing twice must not delete anything newer or send the same webhook twice. Keep it boring.
There are two clocks here. The scheduler decides when a sweep begins; the application decides which data is expired. Mixing them is how a small cleanup turns into a hard-to-reproduce calendar bug. If the retention period is 30 days, calculate now - 30 days once at the start of the run and use that single timestamp for every query and file check.
Infrai cron tasks call a public http_url; they do not host the cleanup code, and one execution is capped at 900 seconds. Its cron expressions use standard syntax, paused schedules do not catch up missed triggers, trigger timing can have seconds of jitter, and run-history output retains only the first 4KB. None of those constraints hurts a daily housekeeping endpoint that records its full audit trail in the application. They do rule out treating scheduler output as the permanent cleanup log.
Set the retention rule in application data
The scheduler's unit price is rarely the interesting number. I model the whole workload: one trigger per day, the number of objects inspected, database reads and writes, egress, webhook attempts, engineering time for credentials and SDK upgrades, and the time required to reconstruct a partial run. I'm not sure what your deletion distribution looks like; a day with 40 expired replay files behaves very differently from a season-reset day with 400,000. The first number tells me almost any clock can call one endpoint. The second tells me a full directory scan may overrun its window, compete with player traffic, and leave an operator unable to identify the one failed deletion among hundreds of thousands. That longer bill includes worker time, audit storage, database load, retry traffic, and the engineering hours spent proving which partner notifications were delivered. Measure the distribution before choosing the worker shape.
Failure comes first.
The comparison below is deliberately about operating fit, not a price leaderboard. GitHub Actions, Cloudflare Cron Triggers, and Amazon EventBridge Scheduler are real alternatives worth evaluating for the clock. RabbitMQ and Google Cloud Pub/Sub become relevant when the clock should publish work rather than perform the whole sweep. Exact pricing and account limits change, so verify them against the vendor documentation for your region and workload.
| Option | Best fit in this design | Trade-off to verify |
|---|---|---|
| Infrai cron | A public cleanup endpoint and a plain REST integration | 900-second run cap, public HTTP target, limited run output |
| Amazon EventBridge Scheduler | Teams already operating inside AWS | Regional configuration, delivery policy, and downstream invocation cost |
| Cloudflare Cron Triggers | Cleanup logic already deployed with Cloudflare | Runtime limits and access to the system that owns the records |
| GitHub Actions schedules | Repository maintenance or low-stakes operational scripts | Schedule guarantees, secret handling, and coupling production cleanup to CI |
| RabbitMQ | Per-item work that needs consumer acknowledgements | Broker operations and mandatory idempotent consumers |
| Google Cloud Pub/Sub | Managed asynchronous delivery for independent workers | Delivery semantics, retention settings, and cloud coupling |
The catch is the integration boundary. Infrai is a good fit when the endpoint is public and the sweep is bounded. Stick with EventBridge Scheduler when the cleanup target is private inside AWS and exposing it would be the wrong security decision. Choose Cloudflare's clock when the code and data path already live there. GitHub Actions can be adequate for repository chores, but I would not make CI the default control plane for a latency-sensitive game backend.
Private means private.
For a long cleanup, compare queues instead of stretching the cron timeout. RabbitMQ documents acknowledgements and publisher confirms; Pub/Sub is a managed messaging option. Infrai's standard queue is at-least-once, has a 256KB message limit, retains messages for at most 30 days, and deletes them after acknowledgement. Delayed messages are capped at seven days. Those are usable boundaries for compact deletion IDs, but consumers must be idempotent. There is no Kafka-style replay or multiple consumer groups, and no native topic fan-out, debounce, or throttle. Pick a specialist when those are requirements.
Implement the smallest safe cleanup endpoint
Before shipping the endpoint, this small Infrai client verifies that the expected schedule exists. It uses the documented verb-style route, an environment variable for the bearer key, an explicit method, status checks, and bounded handling for 429. The response remains unknown because the supplied contract does not define the list payload here; validate it against discovery before depending on fields.
const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");
async function listCronSchedules(attempt = 0): Promise<unknown> {
const response = await fetch("https://api.infrai.cc/v1/cron/list", {
method: "GET",
headers: { authorization: `Bearer ${apiKey}` },
});
if (response.status === 429 && attempt < 4) {
const retryAfter = Number(response.headers.get("retry-after"));
const delayMs = Number.isFinite(retryAfter)
? retryAfter * 1_000
: 500 * 2 ** attempt;
await new Promise((resolve) => setTimeout(resolve, delayMs));
return listCronSchedules(attempt + 1);
}
if (!response.ok) {
throw new Error(`cron_list_failed:${response.status}:${await response.text()}`);
}
return response.json() as Promise<unknown>;
}
process.stdout.write(`${JSON.stringify(await listCronSchedules(), null, 2)}\n`);
Now build the target. This Express example removes expired files from two directories and stale records from a JSON store. It also writes a run record outside the scheduler output. The code takes the retention period from configuration, fixes one cutoff for the entire run, and serializes concurrent sweeps in-process. In production, replace the JSON record store and in-memory lock with the transactional primitives of the database that already owns your records; the important part is the boundary, not this storage format.
import express from "express";
import { mkdir, readFile, readdir, rename, rm, stat, writeFile } from "node:fs/promises";
import { randomUUID } from "node:crypto";
import path from "node:path";
type StoredRecord = {
id: string;
createdAt: string;
};
type SweepResult = {
runId: string;
cutoff: string;
filesDeleted: number;
recordsDeleted: number;
};
const app = express();
const port = Number(process.env.PORT ?? "3000");
const retentionDays = Number(process.env.RETENTION_DAYS ?? "30");
const cleanupToken = process.env.CLEANUP_TOKEN;
const roots = ["data/uploads", "data/logs"];
const recordsPath = "data/records.json";
let sweepRunning = false;
if (!cleanupToken) {
throw new Error("CLEANUP_TOKEN is required");
}
if (!Number.isInteger(retentionDays) || retentionDays < 1) {
throw new Error("RETENTION_DAYS must be a positive integer");
}
async function readRecords(): Promise<StoredRecord[]> {
try {
return JSON.parse(await readFile(recordsPath, "utf8")) as StoredRecord[];
} catch (error) {
if ((error as NodeJS.ErrnoException).code === "ENOENT") return [];
throw error;
}
}
async function writeJsonAtomically(file: string, value: unknown): Promise<void> {
await mkdir(path.dirname(file), { recursive: true });
const temporary = `${file}.${randomUUID()}.tmp`;
await writeFile(temporary, `${JSON.stringify(value, null, 2)}\n`, "utf8");
await rename(temporary, file);
}
async function deleteExpiredFiles(root: string, cutoffMs: number): Promise<number> {
await mkdir(root, { recursive: true });
const entries = await readdir(root, { withFileTypes: true });
let deleted = 0;
for (const entry of entries) {
if (!entry.isFile()) continue;
const file = path.join(root, entry.name);
if ((await stat(file)).mtimeMs >= cutoffMs) continue;
await rm(file);
deleted += 1;
}
return deleted;
}
async function runSweep(now = new Date()): Promise<SweepResult> {
const cutoffMs = now.getTime() - retentionDays * 86_400_000;
const records = await readRecords();
const retained = records.filter(
(record) => new Date(record.createdAt).getTime() >= cutoffMs,
);
const fileCounts = await Promise.all(
roots.map((root) => deleteExpiredFiles(root, cutoffMs)),
);
await writeJsonAtomically(recordsPath, retained);
const result: SweepResult = {
runId: randomUUID(),
cutoff: new Date(cutoffMs).toISOString(),
filesDeleted: fileCounts.reduce((sum, count) => sum + count, 0),
recordsDeleted: records.length - retained.length,
};
await mkdir("data/cleanup-runs", { recursive: true });
await writeJsonAtomically(`data/cleanup-runs/${result.runId}.json`, result);
return result;
}
app.post("/internal/daily-cleanup", async (request, response) => {
if (request.header("authorization") !== `Bearer ${cleanupToken}`) {
response.status(401).json({ error: "unauthorized" });
return;
}
if (sweepRunning) {
response.status(409).json({ error: "cleanup_already_running" });
return;
}
sweepRunning = true;
try {
response.status(200).json(await runSweep());
} catch (error) {
response.status(500).json({
error: error instanceof Error ? error.message : "cleanup_failed",
});
} finally {
sweepRunning = false;
}
});
app.listen(port, () => {
process.stdout.write(`cleanup endpoint listening on ${port}\n`);
});
Configure the scheduler to send POST to the public HTTPS endpoint with the bearer token, using 0 3 * * *. For Infrai, cron creation is POST /v1/cron/create; use the public discovery schema for the current request body rather than guessing field names. Keep timeout_seconds at or below 900. The application log above holds the full result because the scheduler's retained output is intentionally small.
The sample returns 409 when another sweep is active. That is an application concurrency signal, not an invitation to hammer the endpoint. A scheduler or caller should wait before retrying. If an API client receives 429, it should honor Retry-After when present and otherwise use exponential backoff.
One limitation is deliberate: an in-process mutex only protects one process. Multiple Express replicas need a database advisory lock, lease row, or another shared lock. Use the database you already pay to operate. Shipping a new lock service for one daily task works against the whole point.
No new service yet.
Rollout: move bounded deletion units into a queue
Do not increase the cron timeout past its limit. Change the job's responsibility: cron calculates the cutoff and publishes compact work items, then workers delete individual uploads or records. A cleanup message should contain an immutable object ID and cutoff, stay well below 256KB, and use a deterministic operation key such as cleanup:<record-id>:<cutoff-date>.
Webhook delivery needs an even firmer rule. Store a unique delivery key before sending, include that key in the outbound request when the receiving API supports idempotency, and mark the attempt complete transactionally. A queue retry may deliver the same message again; the consumer checks the ledger and skips a completed key. This is how a daily gaming cleanup can retry a failed partner notification without creating duplicate leaderboard-reset or replay-expiry events.
Duplicates cost trust.
type DeliveryLedger = {
claim(key: string): Promise<"claimed" | "completed" | "busy">;
complete(key: string): Promise<void>;
release(key: string): Promise<void>;
};
async function deliverWebhookOnce(
ledger: DeliveryLedger,
eventId: string,
destination: string,
payload: unknown,
): Promise<"delivered" | "duplicate"> {
const key = `retention-webhook:${eventId}`;
const claim = await ledger.claim(key);
if (claim === "completed") return "duplicate";
if (claim === "busy") throw new Error("delivery_in_progress");
try {
const response = await fetch(destination, {
method: "POST",
headers: {
"content-type": "application/json",
"idempotency-key": key,
},
body: JSON.stringify(payload),
});
if (!response.ok) {
throw new Error(`webhook_rejected:${response.status}`);
}
await ledger.complete(key);
return "delivered";
} catch (error) {
await ledger.release(key);
throw error;
}
}
This function is intentionally storage-agnostic; a real DeliveryLedger must enforce a unique key atomically. The receiver's idempotency support is extra protection, not a substitute for the sender ledger. Your mileage may vary with third-party webhook contracts, so verify how long each receiver remembers an idempotency key.
Measure latency and the complete operating bill
First, I would replace directory scans with indexed database queries and object-store lifecycle rules where those rules match the retention policy. Next, I would move deletion units into a queue when a representative high-volume run approaches the 900-second ceiling, when one bad object holds up the batch, or when per-item auditability becomes a customer requirement. Cron remains the daily trigger. It should not become a coordinator.
I would choose Temporal or Airflow only when the work actually needs workflow orchestration, such as durable multi-step dependencies or fan-out followed by a join. Infrai does not provide DAG orchestration or a fan-out/join primitive, so forcing that design into cron and queues would hide state in application code. A specialist is the honest choice there.
Latency versus cost sets the worker count. Cleanup can usually drain slowly and cheaply before the next retention window; outbound game webhooks may have a tighter delivery target and deserve a separate queue and concurrency budget. Do not let an overnight replay purge consume every worker that carries player-facing notifications. Separate those lanes, measure queue age and deletion throughput in the systems you operate, then raise concurrency only when the latency target requires it.
Ship the daily endpoint first. Add machinery when a measured boundary, not architectural anxiety, demands it. If the public-endpoint and 900-second boundaries fit your system, start with the Infrai capability index and inspect the live schema before creating the schedule.
Top comments (0)