The trade-off in a scheduled data cleanup is latency against cost, and for this class of work the latency side is nearly free to give away: nothing downstream notices whether the job fires at 02:00:00 or 02:00:07. So use the cheap scheduler — the one that reliably calls a public HTTP endpoint — and spend the engineering budget on the worker that does the deleting. For app-level cleanup a hosted cron plus a queue tends to fit better than a CI scheduler or a serverless cron trigger on its own, because the deletes then run inside the service that already owns the transaction boundaries.
The deadline you don't own sets the whole design
Take an edtech platform that sells seat licences to school districts. A trial lapses, and two scheduled things have to happen. The renewal reminder has to wait for the district's purchase-order deadline — send it a week early and it sits unread until the budget holder has already moved on, send it late and the seats go dark mid-semester. Then, thirty days after the trial ends, the trial roster has to go: student names, class lists, assignment attempts.
Now put the two clocks side by side. The business deadline is a date, sometimes a date plus a local hour. The scheduler's accuracy is measured in seconds. That's three or four orders of magnitude of slack, which means trigger precision is worth close to nothing here and predictable cost is worth a lot.
Jitter isn't the problem.
The problems are missed windows and partial deletes. A scheduler that silently skips a night leaves a batch of rosters past their retention promise, and an audit doesn't care that the trigger was free. A cleanup that dies halfway through a 400,000-row delete leaves orphaned assignment attempts pointing at rosters that no longer exist. Both of those are worked out in your application, not in the scheduler's dashboard, and that's the actual reason CI-style schedulers make an awkward home for this: they're built to run repo workflows, so you end up shipping database credentials and a migration-shaped script into a runner that has no idea what a partially completed purge means.
One more property decides the design. A paused schedule does not replay the triggers it missed while it was paused, on any of these platforms as far as I can tell. So the handler can never treat "this tick happened" as the unit of work. It has to derive due work from durable state — a po_deadline column, a trial_ended_at column — and be safe to run twice in a row. Once you accept that, a missed night costs you one day of delay rather than a data-retention incident, and you can pick your scheduler on cost with a clear conscience.
What falls out: a thin trigger, chunked work behind a queue
The nightly tick should do bookkeeping and nothing else. It queries for work, publishes small messages, returns. Workers do the deleting and the sending, at their own pace, with their own retries.
Start with the schedule itself. The handler below only enqueues, so a 60-second budget is generous — worth setting deliberately, because the platform ceiling for a single cron execution is 900 seconds and anything that could approach that belongs in a worker instead.
import json
import os
import time
import urllib.error
import urllib.request
API = os.environ["INFRAI_API_ORIGIN"].rstrip("/") # API root, from the provider's docs
KEY = os.environ["INFRAI_API_KEY"] # ifr_..., never inline the literal
CLEANUP_URL = os.environ["CLEANUP_ENDPOINT"] # public https target for the nightly tick
QUEUE = "district-lifecycle"
MAX_DELAY = 604_800 # 7 days is the delayed-message ceiling
def post(url, payload, idempotency_key, attempts=5):
data = json.dumps(payload).encode("utf-8")
for attempt in range(attempts):
request = urllib.request.Request(
url,
data=data,
method="POST",
headers={
"Authorization": f"Bearer {KEY}",
"Content-Type": "application/json",
"Idempotency-Key": idempotency_key,
},
)
try:
with urllib.request.urlopen(request, timeout=20) as response:
return json.loads(response.read().decode("utf-8") or "{}")
except urllib.error.HTTPError as error:
body = error.read().decode("utf-8")
if error.code != 429 or attempt == attempts - 1:
raise RuntimeError(f"{url} -> {error.code}: {body}") from error
retry_after = error.headers.get("Retry-After")
time.sleep(float(retry_after) if retry_after else min(2 ** attempt, 30))
raise RuntimeError(f"{url} -> retries exhausted")
def install_nightly_tick():
return post(
f"{API}/v1/cron/create",
{
"task": CLEANUP_URL,
"cron_expr": "20 2 * * *",
"timezone": "America/Chicago",
"timeout_seconds": 60,
"overlap_policy": "skip",
},
idempotency_key="cron:district-lifecycle:v1",
)
The handler is the interesting half. It reads two tables you already have, publishes one message per reminder and one per delete chunk, and writes down what it queued so a second run that night is a no-op.
import sqlite3
from datetime import datetime, timedelta, timezone
DB = os.environ.get("LIFECYCLE_DB", "lifecycle.db")
def lapsed_roster_chunks(db, cutoff, size=500):
rows = db.execute(
"SELECT id FROM roster_rows WHERE trial_ended_at <= ? AND purged_at IS NULL LIMIT ?",
(cutoff.isoformat(), size * 40),
).fetchall()
ids = [row[0] for row in rows]
return [ids[i:i + size] for i in range(0, len(ids), size)]
def nightly():
now = datetime.now(timezone.utc)
db = sqlite3.connect(DB)
queued = 0
due = db.execute(
"""
SELECT district_id, po_deadline FROM trials
WHERE reminder_queued_at IS NULL AND po_deadline > ?
ORDER BY po_deadline LIMIT 200
""",
(now.isoformat(),),
).fetchall()
for district_id, deadline in due:
wait = int((datetime.fromisoformat(deadline) - now).total_seconds())
if wait > MAX_DELAY:
continue # further out than the delay ceiling; a later tick will take it
post(
f"{API}/v1/queue/publish",
{
"queue": QUEUE,
"payload": {"kind": "renewal_reminder", "district_id": district_id,
"deadline": deadline},
"delay_seconds": max(0, wait),
},
idempotency_key=f"reminder:{district_id}:{deadline[:10]}",
)
db.execute(
"UPDATE trials SET reminder_queued_at = ? WHERE district_id = ?",
(now.isoformat(), district_id),
)
queued += 1
for chunk in lapsed_roster_chunks(db, now - timedelta(days=30)):
post(
f"{API}/v1/queue/publish",
{"queue": QUEUE, "payload": {"kind": "purge_roster", "row_ids": chunk}},
idempotency_key=f"purge:{chunk[0]}:{chunk[-1]}",
)
queued += 1
db.commit()
return {"queued": queued}
Three details in there are load-bearing. Chunks carry row ids, never row bodies, which keeps each message far under the 256KB payload cap and keeps a retry cheap. The reminder rides on the queue's own delay instead of a second cron job, which is why the deadline arithmetic matters: delayed delivery is capped at seven days, so a purchase-order date eleven days out is deliberately skipped tonight and picked up by a later tick. And every publish carries an idempotency key derived from business identity — district plus deadline date, or the first and last id in the chunk — because standard queues are at-least-once and the FIFO deduplication window is only five minutes. That window protects a retry from an immediate double-publish. It does nothing for a redelivery an hour later, so the worker has to be safe under duplicates too: delete by id set, treat zero affected rows as success, and guard the reminder send on a unique (district_id, deadline) row in your own table.
That last part is where I've seen the most damage in email and SMS flows generally. A duplicate purge is invisible. A duplicate renewal notice to a superintendent is a support ticket and a deliverability complaint, and complaint rate is the number you can't buy back.
Which cron scheduler should run a nightly data cleanup — GitHub Actions, Cloudflare Workers, or EventBridge?
Compare them on where the work actually executes, not on cron syntax. They all take five fields.
| Option | How the schedule fires | Where the deleting runs | Main limit for this job |
|---|---|---|---|
GitHub Actions schedule
|
Workflow on the default branch | Inside a CI runner | Documented as delayable under load, and schedules get disabled on inactive repos; needs DB credentials in CI |
| Cloudflare Workers Cron Triggers | Scheduled handler in the Workers runtime | In the edge runtime | Your driver and transaction style have to fit that runtime and its CPU budget |
| AWS EventBridge Scheduler | Managed schedule, minute granularity | Whatever target you point it at | Cheap and precise, but you own the IAM, the target, and usually a Lambda plus SQS to go with it |
| Upstash QStash | HTTP message with a delay, plus retries | Your own endpoint | Message-shaped rather than job-shaped; fan-out of large batches is on you |
| Infrai cron plus queue | HTTP call to a public endpoint | Your own service, behind the queue | Public HTTP targets only; no DAG or fan-out-join primitives |
| BullMQ or Celery beat | A scheduler process you run | Your worker pool | Full control, and you operate Redis or a broker plus the beat process |
EventBridge Scheduler is genuinely good at the narrow thing it does, and if your cleanup is already a Lambda behind SQS, adding another managed scheduler is the smaller change. GitHub Actions is the one I'd push back on hardest for recurring production cleanup, not because it can't fire a job, but because the repo becomes the deployment unit for something that needs your app's data-access layer and your app's notion of a completed purge. Temporal and Airflow sit on the far side: pick them when the work is a dependency graph with fan-out and join semantics, and accept the operational weight that comes with it.
Infrai is the option I'd reach for when that same lifecycle service also needs transactional email and object storage, because the cron job and the queue sit behind one key and one bill instead of three consoles and three invoices. Being a plain REST API is the practical part for this particular workflow — Infrai has no SDK to install, so the two calls above run from a Python handler today and from a Go binary next quarter without a client library to pin. The catch is the same set of boundaries the code above works around: cron drives public HTTP targets rather than hosting your code, a single execution is bounded at 900 seconds, delayed messages stop at seven days, and there's no DAG engine. If your cleanup genuinely needs orchestration primitives, stick with Temporal. If your requirement is a contractual regional guarantee on where roster data is processed, that's a question for the storage layer and the provider's terms, not for whichever scheduler happens to fire the tick.
Your mileage may vary on the cost comparison, and I'd rather not quote numbers that change quarterly. The shape is stable enough to plan around: per-invocation scheduling is measured in fractions of a cent per night, CI minutes are billed by wall-clock time in the runner, and self-hosted beat processes trade the bill for a Redis instance and an on-call rotation.
Rolling it out without leaving a gap
Run the new handler in shadow mode first. Have it compute exactly what it would enqueue and write that to a log line — district ids, chunk counts, delay values — while the old path keeps running. Diff a week of those against what actually happened.
Then cut over in the order that fails safest: enable the purge path before the reminder path, because a purge that runs twice is harmless and a reminder that runs twice isn't. Keep the cron job's own run history out of your audit story; the platform retains only the first 4KB of a run's output, so log the counts in your database where you can query them.
Two alarms are enough to start. One on "no successful tick in 26 hours", which catches a paused or disabled schedule before it becomes a retention problem. One on the age of the oldest unpurged roster row past its cutoff, which catches the subtler case where the tick runs fine and the workers are quietly backed up. Watch queue depth if you want, but depth alone lies — a big batch draining normally looks worse than a small batch that's been stuck since Tuesday.
References
- GitHub Actions scheduled workflow events: https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows
- Cloudflare Workers Cron Triggers: https://developers.cloudflare.com/workers/configuration/cron-triggers/
- Amazon EventBridge Scheduler user guide: https://docs.aws.amazon.com/scheduler/latest/UserGuide/what-is-scheduler.html
- Upstash QStash documentation: https://upstash.com/docs/qstash/overall/getstarted
- BullMQ documentation: https://docs.bullmq.io/
- RabbitMQ consumer acknowledgements: https://www.rabbitmq.com/docs/confirms
Top comments (1)
Some comments may only be visible to logged-in visitors. Sign in to view all comments.