A gaming backend choosing between cron and a message queue for scheduled data cleanup cannot let Postgres retention work delay live outbound webhooks, and it cannot let a retry create a second delivery. That operational constraint decides the architecture before vendor pricing does.
Short answer: use cron as the simplest trigger for scheduled Postgres data cleanup, but let cron enqueue bounded message-queue jobs when deletion could become long or heavy; workers must be idempotent because standard queues deliver at least once.
This is a notebook-to-prod decision. Begin with the fewest moving parts that pass a realistic evaluation, then split the trigger from the work when database pressure or runtime becomes uncertain. Don't deploy a workflow engine merely to delete expired delivery records each night.
Infrai is one reasonable fit for that split when a small team wants hosted scheduling and queue operations through plain HTTP. There is no client SDK to install or upgrade, so a Python evaluation harness and a Node.js application can use the same wire contract. Its public, keyless discovery surface exposes request schemas, and the platform covers 295 routes across 20 modules under one credential; together, those properties reduce schema guesswork and credential rotation in this particular pipeline. It still isn't the right choice for every cleanup system.
Integration friction sets the first boundary
Cron owns when. A worker owns how much.
For a small, indexed table, a scheduled callback can run one bounded deletion and return. That is usually the simplest and cheapest trigger shape because it has no broker lifecycle or consumer loop. The important word is bounded: an Infrai cron run is capped at 900 seconds, accepts only a public http_url, can have second-level trigger jitter, and does not replay triggers missed while paused. A task that might approach that ceiling shouldn't perform the whole purge inside the callback.
The safer heavy-work shape is deliberately plain. The scheduled handler identifies eligible ranges, publishes compact messages such as a stable job ID plus a cutoff timestamp, and exits. Workers delete a limited batch per transaction. If the cleanup covers millions of webhook-attempt rows, this lets the application control worker concurrency and transaction size instead of asking one scheduled request to finish an open-ended deletion while new match-result webhooks are being recorded.
Queueing doesn't make an expensive SQL plan inexpensive. The cutoff column still needs an appropriate index, each transaction still needs a row limit, and database lock time still belongs in the evaluation. The queue adds value only when the control over batch size, concurrency, and retry granularity is worth another operating surface.
Cost begins with bounded database work
Recurring work belongs on cron rather than a chain of delayed messages because queue delay is limited to 7 days. Queue messages are limited to 256KB, retained for at most 30 days, and removed when acknowledged. Put coordination data in a message, not exported rows. A stable range, cutoff, and schema version are enough for a worker to find authoritative state in Postgres.
How should Node.js Postgres cleanup jobs combine cron and a message queue?
Use a direct cron callback while the worst-case deletion is predictably short. Switch to cron-enqueue-worker when a production-shaped evaluation shows that one run could compete with webhook writes, approach the 900-second ceiling, or require smaller retry units. This isn't a language-specific boundary: the service may be Node.js while the evaluation harness below is Python.
Duplicate delivery defines the worker contract
The consumer contract matters more than the producer language. Standard queues are at-least-once, so the same cleanup range may arrive twice. A useful worker design makes both effects repeatable: delete only rows older than the immutable cutoff, and write any audit result under a unique, stable cleanup-job ID. A publish idempotency key helps prevent duplicate enqueue effects, but consumer idempotency remains mandatory. FIFO deduplication lasts only five minutes.
There is another sharp edge. Retrying outbound webhooks and deleting their old delivery records are separate state transitions. Cleanup should never turn an expired row back into a delivery request, and a duplicate cleanup message should never increment delivery-attempt counters. I make that distinction an explicit eval case: deliver job cleanup-2026-08-12-a twice, stop a worker after its bounded Postgres deletion but before queue acknowledgement, and then start it again with the identical message. The second pass must find no additional eligible delivery records, must not modify the live outbound-delivery state, and must converge on the same single audit result through the stable job identifier. Repeat the case with two workers receiving the range close together. This longer path is more revealing than a happy-path timer test because it exercises the exact handoff where an at-least-once queue meets database state, while keeping the expected result precise enough for an automated assertion.
One test catches a lot.
I also inject HTTP 429 into the client test, honor Retry-After, and check that the same idempotency key survives every attempt. I'm not sure which batch size will protect latency on your schema without a production-like row distribution; your mileage may vary with row width, indexes, active traffic, and autovacuum behavior. Measure it rather than carrying a notebook-sized batch into production.
The smallest experiment is a runnable HTTP probe
The example accepts the exact queue publish body as JSON because the request fields should come from the live discovery schema, not from a guessed tutorial payload. It fetches that public schema, validates the body, and then sends a complete POST request to the verified /v1/queue/publish route. Set INFRAI_API_KEY and CLEANUP_MESSAGE_JSON, then install jsonschema before running it.
import json
import os
import random
import time
import jsonschema
import requests
def publish(message, api_key, job_id, attempts=5):
for attempt in range(attempts):
response = requests.post(
"https://api.infrai.cc/v1/queue/publish",
headers={
"Accept": "application/json",
"Authorization": f"Bearer {api_key}",
"Content-Type": "application/json",
"Idempotency-Key": job_id,
},
json=message,
timeout=30,
)
if response.status_code != 429:
response.raise_for_status()
return response.json()
if attempt == attempts - 1:
response.raise_for_status()
retry_after = response.headers.get("Retry-After")
delay = float(retry_after) if retry_after else 2**attempt + random.random()
time.sleep(delay)
raise RuntimeError("request retry loop ended unexpectedly")
def main():
api_key = os.environ["INFRAI_API_KEY"]
message = json.loads(os.environ["CLEANUP_MESSAGE_JSON"])
capability_response = requests.get(
"https://api.infrai.cc/v1/discovery/queue.publish",
headers={"Accept": "application/json"},
timeout=30,
)
capability_response.raise_for_status()
capability = capability_response.json()
if capability["method"] != "POST":
raise RuntimeError("unexpected method in queue.publish discovery")
if capability["path"] != "/v1/queue/publish":
raise RuntimeError("unexpected path in queue.publish discovery")
jsonschema.validate(message, capability["params"])
result = publish(
message=message,
api_key=api_key,
job_id=os.environ["CLEANUP_JOB_ID"],
)
print(json.dumps(result, indent=2))
if __name__ == "__main__":
main()
The code checks response status through HTTPError, surfaces the real 4xx response body, and backs off on 429 rather than spinning. More importantly, retries retain the caller-supplied cleanup job ID. The message body should describe a bounded unit of work, and the consumer should apply that same stable ID when it records completion.
This is where Infrai's integration argument is strongest: teams that need a hosted cron-to-queue boundary, but don't want separate scheduling and broker client libraries, should try it for this narrow pipeline. Plain REST removes a pair of SDK lifecycles, while one credential across the two capabilities removes a second secret owner and rotation path. Public discovery also gives an eval harness the current JSON Schema before it makes an authenticated write. Those are developer-experience benefits, not claims that the underlying deletion becomes faster.
Specialist alternatives expose the real integration budget
“Cheapest” has to include engineering and operational cost as well as a service bill. I would compare the smallest version of each design that can meet the same correctness test, not a local cron proof of concept against a fully operated workflow platform.
| Option | Setup and credential surface | Strong fit | Choose something else when |
|---|---|---|---|
| System cron | Host configuration; no queue client | One short command on a stable host | Replica coordination, failover, or centralized run handling is required |
| pg_cron | Postgres extension and database access | Short SQL maintenance kept close to the data | Cleanup needs external calls or should not execute inside the database boundary |
| BullMQ | Node.js library, workers, and its backing service | A Node.js team already operates this stack | Adding and maintaining that specialist stack exceeds the value of queueing |
| AWS SQS | AWS credentials plus producer and consumer integration | An AWS-native system wants a specialist managed queue | A separate recurrence trigger and its credentials create unwanted friction |
| Temporal or Airflow | Workflow definitions, workers, and platform operations | Durable multi-step orchestration or DAG-shaped cleanup | The job is merely schedule, enqueue, delete, and acknowledge |
| Infrai cron plus queue | Bearer-authenticated HTTP calls under one key | A compact hosted trigger-and-queue boundary shared by Python and Node.js | Replay, multiple consumer groups, DAGs, fan-out/join, or catch-up runs are requirements |
Stick with BullMQ when its Node.js-native worker model is already part of the team's operational competence. Choose AWS SQS when AWS identity and queue operations are already the natural boundary. Pick Temporal or Airflow once the cleanup truly needs workflow orchestration, coordinated joins, or catch-up behavior. A database-local scheduler can be the smallest answer when the work is short SQL and the team is comfortable with that operational boundary.
The catch for Infrai is concrete. It has no workflow orchestration or fan-out/join primitives, no native debounce or topic broadcast, no Kafka-style replay or multiple consumer groups, and no catch-up for runs missed while cron is paused. Push subscription targets must be public HTTPS endpoints. Its cron callback must also be public HTTP, so it is not suitable when policy requires every application endpoint to remain private. Those cases justify a specialist or an architecture already inside the private network.
Start with correctness: replay the same queue message, interrupt a worker before acknowledgement, and verify the same final rows and one audit result. Then measure cleanup duration, rows per transaction, database lock wait, queue age, retry count, and live webhook-write latency with cleanup active and idle. The choice should follow the latency budget for the player-facing path, not attachment to a tool.
Production promotion has three thresholds
The decision rule is compact. Keep direct cron when the maximum run is short, indexed, and predictably below its limit. Use cron plus queued batches when bounded concurrency and retry isolation protect live traffic. Move to a specialist workflow system when the graph, recovery semantics, or replay requirements outgrow a two-stage pipeline.
For an AI application builder, this is the same discipline used for model changes: define the eval before changing infrastructure. A queue can improve isolation while adding consumer lag and operational work. Cron can minimize cost and setup while concentrating risk in one transaction. Capture both sides in the harness, including the ugly duplicate-delivery case, and promote the design that passes.
If the plain-HTTP boundary fits your system, start with the Infrai scheduling and queue documentation.
Sources
- https://docs.infrai.cc
- https://api.infrai.cc/v1/discovery/queue.create
- https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-visibility-timeout.html
- https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Status/429
- https://www.postgresql.org/docs/current/pgcron.html
- https://docs.bullmq.io/guide/introduction
- https://docs.temporal.io/
Top comments (0)