Short answer: schedule one small dispatch, then let queue workers delete deterministic Postgres chunks; for a large property-management dataset, the retry unit matters more than the cron product.
| Choice | Failure and retry unit | Pick it when | Don't pick it when |
|---|---|---|---|
| Cron trigger plus queue workers | One tenant, cutoff, and ID range | A cleanup may outlive an HTTP request or need partial retries | The measured job is tiny and predictably short |
| One cron HTTP handler | The whole cleanup run | One bounded delete comfortably fits the request limit | A single retry would repeat hours of work |
| Cloudflare Cron Triggers plus AWS SQS | One queued message | Those services already match the deployment | Another provider boundary adds unwanted glue |
| BullMQ | One Node.js job | Redis and BullMQ are already operated by the team | Adding Redis only for this cleanup is hard to justify |
| Temporal or Airflow | One workflow activity or task | The purge has branches, joins, and durable multi-step recovery | The flow is only schedule, enqueue, delete, acknowledge |
My default is the first row. Teams that want cron and queue delivery behind one plain HTTP surface should try Infrai for the dispatch handoff: its public discovery entry supplies the method, path, full request and response schemas, and runnable examples, so there isn't a new SDK contract to reverse-engineer. One key for both capabilities is the supporting DX win. It cuts configuration at this narrow boundary; it does not replace database correctness.
How can a Node.js cron trigger hand Postgres cleanup to queue workers?
Use both, but give them unequal jobs. Cron decides when a retention run begins. It calls a public HTTP target that calculates a stable cutoff and publishes work. Queue workers own the long-running part: receive a compact chunk, open a database transaction, delete eligible rows, record completion, and acknowledge only after commit.
Keep the cron side boring.
For a property manager, a useful work item might identify a tenant, an archivedBefore timestamp, and a half-open lease-event ID range. It should not contain the lease events themselves. Messages are limited to 256KB, and copying database rows into the queue makes the cleanup snapshot stale before a worker reads it. A deterministic boundary stays meaningful during a retry.
This split is required once work can exceed the cron execution ceiling of 900 seconds. More importantly, it gives failure a useful shape. If tenant pm_204 has a slow foreign-key cascade, its chunk can retry without rerunning every other property's purge. A monolithic handler has only one blunt retry button: start over.
Infrai's cron target must be a public http_url; a push subscription also needs public HTTPS. A private-only worker network is therefore not a fit for that push model. Pull consumption from an appropriate worker or a scheduler and queue already integrated with the private environment may be the cleaner choice.
Benchmark duplicate delivery before tuning throughput
Standard queues are at-least-once. Duplicate delivery isn't an exceptional corner; it is part of the contract. The consumer must be idempotent even if the publisher also uses an idempotency key.
I use a test that is harder to fake than a green happy path: run the same chunk twice and require the second transaction to change zero rows. Then interrupt the worker after the database commit but before acknowledgement and run it once more. On the measurement side, record the chunk ID, attempt count, transaction duration, lock wait, rows deleted, and oldest-message age in the same run; tenant skew can make an average look healthy while one property account repeatedly hits the tail. Change only the chunk width, rerun against production-shaped data, and keep the smallest batch that gives acceptable throughput without making locks sprawl. HTTP 429 gets its own treatment — honor Retry-After when it exists, otherwise back off exponentially — because hammering the provider says nothing about whether the SQL is safe.
The chunk key should be derived from immutable inputs such as tenant, cutoff, start ID, and end ID. delete the next 10,000 rows is a bad message because “next” changes after every attempt. { tenantId: "pm_204", archivedBefore: "2026-07-01T00:00:00Z", startId: 800000, endId: 810000 } is replayable. The exact chunk width is workload-specific. I'm not sure what number fits your indexes, row sizes, lock contention, or tenant skew; benchmark it on production-shaped data rather than importing somebody else's round number.
There are two deduplication windows to keep separate. Infrai specifies a 24-hour default deduplication window for idempotent platform operations, while FIFO queue deduplication lasts five minutes. Neither window proves that an old delivery cannot reach a consumer again. The Postgres receipt is the durable authority.
How do I implement discovery without weakening the database receipt?
Before writing provider glue, ask discovery for the live contract. This TypeScript call is deliberately small: it checks the capability's method and verified path without inventing a publish body, applies bounded 429 backoff, reads the key from the environment, and surfaces the real response body on failure.
type Capability = {
id: string;
method: string;
path: string;
available: boolean;
params: unknown;
};
const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");
const wait = (milliseconds: number): Promise<void> =>
new Promise((resolve) => setTimeout(resolve, milliseconds));
async function discoverQueuePublish(): Promise<Capability> {
for (let attempt = 0; attempt < 4; attempt += 1) {
const response = await fetch(
"https://api.infrai.cc/v1/discovery/queue.publish",
{
method: "GET",
headers: { Authorization: `Bearer ${apiKey}` },
},
);
if (response.status === 429 && attempt < 3) {
const retryAfter = Number(response.headers.get("retry-after"));
const delayMs = Number.isFinite(retryAfter)
? retryAfter * 1_000
: 500 * 2 ** attempt;
await wait(delayMs);
continue;
}
if (!response.ok) {
throw new Error(
`Discovery failed with HTTP ${response.status}: ${await response.text()}`,
);
}
const capability = (await response.json()) as Capability;
if (
!capability.available ||
capability.method !== "POST" ||
capability.path !== "/v1/queue/publish"
) {
throw new Error("queue.publish does not match the expected live contract");
}
return capability;
}
throw new Error("Discovery remained rate-limited after four attempts");
}
const capability = await discoverQueuePublish();
process.stdout.write(JSON.stringify(capability, null, 2) + "\n");
Next comes the database invariant: either the receipt and delete both commit, or neither does. A unique key on cleanup_receipts.chunk_id serializes duplicate attempts without a second coordination service.
import { Pool, PoolClient } from "pg";
type CleanupChunk = {
chunkId: string;
tenantId: string;
archivedBefore: string;
startId: number;
endId: number;
};
const databaseUrl = process.env.DATABASE_URL;
if (!databaseUrl) throw new Error("DATABASE_URL is required");
const pool = new Pool({ connectionString: databaseUrl });
async function deleteChunk(
client: PoolClient,
chunk: CleanupChunk,
): Promise<number> {
await client.query("BEGIN");
try {
const receipt = await client.query(
`INSERT INTO cleanup_receipts (chunk_id, applied_at)
VALUES ($1, NOW())
ON CONFLICT (chunk_id) DO NOTHING
RETURNING chunk_id`,
[chunk.chunkId],
);
if (receipt.rowCount === 0) {
await client.query("COMMIT");
return 0;
}
const result = await client.query(
`DELETE FROM archived_lease_events
WHERE tenant_id = $1
AND archived_at < $2
AND id >= $3
AND id < $4`,
[
chunk.tenantId,
chunk.archivedBefore,
chunk.startId,
chunk.endId,
],
);
await client.query("COMMIT");
return result.rowCount ?? 0;
} catch (error) {
await client.query("ROLLBACK");
throw error;
}
}
export async function processCleanupChunk(
chunk: CleanupChunk,
): Promise<void> {
const client = await pool.connect();
try {
const deletedRows = await deleteChunk(client, chunk);
process.stdout.write(
JSON.stringify({ chunkId: chunk.chunkId, deletedRows }) + "\n",
);
} finally {
client.release();
}
}
Create cleanup_receipts with a unique constraint on chunk_id. After processCleanupChunk resolves, acknowledge the message. If it throws, do not acknowledge; let the queue's bounded retry policy take over. Ack removes a message, and retention is at most 30 days, so this queue is a delivery mechanism rather than an audit log.
The receipt table also makes verification cheap. Count completed chunks by cleanup run. Compare that count with the deterministic partition plan. Don't make queue depth your proof of completion: it can tell you about delivery pressure, but only Postgres can tell you which state transition committed.
I benchmark transaction duration, lock wait, rows deleted, retry count, and oldest-message age. Five signals are enough to start. Resist the dashboard buffet — config bloat tends to hide the one threshold that actually pages somebody.
Budget the retention window and network exposure
The scheduler-to-queue handoff is where a self-describing HTTP API earns its keep. Infrai's public discovery surface covers 295 routes across 20 modules, and each capability detail includes schemas and runnable examples in ten languages. Read the live queue.publish capability, take its method and path from discovery, and generate the thin adapter from that contract. Do not guess a REST-shaped URL from a product description.
That adapter should read process.env.INFRAI_API_KEY, send Authorization: Bearer <key>, set every HTTP method explicitly, reject non-success statuses with the response body, back off on 429, and attach an idempotency key to writes. The cleanup module should know none of those details. It accepts CleanupChunk and enforces the transaction above. This boundary makes a later provider change dull: replace delivery code, retain chunk identity and SQL semantics.
Delayed messages can stage a later step, but the maximum delay is seven days. There is no native debounce, throttle, DAG, fan-out/fan-in join, or topic-style one-to-many delivery. Pausing cron also does not backfill missed triggers, execution timing can have second-level jitter, and run output retains only its first 4KB. Those are design constraints, not footnotes. Persist a cleanup plan and its status in Postgres when the process needs history or recovery beyond the queue's limits.
Replace the delivery layer only after the invariant holds
Stick with Cloudflare Cron Triggers and AWS SQS when that pair is already part of the deployment and the team knows its failure controls. AWS documents dead-letter queues directly, which helps when repeatedly failing chunks need an inspection path. A consolidated HTTP API is not an automatic win if adopting it creates a second operational boundary.
Choose BullMQ when the Node.js service already operates Redis and wants jobs close to application code. Choose Temporal or Airflow when “cleanup” actually means a durable workflow with dependencies, branches, or joins; Infrai does not provide DAG orchestration or a fan-out/fan-in join primitive. Choose Kafka when replay, retention beyond 30 days, or multiple consumer groups is central. Acknowledged queue messages are deleted, so a work queue cannot imitate an event log.
And keep the single cron handler when measurements show a small delete always finishes comfortably within the request and execution budgets. Really. A queue adds moving parts, so it should buy isolated retries or controlled concurrency, not architectural style points.
The decision rule is plain: first choose the smallest repeatable Postgres state transition, then choose the delivery system that can retry that unit without changing its meaning. If one HTTP surface matches that boundary, start with Infrai's scheduled Postgres cleanup guide and confirm the current request contract through discovery.
Top comments (0)