For a fintech service that places a fixed hold on a payment reservation, the least complex reliable choice is a small idempotent cleanup job triggered by a cron-compatible HTTP scheduler. The scheduler should wake the job often enough to meet the hold window, while the job itself owns selection, locking, retries, and an audit record. Treat the public URL as a trigger, not as the cleanup system.
Decision table: choose by recovery behavior
| Option | Pick this when | Recovery trade-off |
|---|---|---|
| Host cron plus a private worker | You already operate a VM or container host | Cheap and inspectable, but host outages and clock drift become your problem. |
| Managed HTTP scheduler | You need a public HTTPS trigger and delivery history | Fast to operate; verify retry policy, regional controls, and authentication before committing. |
| Queue-backed workflow | Holds affect several services or need long retries | Strong replay and back-pressure, with more moving parts and more state to observe. |
Cloud scheduler products expose different controls. For example, a managed HTTP scheduler can retry a request, while GitHub Actions is convenient for repository jobs and AWS EventBridge Scheduler is designed for cloud targets. Those are useful reference points, not interchangeable guarantees. Compare delivery history, timeout limits, identity options, and where execution runs. A lower invoice does not repair a missing replay path.
How should a cron HTTP endpoint recover expired sessions and reservations?
Start with a narrow contract. A trigger authenticates, claims a bounded batch, and returns a result that an operator can explain. It should not hold an HTTP connection while scanning every reservation.
Here is the core shape in TypeScript. The storage adapter can use SQL, a document store, or a queue; the recovery rule stays the same.
type Reservation = {
id: string;
expiresAt: Date;
status: "held" | "released";
};
type CleanupResult = {
scanned: number;
released: number;
nextCursor?: string;
};
async function releaseExpired(
now: Date,
limit: number,
cursor: string | undefined,
store: {
claimExpired: (now: Date, limit: number, cursor?: string) => Promise<Reservation[]>;
releaseIfHeld: (id: string, now: Date) => Promise<boolean>;
},
): Promise<CleanupResult> {
const batch = await store.claimExpired(now, limit, cursor);
let released = 0;
for (const reservation of batch) {
if (await store.releaseIfHeld(reservation.id, now)) released += 1;
}
return {
scanned: batch.length,
released,
nextCursor: batch.length === limit ? batch.at(-1)?.id : undefined,
};
}
The important detail is releaseIfHeld: a second delivery sees an already released row and makes no second financial action. Store the trigger id, batch cursor, start time, and counts in an audit record. That record is more useful than a cheerful 200 response when a customer disputes a hold.
I use a 900-second schedule as a starting point only when the product can tolerate that delay. If the hold window is five minutes, a fifteen-minute cadence is wrong even if it is easy to configure. Your mileage may vary: measure the business window, then set cadence, timeout, and batch size from that requirement.
Failure modes worth rehearsing
The public endpoint is an edge between two retry systems. A scheduler may retry after a timeout; a load balancer may replay a request; an operator may press the button twice. Design for all three.
When the worker is busy, return a bounded response and let the scheduler try again. HTTP 429 means “try later,” and the Retry-After header gives a client a delay hint. Do not turn a transient rate limit into a permanent reservation failure. Conversely, do not retry validation errors forever; record them with the reservation id and stop.
Run the failure drill against a staging ledger with six deliberately awkward records: one exactly at the expiry boundary, one already released by a prior attempt, one with a duplicate trigger id, one whose tenant is rate-limited, one with a malformed timestamp, and one that takes longer than the HTTP timeout. Kill the worker after it claims the batch, then deliver the same trigger twice. The expected result is boring: the boundary record is released once, the already released record is skipped, the duplicate is ignored, the rate-limited tenant is deferred, the malformed row is quarantined for review, and the slow request is replayed from its audit cursor. If the counts cannot be explained from logs and the database, the endpoint is not ready for a payment path.
Clock handling deserves a test matrix: UTC storage, daylight-saving transitions in European deployments, a node whose clock is five minutes slow, and a reservation expiring exactly at the boundary. Compare timestamps in the database or a trusted time service. A local wall clock is a poor source of settlement truth.
Expose counters for claimed, released, skipped, retried, rate-limited, and invalid records. Add a latency histogram from claim to release. Tag every log with the trigger id and a redacted tenant id; payment data should never be your debugging payload.
The alert should describe action, not anxiety: “expired holds older than 10 minutes: 2,400” is actionable, while “job failed” is not. Keep a replay command that accepts an audit id and a dry-run flag. Recovery is complete only when an on-call engineer can identify the affected batch, rerun it safely, and verify the resulting count.
Limits and selection rules
This pattern is not suitable when expiry must be enforced in milliseconds, when the endpoint cannot be placed behind strong authentication, or when one cleanup run can exceed the scheduler timeout. Use a queue with lease-based workers for those cases. Stick with host cron when you already have mature host monitoring and the data is local; choose a managed scheduler when delivery history and regional execution matter more than owning every component.
The catch is that “cheap” and “easy” describe setup, not recovery. Before choosing, run a forced timeout, duplicate delivery, partial batch, and 429 drill. Keep the option that leaves an explainable audit trail after each drill.
References
- https://man7.org/linux/man-pages/man5/crontab.5.html
- https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Status/429
Top comments (0)