Short answer: use a durable queue message for each reservation expiry, and keep a small cron scan as a repair path. The queue gives every delayed webhook task its own retry and idempotency boundary; cron remains useful for finding records whose scheduling transaction was interrupted. Neither mechanism should own a long-running job or hide an unreliable public HTTPS endpoint.
Here is the decision matrix I use before writing code:
| Requirement | Cron scanner | Delayed queue message |
|---|---|---|
| Per-event deadline | A scan interval defines lateness | A due time travels with the event |
| Retry and backoff | Application logic | Consumer and application logic |
| Duplicate delivery | Locks and claims are your job | Visibility expiry makes duplicates normal |
| Public HTTPS call | Worker controls timeout | Worker controls timeout |
| Long-running work | Easy to overrun a scan | Separate job state and lease |
For a hold window that expires reservations, I would publish reservation.expire in the same database transaction as expires_at. A five-minute cron pass then audits the table. That is a reliability choice, not a brand preference.
Measure twice.
What should a delayed webhook task remember before it waits?
Persist intent first. A reservation row needs an immutable identifier, its expires_at, current status, and an idempotency key such as reservation:7f3:expire. The outbox row carries the event name and payload. A publisher can retry the outbox independently of the API process that accepted the reservation. The transactional outbox pattern exists for exactly this split-brain failure: the database commit succeeds, then the process dies before a broker call.
The expiry transition must be monotonic. A consumer may receive the same message twice, or receive it after a customer confirmed the reservation. A conditional update turns those races into harmless no-ops.
type ExpiryResult = "expired" | "already_done" | "missing";
interface Reservations {
expireIfHeld(id: string, now: Date): Promise<ExpiryResult>;
}
interface WebhookSender {
send(event: {
type: string;
reservationId: string;
idempotencyKey: string;
}): Promise<void>;
}
export async function consumeExpiry(
message: { reservationId: string },
reservations: Reservations,
sender: WebhookSender,
): Promise<void> {
const result = await reservations.expireIfHeld(message.reservationId, new Date());
if (result !== "expired") return;
await sender.send({
type: "reservation.expired",
reservationId: message.reservationId,
idempotencyKey: `reservation:${message.reservationId}:expire`,
});
}
The sender also needs an idempotency record keyed by that value. Marking the database row expired and sending the webhook are two side effects, so an acknowledgement must happen only after the sender has a durable result. If the process stops between the send and the acknowledgement, the next delivery repeats the key; the receiver should treat that as the same event.
How should cron or a message queue run delayed webhook tasks?
Cron is a clock-driven query. A job such as */5 * * * * asks for every held row with expires_at <= now(), claims a bounded batch, and exits. It is easy to operate and surprisingly easy to make imprecise: a five-minute cadence creates a lateness floor, overlapping workers can select the same rows, and a large backlog can starve newer expirations. Use row-level claims, a maximum batch size, jitter, and a metric for age of the oldest due row.
I once hit the ugly version of this in a fixture with 10,000 reservations. The scan itself was fine; the unbounded webhook fan-out exhausted the connection pool, consumed all available sockets, and made the next scan look late even though its database query was fast. Operators initially raised the scan frequency, which increased overlap and made the queue of outbound requests larger. The fix was boring: 200-row batches, a concurrency limit, a claim timestamp, and an alert on pending work rather than on cron duration alone. Boring is good.
A queue changes the failure shape. Delivery is at-least-once in many systems, and a message can reappear when its visibility lease expires. Set that lease longer than the normal HTTPS request, renew it only while a bounded handler is making progress, and cap retries before moving a poison event to a review stream. Do not call a multi-hour carrier workflow while holding a delivery lease. Create a job record, acknowledge the short webhook dispatch, and emit a completion event later.
Your mileage may vary on retry counts. I’m not sure a fixed number transfers between carriers; measure endpoint latency, status-code classes, and duplicate rates from your own traffic. Retry network timeouts and selected 5xx responses with exponential backoff and jitter. Do not retry a deterministic 4xx forever.
Can a public HTTPS endpoint be the scheduler for these tasks?
No. HTTPS is a delivery boundary, not durable time storage. The endpoint can authenticate a request, validate a signature, enforce a body limit, and return quickly. It cannot guarantee that a process will remember an event for six hours after a deploy or a worker restart. Put the due timestamp and payload in durable storage first, then let a scheduler or queue perform delivery.
Treat response handling as part of the contract. A 2xx response can acknowledge receipt; a 4xx may mean a bad payload that needs operator action; a timeout gives no knowledge of whether the receiver committed the event. This is why the idempotency key belongs in the payload or header and in the receiver's durable store. TLS protects the channel, but it does not make retries safe.
For long-running jobs, return an accepted status after enqueueing work and expose a separate status event or polling record. Holding an HTTP connection open until a route is allocated is a resource leak disguised as simplicity.
Where does the runner-up, cron, fit better?
The catch is operational weight. A queue brings visibility settings, dead-letter handling, replay tooling, and another set of dashboards. A small internal tool with a few dozen reservations per day may be better served by one table and a one-minute cron worker, especially when five minutes of lateness is acceptable and the team can test the claim query under contention.
Choose cron when deadlines are soft, the data already lives in a database, and a repair scan is the primary requirement. Choose a queue when each event has a distinct deadline, retries need backoff, or a missed expiry changes customer or inventory state. In both cases, keep the conditional state transition and idempotent receiver. Swapping the timer does not remove those obligations.
I benchmark the boring parts: claim-query time at the target row count, p95 and p99 webhook latency, retry delay, and recovery time after a consumer restart. The numbers decide the interval, lease, and concurrency limits. Configuration should fit on one screen. If it takes a page of knobs to explain a reservation expiry, the design is already asking for trouble.
Top comments (0)