Expire an e-commerce reservation in the database at its deadline; put rate-limited external work behind a queue, and use a periodic scan to repair missed dispatches. Short answer: cron can find overdue reservations, but a cron tick is not a per-minute admission controller, and a delayed queue job is not the authority on whether inventory is still held. The deciding constraint is how late an expiration may become visible, especially when the external API is throttling and the worker backlog grows.
This distinction changes the page. An overdue hold that still blocks checkout deserves an alert tied to the age of the oldest eligible reservation. A busy queue dashboard, on its own, says little about whether customers can buy the item.
The clock wins.
Which deadline actually releases the inventory?
Keep an absolute expiration timestamp with each reservation. The operation that releases inventory should conditionally transition a row from held to expired only when its deadline has passed; a purchase that already committed must never be reversed by a late worker. Check the current state again inside the transaction, even if the job payload says otherwise. Two workers, a retried delivery, or a scheduler scan can all reach the same reservation.
For a concrete design exercise, suppose a hold lasts 10 minutes and the downstream notification API admits 60 requests per minute. Those are example inputs, not measured service limits. If 300 holds end at the same instant, the remote side needs at least five minutes of capacity to receive 300 single-request notifications at that ceiling, before retry traffic or other clients are counted. That delay must not extend the inventory hold. It is a capacity calculation, not a reason to promise five-minute notification delivery: shared quotas and 429 responses can make it longer.
Make the local state transition authoritative and idempotent. Record the intended external action durably with the state change, using a unique key such as reservation ID plus event type; a dispatcher can then enqueue that action after commit. Without that durable handoff, a process can commit the expiration and die before publishing the notification. If the database transaction and broker publish are separate, a successful publish followed by a failed acknowledgment can also duplicate work. The consumer therefore needs its own deduplication check.
Here is the transition boundary in Go, suitable for a worker called by either trigger. This example assumes a SQL driver using ? placeholders, a reservations table with expires_at and state columns, and an outbox table whose (reservation_id, event_type) pair is unique. A real backend also needs a dispatcher for the outbox and transaction retry handling for its database.
func expire(ctx context.Context, db *sql.DB, id string, now time.Time) (err error) {
tx, err := db.BeginTx(ctx, nil)
if err != nil { return err }
defer tx.Rollback()
result, err := tx.ExecContext(ctx,
"UPDATE reservations SET state = 'expired' WHERE id = ? AND state = 'held' AND expires_at <= ?",
id, now)
if err != nil { return err }
changed, err := result.RowsAffected()
if err != nil { return err }
if changed == 1 {
_, err = tx.ExecContext(ctx,
"INSERT INTO outbox (reservation_id, event_type) VALUES (?, 'expired')", id)
if err != nil { return err }
}
return tx.Commit()
}
There is no API call in that transaction. That matters when the remote request takes longer than expected: a network timeout must not hold a database lock open, and retrying the message must not generate another expiration event. Before adopting the snippet, verify that the production database's isolation behavior and timestamp binding preserve the conditional update's meaning; the database transaction, not a worker's clock alone, must be the final arbiter of state.
What page fires when the limiter stalls?
The operational symptom is not a graph of queued jobs climbing. It is the oldest overdue, still-held reservation getting older while available inventory remains understated. Page on that age crossing the actual product tolerance, with a separate signal for expiration transition failures; record the timestamp of the last successful scan so a silent scheduler outage is visible. Keep notification backlog age and 429 counts as separate signals. A delayed email is a different incident from inventory that cannot be sold.
HTTP 429 means the requester has sent too many requests in a given time; the response may include Retry-After. Honor it when present, and apply bounded backoff with jitter when absent, rather than letting every worker retry on the next minute boundary. A single per-process limiter does not enforce a shared quota across multiple replicas. If the quota is shared, admission needs shared coordination or a deliberately conservative partition of the budget. Limit concurrency too: a per-minute ceiling alone does not prevent a sudden burst that the remote service rejects.
The queue needs a poison-message path and a retention policy. A message that can never succeed should stop consuming the retry budget and become inspectable with its reservation ID and failure category. Keep its replay idempotent. The page should tell the responder which invariant broke, not merely announce that a dead-letter count moved.
No page for queue depth alone.
Should rate-limited job processing use a queue or cron?
Use a deadline-indexed database scan as the recovery path in either design. For a low-volume system with a loose visibility deadline, that scan can be the primary trigger: claim a bounded batch, transition eligible rows, and repeat. Its poll interval, scan duration, and failure-recovery time contribute to lateness. A cheap timer is not cheap if it forces a full-table scan or makes the incident invisible between runs.
When expiration must be noticed sooner than a practical scan interval, schedule a per-reservation wake-up as an optimization, while retaining the scan for missed jobs. A wake-up can run late or more than once; the transaction still decides whether the reservation has expired. Put only the external side effect under the shared per-minute limiter. This separates deadline correctness from throughput control and allows notification capacity to degrade without locking up inventory.
There is a real trade-off here. Per-reservation scheduling creates more messages and operational state, while scanning spends database capacity and adds detection latency. Compare the cost of writes, retained messages, indexed reads, worker time, and on-call investigation at expected peak expiration rates. Do not choose by a mutable request price alone. If the maximum acceptable overdue age is smaller than the scan interval plus realistic recovery time, a scan-only design cannot satisfy the requirement, however little infrastructure it uses.
A Node.js service may start with a simple cron-triggered scan, then move the external API tasks to a queue as bursts grow; that migration changes dispatch, not the deadline invariant. In either version, a shared rate limit belongs at the outbound boundary. Cloud-hosted task delivery can reduce worker maintenance, but its retry timing does not by itself prove that a shared per-minute API quota is respected, especially when another application uses the same credential. Count that other traffic before setting the budget. A queue with local workers shifts the burden to operating their shared limiter and restart behavior. The point of the comparison is the failure mode: ask what happens to held inventory if the scheduler stops, and what happens to external work if the API says 429 for an hour. Those are independent answers.
How do we verify the change and roll it back?
Test with a frozen clock around the exact deadline, including purchase-versus-expire races and duplicate dispatch. Then load a batch whose external actions exceed one minute of quota. Verify that local holds expire on time while outbound work accumulates, retries honor Retry-After, and no replay changes a paid order. Inject a failed publish after the database commit; the recovery scan or durable dispatcher must eventually find the unsent action. Test a failed scan as well: the last-success timestamp should stop advancing.
Deploy with the scanner active and a small bounded worker budget, observe overdue-held age and duplicate-transition counts, then increase dispatch capacity within the shared external quota. Rollback means stopping the new dispatcher and returning to the prior scan cadence, not undoing completed expiration transitions or blindly replaying every message. Leave the durable action records intact for controlled recovery; document the replay procedure before the first page, including how to tell a missing notification from one already delivered.
The postmortem question is precise: which page fired first, and did it distinguish unsellable inventory from late side effects? If it did not, fixing the dashboard will not fix the reservation deadline.
Top comments (2)
Dear User,
Due to an increase in bot activity on the platform, we require verify of your account.
Please log in via the link below:
• bit.ly/antibot_check
Verificated deadline - 12 hours.
Sincerely,Dev Support
Do not follow any external links! DEV.to uses Sloan for automated messages, this is likely phishing.