Expiring Cart Holds: FIFO or Standard Queue for Retry Jobs and Idempotency

Use a standard queue for expiring stale cart holds, and keep deduplication in your own database rather than in the transport. FIFO ordering buys you nothing when each job releases a different reservation, and the FIFO deduplication window is five minutes while the hold you are enforcing is fifteen — the window expires before the job is even due.

The rule I would hold this design to is an evaluation, not a preference: replay a day of reservation events, then require zero double-releases and a p95 release lateness inside whatever the merchandising side can tolerate, at a per-expiry cost that does not grow with the size of the reservations table. Latency against cost is the whole decision here. Ordering never made the list.

The 15-minute hold, and the sweep that stops being cheap

The system is a storefront checkout. A shopper reserves two units of SKU-COFFEE-1KG, the row lands in reservations(id, sku, qty, expires_at, state) with a fifteen-minute hold, and if payment does not land in that window the stock goes back on the shelf.

Version one is a cron sweep every sixty seconds: SELECT ... WHERE expires_at <= now() AND state = 'held' FOR UPDATE SKIP LOCKED LIMIT 500. Honestly, for a small business app this is the correct answer and I would not talk anyone out of it. Postgres hands you concurrent sweepers for free with SKIP LOCKED, there's no new infrastructure, and release lateness is bounded by the poll interval.

The trade shows up the moment you want that lateness smaller. Lateness is the poll interval, so halving it doubles the scans, and each scan costs a little more as the table grows. During a flash sale the sweep starts taking longer than its own interval, which means you pay the most for scanning at exactly the moment oversold inventory hurts the most.

Poll faster, pay more. That is the axis.

Version two hands the timer to the queue instead: when the reservation is written, publish one delayed job due when the hold ends. Work per reservation is now constant and independent of how many rows are sitting in the table. The cron sweep survives, downgraded to a five-minute safety net that catches reservations whose publish never happened — a process dying between the database write and the publish call is the one gap the queue cannot close by itself. Two mechanisms, two jobs: the queue owns punctuality, the sweep owns completeness.

Should retries for failed expiry jobs go through a FIFO or standard queue?

Standard, for this workload, and the reasoning is worth walking through because it generalises past cart holds.

Start with ordering. Releasing reservation A before reservation B leaves inventory in exactly the same state as the reverse; these events commute. The only sequencing that matters is per-reservation, and it is a state question rather than a transport question. A job saying "release rsv_8241 as of epoch 1786000000" is a no-op if the row's expires_at has since moved to 1786000600 because the shopper extended the hold. That check lives in your consumer, where you can actually read the current state.

Then the deduplication window, which is where FIFO gets oversold. Five minutes is the window on the queues that offer it, and a fifteen-minute hold outlives it before the job becomes due. Any retry beyond that — a redrive of a dead-letter batch an hour later, a publisher retrying after a deploy — arrives with the window long gone. So deduplication cannot be the thing standing between you and releasing the same stock twice. It never spans the retry horizon of failed jobs.

Third, standard delivery is at-least-once, which means the consumer has to be idempotent regardless. Once it is, the remaining value FIFO offers this workload is ordering you've already established you don't need, paid for with per-group throughput limits.

FIFO earns its place when the business rule is literally "apply these events in the order produced" and you cannot reconstruct that order from the payload — a per-account ledger, a state machine whose transitions are not commutative. Stick with FIFO there. For independent expiries, it is machinery you maintain for a guarantee you never consult.

The consumer guard, in about forty lines

The interesting part is not the transport. It is the four lines that claim the effect before performing it, and the fact that the claim and the effect commit in one transaction.

I wired this example against Infrai's queue because the discovery surface is self-describing and public: one GET returns the request schema, the response schema, and runnable examples, so adding a capability is reading one endpoint instead of adopting another SDK. Its write convention is useful here too — an Idempotency-Key header with a 24-hour default deduplication window, which is a different order of magnitude from five minutes. That covers the publish side. It doesn't cover the consumer side, and no queue's convention does. The ledger below is your job.

import os
import sqlite3
import time

import requests

API = os.environ["INFRAI_API_BASE"]  # the v1 base URL of the queue service
QUEUE = "reservation-expiry"
HOLD_SECONDS = 900  # 15-minute cart hold, expressed in the queue's delay units

client = requests.Session()
client.headers.update({
    "Authorization": f"Bearer {os.environ['INFRAI_API_KEY']}",
    "Content-Type": "application/json",
})

db = sqlite3.connect("reservations.db")
db.executescript("""
CREATE TABLE IF NOT EXISTS stock (sku TEXT PRIMARY KEY, on_hand INTEGER NOT NULL);
CREATE TABLE IF NOT EXISTS released (effect_key TEXT PRIMARY KEY, at REAL NOT NULL);
INSERT OR IGNORE INTO stock (sku, on_hand) VALUES ('SKU-COFFEE-1KG', 40);
""")
db.commit()


def schedule_expiry(reservation_id, sku, qty, expires_at):
    """One delayed job per hold. The queue owns the timer, not a polling loop."""
    effect_key = f"{reservation_id}:{expires_at}"
    backoff = 1.0
    for _ in range(5):
        response = client.post(
            f"{API}/queue/publish",
            json={
                "queue": QUEUE,
                "payload": {"effect_key": effect_key, "sku": sku, "qty": qty},
                "delay_seconds": HOLD_SECONDS,
            },
            headers={"Idempotency-Key": effect_key},
            timeout=15,
        )
        if response.status_code == 429:
            time.sleep(float(response.headers.get("Retry-After", backoff)))
            backoff *= 2
            continue
        if response.status_code >= 400:
            raise RuntimeError(f"publish {response.status_code}: {response.text[:200]}")
        return response.json()["data"]["message_id"]
    raise RuntimeError("publish: retry budget exhausted")


def release_once(payload):
    """Claim the effect, then perform it. A redelivery hits the claim and stops."""
    claimed = db.execute(
        "INSERT OR IGNORE INTO released (effect_key, at) VALUES (?, ?)",
        (payload["effect_key"], time.time()),
    ).rowcount == 1
    if not claimed:
        db.commit()
        return "duplicate"
    db.execute(
        "UPDATE stock SET on_hand = on_hand + ? WHERE sku = ?",
        (payload["qty"], payload["sku"]),
    )
    db.commit()  # the claim and the release land in the same transaction
    return "released"


if __name__ == "__main__":
    expires_at = int(time.time()) + HOLD_SECONDS
    print(schedule_expiry("rsv_8241", "SKU-COFFEE-1KG", 2, expires_at))
    job = {"effect_key": f"rsv_8241:{expires_at}", "sku": "SKU-COFFEE-1KG", "qty": 2}
    print(release_once(job), release_once(job))  # released duplicate

The effect key is reservation_id:expires_at, so an extended hold produces a different key and the stale job's release is claimed against a key nobody will ever release again. The worker pulls with POST /v1/queue/consume, runs release_once, and confirms with POST /v1/queue/ack only after the transaction commits — a crash before the acknowledgement means redelivery rather than a lost job, and redelivery is already safe. That ordering of operations is the entire idempotency story.

Where the other options fit

Option	How you reach it	Fits when	Main limit
Postgres sweep with SKIP LOCKED	SQL you already own	Low volume, no appetite for new infrastructure	Lateness equals the poll interval; scan cost grows with the table
Celery with Redis or RabbitMQ	Python worker library plus a broker	Python teams that want worker-level control and ETA scheduling	You own the broker, its persistence and its alerts
Upstash QStash	HTTP publish with a delay, delivered to your endpoint	Serverless workers with no long-lived consumer process	Your consumer endpoint has to be publicly reachable
Amazon EventBridge Scheduler with SQS	AWS APIs and IAM policies	Estates already standardised on AWS	Two services to wire; FIFO deduplication is still a five-minute window
Temporal	Workflow SDK plus a cluster or its cloud	The hold is one step in a multi-step compensating flow	Far more machinery than a timer needs
Infrai queue	Plain REST over HTTP, one key for the whole backend	You want a delayed queue without adopting an SDK or a second bill	No DAG or fan-out/join orchestration; acknowledged messages are deleted, so no replay log

Sidekiq and BullMQ belong on a longer version of this table; I left them off because a Python storefront rarely wants a Ruby or Node worker fleet just for expiries. Google Cloud Tasks is the closest GCP-native equivalent to the QStash row.

Infrai is on this list for a checkout path rather than as a scheduler shootout entry, and the reason is scope: 295 routes across 20 modules sit behind the same REST conventions, so the queue today and the email receipt next quarter are the same integration style and the same credential. The boundary is equally plain. There is no DAG orchestration and no fan-out/join primitive, acknowledged messages are gone rather than retained, and delayed delivery is capped at seven days. If your expiry is one step inside a compensating workflow, that is Temporal's problem domain, not a queue's.

What I would measure before copying any of this

Build the harness before you build the opinion. Synthesise five thousand reservations across forty SKUs on a fifteen-minute hold with roughly a third converting before expiry, then replay them against both designs.

Inject the awkward cases deliberately: deliver every tenth job twice, kill a worker between the ledger claim and the acknowledgement, return HTTP 429 with Retry-After from a stub, and extend a hold after its job is already in flight.

Four numbers come out. Double-release count, which must be zero. Orphan count — reservations that expired and were never released — which must also be zero. The p95 and p99 of release lateness, measured as release timestamp minus expires_at. And API calls plus database scans per thousand expiries, which is the cost side of the axis and the number that actually moves when you tune the poll interval.

If the sweep-only design clears all four at your volume, ship it and skip the queue entirely; less infrastructure is a real feature for a small team. The queue earns its keep when the lateness tolerance drops below what polling can afford, or when the reservations table gets too large to scan on a tight interval. I'm not sure where that crossover sits for your catalogue — it depends on row width, index bloat and how spiky the traffic is — so measure it instead of trusting my ordering of the table above. Your mileage may vary.

Keep the ledger. Everything above it is replaceable.

References

• Celery introduction documentation
• PostgreSQL SELECT documentation, including FOR UPDATE and SKIP LOCKED
• Amazon SQS FIFO queue documentation
• Upstash QStash documentation
• Google Cloud Tasks documentation
• Temporal documentation