Make generation quotas safe to retry

Any web action that calls a slower service has to handle a familiar failure: the server may finish the work, but the response can disappear before the browser receives it.

If a user retries and the backend treats the retry as a new operation, the same request may consume quota twice or create duplicate output. A button being clicked once does not guarantee one request reached the server.

A useful design pattern is to give each intended operation a stable idempotency key. The server records that key before starting expensive work. A repeated request with the same key returns the existing operation state instead of starting another one.

One possible state model is:

reserved -> completed
         -> failed

The reservation should be scoped to the user and the intended operation. A separate operation ID can let the client ask for status after a timeout. The UI can then show “still processing,” “completed,” or “failed” based on the server’s recorded state instead of guessing from the button click.

The tricky case is an upstream timeout. A timeout does not always mean the provider did no work. Before releasing a reservation or creating a new provider request, check whether the original operation can be found or safely retried with the same idempotency key. Otherwise a timeout can turn into duplicate work.

This is a design pattern, not a claim about any particular product implementation. The right quota rule depends on the product promise: whether a failed attempt is charged, whether pending work reserves capacity, and how long an operation may remain unresolved. Make those choices explicit, then test concurrent starts, browser retries, provider timeouts, and delayed responses.

I’m Prem, founder of Zextha. This post was prepared with AI assistance.

Comments

[Launch Gate]

The reserved state deserves its own fixture: reserve a key, then never come back (tab closed before the response arrived). Does that reservation hold quota forever, or expire? A TTL plus a reconcile path on the next request with the same key - check the provider for the original operation before charging a new one - separates "user gave up" from "response lost." Two more cases I'd test before trusting the model: two concurrent first requests carrying the same key (is reservation one atomic insert, or read-then-write where both pass the check?), and an upstream timeout where a provider-side lookup finds the work actually completed - does the reservation flip to completed, or stay failed while the output exists? Not a critique of the pattern; these are the cases where "safe to retry" quietly becomes double quota.