Every API call over a network can fail in a way that tells you nothing. The request might never have reached the server. It might have reached the server, been processed successfully, and then the response got lost on the way back. From the client's point of view, both look identical: a timeout.
The natural instinct is to retry. For a search query, that's harmless, worst case you run the same read twice. For a payment, a signup, or anything that changes state, a naive retry can charge a customer twice, send a duplicate email, or create two identical orders from one click. Idempotency keys are the standard fix, and they show up in almost every serious payments API for a reason.
The core idea
An idempotency key is a unique value the client generates and attaches to a request, usually as a header, like Idempotency-Key: 8f14e45f-ceea-467e-bd53-.... The server uses that key to recognize "I've seen this exact request before" and, instead of processing it again, returns the same result it returned the first time.
The client doesn't need to know whether its earlier request succeeded, failed, or is still in flight. It just retries with the same key, and the server guarantees the underlying operation happens at most once.
This is different from an operation simply being idempotent in the mathematical sense (like PUT, which is supposed to produce the same end state no matter how many times you call it). A POST /charges call is never naturally idempotent, calling it twice should, by default, create two charges. An idempotency key is what lets you bolt idempotent behavior onto an inherently non-idempotent operation.
What happens on the server
A typical implementation looks like this:
When a request arrives with an idempotency key, the server checks a store (usually a database table or a fast key-value store) for that key.
If the key isn't there, the server records it, marks it "in progress," processes the request normally, then stores the response body and status code against that key.
If the key is there and the original request finished, the server returns the stored response immediately without re-running any of the business logic.
If the key is there and the original request is still in progress (a second request arrived while the first was mid-flight), the server should reject or hold the second request rather than let both proceed concurrently. Otherwise you get a race that idempotency keys were supposed to prevent in the first place.
Stripe's implementation is a good reference point: it stores the key alongside a hash of the request parameters, the response, and the status, and reuses that response for up to 24 hours on a matching key.
Handling the edge cases
A few details separate a correct implementation from one that just looks correct in the happy path:
Same key, different payload. If a client sends the same idempotency key with a different request body than before, that's almost certainly a bug on the client side: reusing a key across unrelated requests. Most APIs treat this as an error rather than silently processing the new payload, since guessing which version the client "really" meant is worse than failing loudly.
Concurrent requests, same key. Two requests with the same key can genuinely arrive at nearly the same time, for example, a mobile client retries after a slow response that eventually does arrive. The server needs some form of locking or a unique constraint at the database level on the key column so that only one of the two ever executes the underlying logic; the other should wait and then return the first one's result.
Expiration. Keys shouldn't live forever. Stripe expires them after 24 hours; other systems use shorter or longer windows depending on how long a client might realistically retry. Whatever the window, it needs to outlast the client's own retry logic, or a legitimate retry after the key expires will double-process.
Where the key comes from. The client should generate the key once, before the first attempt, and reuse the exact same value on every retry of that same logical operation, not generate a new key per HTTP call. A UUID v4 generated at the point the user clicks "submit" is the usual pattern.
Beyond payments
Idempotency keys aren't only useful for charging a card. The same pattern applies anywhere a retry could duplicate a side effect: webhook delivery (many providers, including Stripe and GitHub, recommend deduplicating incoming webhooks by event ID for the same reason), message queue consumers that might redeliver a message, and background job systems where a worker crash mid-task could cause a job to run twice. Anywhere "at-least-once delivery" meets "this operation has a side effect," idempotency keys (or the same idea under a different name) tend to show up.
There's also now a proposed IETF standard (draft-ietf-httpapi-idempotency-key-header) working to formalize the Idempotency-Key header across APIs generally, rather than leaving every provider to define its own semantics. It's still a draft, but it's a sign the pattern has moved from "a good idea a few payment companies do" to something closer to a general HTTP convention.
Top comments (0)