Duplicate requests aren’t edge cases - they’re normal behavior in distributed systems.
A client times out, retries, and suddenly your API creates:
- two payments
- two orders
- two subscriptions
Idempotency keys exist to prevent that. But many implementations still fail under real conditions.
The Problem
Consider POST /payments:
- Server processes the payment
- Response is lost (timeout, network issue)
- Client retries
Without idempotency, the retry looks like a new request → duplicate charge.
The Assumption That Breaks
A common approach is:
“Store the idempotency key and reject duplicates.”
This sounds correct—but it’s not.
Two concurrent requests can both:
- check for the key
- see nothing
- execute the side effect
Result: duplicates still happen.
What Actually Works
Idempotency is not just storing keys - it’s about ownership of execution.
The critical rule:
Only one request must be allowed to perform the operation.
This requires an atomic reservation, typically:
- SQL: unique constraint +
INSERT ... ON CONFLICT - Redis:
SET NX
Everything else builds on top of that.
The Minimum Safe Design
A correct implementation must:
- Atomically reserve
(scope, key) - Store a request fingerprint (to detect misuse)
-
Track state:
in_progresscompletedambiguous
Replay the original response on retries
Reject same key with different payload
Use a TTL that matches real retry behavior
The Hard Part: Ambiguous Failures
The real failure mode isn’t duplicates - it’s uncertainty.
Example:
- Payment provider accepts the charge
- Your service times out
- Client retries
You don’t know if the charge succeeded.
Retrying blindly can double-charge.
Safe systems:
- mark the request as ambiguous
- reconcile with downstream systems
- only finalize once certainty is restored
Practical Signals
If idempotency is working, you should see:
- replayed responses (normal)
- occasional in-progress conflicts
- rare payload mismatches
If not, expect:
- duplicate writes
- inconsistent downstream state
- hard-to-debug production issues
The Core Insight
Idempotency keys are not a cache.
They are a correctness boundary:
- they define ownership
- they prevent duplicate side effects
- they preserve system integrity under retries
Without atomic reservation and state modeling, they don’t actually solve the problem.
Full Article
For the complete breakdown (schema design, handler flow, TTL strategy, and failure cases):
👉 https://codenotes.tech/blog/api-idempotency-keys-prevent-duplicate-requests
Top comments (0)