Idempotency is one of those words that shows up everywhere: system design interviews, API docs, architecture diagrams, and postmortems. Everyone nods when it’s mentioned. Few teams actually implement it correctly.
I’ve seen idempotency “implemented” with comments, flags, retries, or blind faith in the network. And I’ve also seen real production outages that could have been avoided with a proper idempotency strategy.
This article is about what idempotency really means in practice, where it breaks down, and how to implement it in real systems—not slides.
What Idempotency Actually Means (Beyond the Definition)
The textbook definition:
An operation is idempotent if performing it multiple times has the same effect as performing it once.
That’s technically correct but operationally incomplete.
In real systems, idempotency is about protecting your system from duplicates caused by retries, timeouts, race conditions, and partial failures.
If your API client retries because:
- the network dropped,
- the load balancer timed out,
- the user double-clicked,
- a worker crashed mid-request,
your system should not:
- charge twice,
- create duplicate records,
- send duplicate emails,
- corrupt state.
That’s the real problem idempotency solves.
Where Idempotency Matters (and Where It Doesn’t)
Common places it does matter
- Payments and billing
- Order creation
- Inventory reservations
- Webhooks
- Background jobs and queues
- External API integrations
Places it usually doesn’t
- Pure reads (
GET) - Analytics counters (if approximate is acceptable)
- Fire-and-forget logging
If an operation changes money, state, or inventory, idempotency is not optional.
The Biggest Misconception: “POST Is Not Idempotent”
You’ll often hear:
GET is idempotent, POST is not.
That’s an HTTP convention, not a system guarantee.
You can (and often must) make POST requests idempotent at the application level. Payments APIs do this all the time.
The HTTP method does not save you. Your backend design does.
The Naive (and Wrong) Implementations
1. “We Just Retry”
Retries without idempotency multiply bugs.
If your payment service retries blindly and your backend creates a new record each time, you’ve just automated duplication.
Retries are only safe after idempotency is enforced.
2. “We Check If It Exists”
Example:
SELECT * FROM orders WHERE user_id = ?
IF NOT EXISTS → INSERT
This fails under concurrency.
Two requests arrive at the same time:
- both see “not exists”
- both insert
- congratulations, you have duplicates
This is a race condition, not idempotency.
3. “We Use a Boolean Flag”
Flags like is_processed = true work only if:
- the operation is atomic
- the write is guaranteed to happen once
- there’s no partial failure
In distributed systems, those assumptions rarely hold.
The Correct Mental Model
Idempotency is about deduplicating intent, not requests.
You don’t care how many times a request arrives.
You care that the same logical action is processed once.
That’s why idempotency keys exist.
Idempotency Keys: The Foundation
An idempotency key is a client-generated unique identifier for a logical operation.
Example:
Idempotency-Key: 7f3c9c2e-...
Rules:
- Same logical action → same key
- Different action → different key
- Client must retry with the same key
A Minimal, Correct Implementation
Step 1: Store the key
Create a table or cache entry:
idempotency_key
request_hash
response
status
created_at
Step 2: Enforce uniqueness
The key must be unique at the database or cache level.
This prevents race conditions.
Step 3: On request
- Check if the key exists
- If yes:
-
return the stored response
- If no:
process the request
store the response
return it
This ensures:
- retries are safe
- duplicates are impossible
- responses are consistent
Handling Partial Failures (The Hard Part)
What if:
- the operation succeeds,
- but the response fails to return?
Without idempotency, the client retries and triggers a duplicate.
With idempotency:
- the retry hits the same key
- you return the stored success response
- the system remains consistent
This is where most “implementations” fail—they don’t store the response.
Real-World Example: Payments
Payment providers treat idempotency as mandatory.
Why?
- Networks are unreliable
- Clients retry aggressively
- Money cannot be duplicated
A payment API without idempotency is reckless.
The same logic applies to:
- order creation
- inventory deduction
- subscription changes
Performance and Storage Trade-offs
Idempotency isn’t free.
Costs:
- extra storage
- lookup overhead
- TTL management
Optimizations:
- Use Redis with TTL for short-lived operations
- Use DB for long-lived financial records
- Expire keys safely (not too early)
Never trade correctness for micro-optimizations here.
Security Considerations
- Treat idempotency keys as untrusted input
- Scope keys per user or client
- Avoid global collisions
- Don’t allow attackers to replay sensitive actions indefinitely
Idempotency improves safety, but careless design can introduce replay risks.
Final Thoughts
Idempotency is not a buzzword. It’s a discipline.
If your system:
- retries requests,
- talks over a network,
- processes money or state,
then idempotency is part of your correctness model, not an “extra feature.”
Most bugs blamed on “network issues” are actually idempotency failures.
Implement it once. Implement it right. Your future self—and your on-call rotation—will thank you.
Top comments (0)