A practical guide to duplicate execution, request fingerprints, concurrency, and safe retries.
A request timeout does not mean the operation failed.
Sometimes the server has already completed the operation, but the response never reaches the client.
Consider:
POST /orders/order-123/pay
with:
{
"amount": 500000
}
The timeline may look like this:
Client -> POST /pay
Server -> payment succeeds
Server -> response lost
Client -> timeout
Client -> retry
Server -> payment succeeds again
The problem is not the duplicate HTTP request itself. The problem is duplicate execution.
The target invariant
For a mutation API, the goal should be:
1 logical operation
=
1 effective side effect
even if the request is delivered multiple times.
Introduce an Idempotency Key
The client generates an identifier for one logical operation:
Idempotency-Key: 550e8400-e29b-41d4-a716-446655440000
The key must remain the same across retries.
same logical operation -> same key
new logical operation -> new key
The basic flow becomes:
Request
↓
Check Idempotency-Key
↓
New key?
↓
Process operation
↓
Store result
↓
Return response
On retry:
Same key
↓
Existing completed result
↓
Replay response
↓
Do not execute side effect again
Why check-then-act fails
This is not enough:
if key does not exist:
process payment
Two concurrent requests may both observe the key as missing:
A -> key not found
B -> key not found
A -> process
B -> process
This is a race condition. The uniqueness decision needs to be atomic.
For a relational database, that normally means enforcing the invariant in storage:
CREATE UNIQUE INDEX idx_idempotency_key
ON payments(idempotency_key);
Application checks are useful. The unique constraint is the final guard.
Same key, different payload
An idempotency key must represent exactly one logical operation.
This should be accepted:
key: abc
amount: 500000
retry
key: abc
amount: 500000
This should not:
key: abc
amount: 500000
retry
key: abc
amount: 800000
The lab uses a request fingerprint based on SHA-256:
func hashRequest(data []byte) string {
sum := sha256.Sum256(data)
return hex.EncodeToString(sum[:])
}
Then:
same key + same fingerprint
-> replay safely
same key + different fingerprint
-> 409 Conflict
PROCESSING vs COMPLETED
The safe flow uses two important states:
PROCESSING
COMPLETED
When a second request arrives:
PROCESSING
-> return 409
-> do not execute payment again
When the operation is already finished:
COMPLETED
-> load stored response
-> replay it
Idempotency is not a transaction
A database transaction provides atomicity inside one execution.
Idempotency protects a logical operation across multiple execution attempts.
These can both succeed:
Request A
BEGIN
create payment
COMMIT
Request B
BEGIN
create payment
COMMIT
The database is consistent. The business result is not. The customer was charged twice.
External side effects have a different boundary
Consider:
Call payment provider
↓
Provider SUCCESS
↓
Local commit fails
↓
ROLLBACK
The local rollback does not undo the external charge.
local rollback
!=
external rollback
If the provider supports provider-side idempotency, the backend should reuse a stable key for that external request too.
Lab implementation
The lab intentionally keeps the infrastructure small.
Unsafe:
func ProcessPayment(req PaymentRequest) (PaymentResult, error) {
return gateway.Charge(req)
}
Safe:
validate key
↓
calculate fingerprint
↓
reserve operation
↓
PROCESSING
↓
execute payment
↓
store result
↓
COMPLETED
The storage implementation uses:
map + sync.RWMutex
This simulates atomic uniqueness without introducing a real database into the lab.
Run both versions:
go test ./labs/01-idempotency/unsafe/... -v -count=1
go test ./labs/01-idempotency/safe/... -v -count=1
Final mental model
Retry is normal.
Duplicate execution is the problem.
Idempotency makes retry safe.
Source code:
https://github.com/lukman-ss/software-engineering-lab/tree/main/labs/01-idempotency
Top comments (0)