Retries are one of those things that look harmless until the first time they duplicate a real business operation.
A request times out, so the client retries it.
Reasonable.
But what if the first request actually reached the server?
What if the application already created the order, reserved the stock, sent the message, or called a payment provider — and only the response was lost?
From the client's point of view, the request failed.
From the application's point of view, it may already be finished.
Send the same request again and you can get the worst kind of bug: one that is technically understandable, difficult to reproduce, and very expensive in production.
This is the problem that pushed me to build HttpIdempotencyBundle, a small Symfony bundle for explicit HTTP request idempotency.
But the interesting part is not the bundle itself.
The interesting part is everything that has to be true before we can safely say:
"This request is a retry of the same operation, so we should not execute it again."
And just as importantly, what we cannot guarantee.
A timeout does not mean the operation failed
Consider a simple endpoint:
#[Route('/orders', methods: ['POST'])]
public function createOrder(): JsonResponse
{
$order = $this->orderService->create();
return new JsonResponse([
'id' => $order->getId(),
], 201);
}
Now imagine this sequence:
Client -> POST /orders
Server -> creates order #742
Server -> sends 201 response
Network -> connection dies
Client -> sees timeout
Client -> retries POST /orders
Nothing unusual happened.
The client did exactly what clients often do after a timeout.
The server did exactly what it was asked to do.
And yet, unless we have another mechanism in place, we may now create order #743 as well.
The key idea is simple:
transport failure and business-operation failure are not the same thing.
HTTP cannot always tell the client whether the operation happened.
Give the operation an identity
A common solution is an Idempotency-Key.
The client generates a unique value for one logical operation and sends it with every retry:
POST /orders HTTP/1.1
Content-Type: application/json
Idempotency-Key: order-7f98b773
{"sku":"ABC-42","quantity":1}
If the request must be retried, the same key is reused.
That gives the server a stable identity to work with.
At first, the implementation seems obvious:
if ($store->has($key)) {
return $store->get($key);
}
$response = $controller();
$store->save($key, $response);
return $response;
Unfortunately, this is not enough.
There are several traps hiding inside those few lines.
A key alone is not enough
What should happen if a client accidentally reuses the same key for a different request?
First request:
Idempotency-Key: order-7f98b773
{"sku":"ABC-42","quantity":1}
Later:
Idempotency-Key: order-7f98b773
{"sku":"XYZ-99","quantity":10}
If we only store the key, the second request might receive the response from the first one.
That would be worse than simply failing.
The server must therefore know not only which key was used, but also which request that key belongs to.
This is where request fingerprinting becomes important.
Fingerprint the request, not just the key
A deterministic fingerprint can describe the concrete request.
For HttpIdempotencyBundle, the default fingerprint includes the important parts of the request, such as:
- HTTP method
- logical operation
- path
- normalized query parameters
- normalized
Content-Type - raw request body
The result is represented as a SHA-256 digest.
Conceptually:
POST
/orders
application/json
{"sku":"ABC-42","quantity":1}
becomes:
sha256(...)
Now there are two very different cases.
Same key, same fingerprint
This is the same logical request again.
If the first request already completed, we can replay its stored response.
Same key, different fingerprint
The key is being reused for another request.
That should not silently replay anything.
In the bundle, this becomes:
422 Unprocessable Content
This is an important detail.
Idempotency should protect us from duplicate execution, not hide application bugs.
The race condition in the obvious solution
Now we get to the part that makes idempotency a concurrency problem.
Imagine two identical requests arrive at almost the same time.
Worker A:
read -> no record
Worker B:
read -> no record
Worker A executes the controller.
Worker B executes the controller too.
Both workers followed the logic correctly.
The result is still wrong.
A cache lookup alone cannot serialize concurrent execution.
We need coordination.
Shared state and a shared lock
The bundle uses two separate pieces:
- shared idempotency state
- Symfony Lock
Both must really be shared across the application.
That distinction matters in production.
If you run multiple PHP workers, containers, or application nodes, a local in-memory cache or a local filesystem lock cannot coordinate all of them.
Redis is one practical option:
framework:
cache:
pools:
cache.http_idempotency:
adapter: cache.adapter.redis
provider: '%env(REDIS_URL)%'
lock:
http_idempotency: '%env(REDIS_URL)%'
http_idempotency:
cache_pool: cache.http_idempotency
lock_factory: lock.http_idempotency.factory
Redis itself is not the requirement.
Shared visibility is the requirement.
Every application instance participating in the same operation must see the same idempotency state and the same locking mechanism.
The second read is easy to miss
There is a subtler race condition.
Suppose request A and request B both perform their first read before either one obtains the lock.
A: read -> no record
B: read -> no record
A gets the lock first.
It executes the controller, stores the completed response, and releases the lock.
B then acquires the lock.
If B trusts its old read, it still believes there is no record.
So it could execute the controller again.
The fix is small but essential:
Read the idempotency record again after acquiring the lock.
The simplified flow becomes:
read record
completed + same fingerprint
-> replay
completed + different fingerprint
-> 422
try to acquire lock
lock unavailable
-> 409
lock acquired
read record again
completed + same fingerprint
-> replay
completed + different fingerprint
-> 422
still no record
-> save processing marker
-> execute controller
-> store completed response
-> release lock
That second read closes the race between the initial lookup and lock acquisition.
It is only one extra read.
It is also one of the most important reads in the whole implementation.
Why the lock is non-blocking
There is another design choice: should a duplicate request wait until the first request finishes?
It can.
But that also means tying up a worker while another request is running, with latency controlled by somebody else's operation.
HttpIdempotencyBundle uses non-blocking lock acquisition.
If another request is already processing the same idempotency identity, the duplicate request gets:
409 Conflict
The client can then decide whether and when to retry.
I prefer this because the behavior is explicit and request latency remains predictable.
"Same key" should not mean "same key globally"
Suppose two authenticated users both happen to send:
Idempotency-Key: request-123
They should not collide with each other.
So the real identity cannot be just the raw idempotency key.
Conceptually, the bundle derives an identity from:
scope + operation + key
The operation tells us which protected application action this belongs to.
The scope separates independent principals.
For authenticated Symfony users, the default scope includes the user identity.
Anonymous requests use a shared anonymous scope by default.
Applications that need a different definition can provide their own scope resolver.
This matters especially for endpoints where authentication happens somewhere else in the stack or where multiple logical principals share the same anonymous HTTP context.
Response replay is more complicated than serialization
Suppose the first execution returns:
HTTP/1.1 201 Created
Content-Type: application/json
{"id":742}
An identical retry should be able to receive that application result without running the controller again.
But an HTTP response contains more than application data.
Some headers belong to the original connection or original request environment.
For example:
ConnectionContent-LengthDateServerTransfer-EncodingSet-Cookie
Blindly storing and replaying the whole response would copy things that should be generated again.
Cookies are particularly sensitive here.
A response cookie from the original request should not accidentally become part of a cached idempotency replay.
The bundle therefore stores a controlled response snapshot rather than serializing the entire Symfony Response object.
On replay, normal Symfony response listeners still get a chance to generate fresh request-specific behavior.
This keeps the idempotency layer focused on the reusable application result.
What about application errors?
Here is another question that does not have an obvious answer at first:
If the controller runs and returns a 4xx or 5xx response, should a retry execute the controller again?
Not necessarily.
The important distinction is whether the application operation completed versus whether the idempotency infrastructure failed.
If the controller genuinely executed and returned an application response, the bundle treats that as a completed execution that can be replayed.
That avoids turning every application error into another execution attempt.
Infrastructure failures are different.
If the idempotency store or lock backend fails before the controller can safely run, the bundle fails closed:
503 Service Unavailable
If an endpoint explicitly requires idempotency protection, silently continuing without that protection would be a dangerous fallback.
A processing marker is useful after crashes
A lock protects concurrent execution while the process is alive.
But processes crash.
The application can die after marking the operation as in progress and before storing the completed response.
For that reason, the bundle also uses a processing marker with a TTL.
The marker is intentionally not eagerly removed during abnormal cleanup.
Its expiration creates a bounded recovery window.
During that period, retries do not immediately re-execute an operation whose previous execution may still have produced an external side effect.
This does not eliminate every failure mode.
It makes the failure behavior more controlled.
The part that should never be hidden: this is not exactly-once
This is the most important limitation in the whole design.
Imagine:
1. controller calls an external payment provider
2. payment succeeds
3. PHP process crashes
4. completed idempotency response is never stored
The business side effect happened.
The HTTP idempotency layer did not get the chance to record completion.
After the processing marker eventually expires, a retry may execute the operation again.
That failure window exists.
A middleware, bundle, cache, or distributed lock cannot simply wish it away.
So HTTP idempotency should not be sold as "exactly-once execution".
For important writes, it should be combined with domain-level guarantees such as:
- database unique constraints
- transactions
- provider-level idempotency
- transactional outbox patterns
- durable jobs
- domain-specific deduplication
For example, if a payment provider supports its own idempotency key, use it.
The HTTP idempotency layer and the provider-level guarantee solve related but different parts of the problem.
This is one reason I wanted the limitation to be prominent in the documentation rather than hidden in a footnote.
Making the choice explicit in Symfony
I also did not want the bundle to automatically change every POST, PUT, or PATCH endpoint.
Idempotency is an application decision.
It should be visible in the controller:
use Alkin\\HttpIdempotencyBundle\\Attribute\\Idempotent;
#[Route('/orders', methods: ['POST'])]
#[Idempotent]
public function createOrder(): JsonResponse
{
// Perform the application operation.
return new JsonResponse([
'created' => true,
], 201);
}
Unmarked controllers are not touched.
That makes the behavior easy to discover during code review.
You can look at the endpoint and immediately see that retries are part of its contract.
Failure behavior should be boring
One of my goals was to make the failure model small enough to reason about.
For protected controllers, the bundle uses a few clear outcomes:
| Status | Meaning |
|---|---|
400 Bad Request |
Missing or invalid idempotency key |
409 Conflict |
The same operation is already processing, or its lock cannot be acquired |
422 Unprocessable Content |
The key was reused for a different request fingerprint |
503 Service Unavailable |
The idempotency storage or locking infrastructure cannot safely protect the operation |
The interesting thing is not the exact status code list.
It is that the implementation tries to fail explicitly rather than silently weakening the guarantee.
Why I turned this into a bundle
The original problem sounds small:
"Do not execute the same retried HTTP request twice."
But following it far enough touches:
- request identity
- request fingerprinting
- concurrency
- distributed locking
- shared state
- response replay
- HTTP headers
- authenticated scope
- process crashes
- failure recovery
- the boundary between transport guarantees and business guarantees
That made it a useful problem to isolate into a small Symfony component.
The first stable release is now available as:
composer require alkinbg/http-idempotency-bundle
It supports:
PHP >= 8.2
Symfony 7.4 or 8.1
The public repository is here:
github.com/alkinbg/http-idempotency-bundle
And the package is on Packagist:
packagist.org/packages/alkinbg/http-idempotency-bundle
Final thought
Retries are not the enemy.
They are a normal part of distributed systems.
The dangerous part is pretending that a timeout tells us what happened on the other side.
A good idempotency mechanism gives the server enough information to recognize the same logical request, enough coordination to stop concurrent duplicate execution, and a clear enough failure model that we still understand what can go wrong.
That last part matters.
Because in distributed systems, the most useful guarantee is often not the one that sounds strongest.
It is the one whose limits you can explain.
Top comments (0)