Designing Idempotent APIs

Say user A make an API call to the backend and there was some issue with the network and hence the call failed!

What to do ?

1. We ignore and move forward
2. We pass the observed error to the use
3. We retry on our own (better user experience hence preferred)

💡 Depending on what we are building, we pick one of these ways to handle

Failure Timeline

Client Know that the request didn’t even reach the server, hence safe to retry.

Failure while server is executing. Client cannot be sure of retrying.

Server complete the execution but the client didn’t got the response. Client cannot be sure of retrying.

What could go wrong if we always retry?

Say the API was about transferring money form A to B.

/payment/B → { amount: 10000 }

If the API call was processed successfully, but we still retried, then the amount will be re-deducted.

For n retries, the amount deducted = $10000 * n

Payments and other critical APIs need to ensure idempotency, So that we safely retry.

Idempotent API

Core idea of building an idempotent API is that the server should somehow know that it has seen this request earlier.

if seeing it the 1st time
        proceed
otherwise
        ignore/throw error

What we are trying to achieve is Exactly Once Semantics.

The way to achieve this is Idempotence Keys

1. Client first talks to the server to generate a random ID
   • The Id may be operation specific e.g.: “Money transfer”
2. Client passes this ID along with regular payload to do actual operation

3. Server checks the ID and validate if it has already handled it.
   

   if already handled it
           ignore/throw error
   if not
           handle it
   

4. If client sees the error. then it retries the request same ID

5. Server extracts the ID, validates, and then decides to handle it, resume or ignore

How to pass the idempotency key?

1. Request Header
2. Query Parameter

💡 Stripe requires client to pass Idempotency key in Request Header “Idempotency-Key”

Architecture

1. Server maintains all idempotency keys in a database.
2. When operation is successful. The server mark as checked or delete the key
3. For every request, server checks the DB for the key

More Here : Stripe Engineering Blog
Reference : Youtube