Designing Idempotent APIs

Bibek Ray — Tue, 15 Oct 2024 06:55:05 +0000

Say user A make an API call to the backend and there was some issue with the network and hence the call failed!

What to do ?

We ignore and move forward
We pass the observed error to the use
We retry on our own (better user experience hence preferred)

💡 Depending on what we are building, we pick one of these ways to handle

Failure Timeline

Client Know that the request didn’t even reach the server, hence safe to retry.

Failure while server is executing. Client cannot be sure of retrying.

Server complete the execution but the client didn’t got the response. Client cannot be sure of retrying.

What could go wrong if we always retry?

Say the API was about transferring money form A to B.

/payment/B → { amount: 10000 }

If the API call was processed successfully, but we still retried, then the amount will be re-deducted.

For n retries, the amount deducted = $10000 * n

Payments and other critical APIs need to ensure idempotency, So that we safely retry.

Idempotent API

Core idea of building an idempotent API is that the server should somehow know that it has seen this request earlier.

if seeing it the 1st time
        proceed
otherwise
        ignore/throw error

What we are trying to achieve is Exactly Once Semantics.

The way to achieve this is Idempotence Keys

Client first talks to the server to generate a random ID
- The Id may be operation specific e.g.: “Money transfer”
Client passes this ID along with regular payload to do actual operation

Server checks the ID and validate if it has already handled it.

if already handled it
        ignore/throw error
if not
        handle it

If client sees the error. then it retries the request same ID
Server extracts the ID, validates, and then decides to handle it, resume or ignore

How to pass the idempotency key?

Request Header
Query Parameter

💡 Stripe requires client to pass Idempotency key in Request Header “Idempotency-Key”

Architecture

Server maintains all idempotency keys in a database.
When operation is successful. The server mark as checked or delete the key
For every request, server checks the DB for the key

More Here : Stripe Engineering Blog
Reference : Youtube

Fundamental of internet networking

Bibek Ray — Wed, 09 Oct 2024 12:38:25 +0000

Internet

Connection of multiple computer in network to share data.

Data is share in air (radio wave form) that goes to nearest tower (ISP → internet service provider)

IP Address

A unique address of a device that is assign when device is connected to internet. It is given by ISP

IPv4
IPv6

MAC Address

Actual physical address of device. Help to identify device in local network like WIFI router help to identify the device with MAC address.

Proxy

If A want to talk to B, then instead of directly communicating A will send message it C and C will send message to B and vice versa.

So, instead of share data directly to the server, we share data to a third server (proxy) and that proxy will share data with targeted server.

Used to access those server where direct interaction is banned or not allowed.

It is on client side.

Reverse Proxy

This is on server side. When client request the server, the request doesn't directly goes to main server it goes to the some third server where the request is checked either it is malicious or not. And after that the request is sent to me main server.

Mostly used for rate limiting.

VPN

Virtual private network, this is very much similar to the proxy i.e. A will use C to send data to B, but in VPN, data that is shared is encrypted and ISP can’t read those data.

Used to maintain highest level of privacy on internet and make user anonymous on internet.

Protocol

A set to rule that is used to share data on internet.

Server

A computer system that is programmed in the way that it can accept request and give response or share file based on request.

Client

A normal computer system that fetch data from server.

ISP

Internet service provider, this is the pillar that help to transfer data from one place to other via wires, hub, tower

TCP

When data is sent to the server, the server acknowledge with the message weather the message is failed or delivered. It guarantee the data is delivered or not.

IP

This is the protocol through which the an IP address (unique) is assigned to the device when it is connected to internet and once the device is disconnected from internet, the IP address is released which can be assign to some other device.

UDP

Protocol that continually sent data without caring weather the data is delivered or not. It is very fast process of sending data. Generally used during video streaming.

DEV Community: Bibek Ray