What Is Idempotency? A Practical Guide for API Developers

#api #architecture #backend #softwareengineering

A user clicks "Pay Now." The request times out. No confirmation screen. So they click again. Totally reasonable thing to do. And now their card has been charged twice.

This isn't some weird edge case. It happens all the time in production. Network blips, client retries, at-least-once message delivery, webhook providers re-firing because they didn't get an ACK fast enough. The same request hits your server more than once, and your code happily processes it each time.

The fix has a name: idempotency.

🛠️ What idempotent actually means

An operation is idempotent if running it multiple times produces the same server-side effect as running it once. That's it. From RFC 9110 §9.2.2.

The response can differ though. A DELETE that returns 200 the first time and 404 the second is still idempotent. The resource is gone either way. Same effect, different status code.

People mix this up with safe. A safe method doesn't change state at all. GET is safe. PUT is not safe (it changes stuff) but it is idempotent (doing the same PUT ten times leaves you in the same state as doing it once).

So: all safe methods are idempotent. Not all idempotent methods are safe.

Here's the HTTP method breakdown per RFC 9110 §9.2.2:

Method	Safe	Idempotent
GET	yes	yes
PUT	no	yes
DELETE	no	yes
POST	no	no

POST is the odd one out. Each call can create a new resource or fire a new side effect. There's nothing in the protocol that prevents it. And that's exactly why payment endpoints (almost always POST) need extra work to become idempotent.

⚡ The idempotency key pattern

Stripe popularized this approach and it's become the industry standard for making POST endpoints safe to retry.

The idea: the client generates a unique key (a UUIDv4 works fine) before sending the request and passes it in an Idempotency-Key header. The server uses that key to detect replays.

On the first request, the server processes normally and stores the key alongside the response. On a retry with the same key and same parameters, it returns the stored response without re-executing anything. No double charge.

But what if someone sends the same key with different parameters? That's a bug on the client side, and the server returns a 409 Conflict. You don't want to silently return a cached response when the request body doesn't match. That would mask real errors.

Stripe expires keys after 24 hours. After that, a new execution happens.

Here's a stripped-down middleware that does this:

async function idempotency(req: Request, res: Response, next: NextFunction) {
  const key = req.headers["idempotency-key"] as string;
  if (!key) return next();

  const fingerprint = hash(req.body);
  const cached = await db.query(
    `SELECT fingerprint, status, body FROM idempotency_keys
     WHERE key = $1 AND created_at > NOW() - INTERVAL '24h'`,
    [key]
  );

  if (cached.rows.length) {
    if (cached.rows[0].fingerprint !== fingerprint) {
      return res.status(409).json({ error: "Key reused with different params" });
    }
    return res.status(cached.rows[0].status).json(JSON.parse(cached.rows[0].body));
  }

  // Wrap res.json to capture the response for storage
  const origJson = res.json.bind(res);
  res.json = (data: any) => {
    db.query(
      `INSERT INTO idempotency_keys (key, fingerprint, status, body)
       VALUES ($1, $2, $3, $4) ON CONFLICT (key) DO NOTHING`,
      [key, fingerprint, res.statusCode, JSON.stringify(data)]
    );
    return origJson(data);
  };
  next();
}

Notice the ON CONFLICT (key) DO NOTHING at the bottom. That's doing the real heavy lifting.

🧠 The database constraint is the actual guard

Here's the mistake I see constantly. People write logic like this:

Check if the key exists (SELECT)
If not, process the request
Insert the key

Looks reasonable. Totally broken in practice.

Two identical requests arrive 5ms apart. Both hit step 1, both see "no key exists," both proceed to step 2. You've just processed the payment twice. This is a classic TOCTOU race — time-of-check to time-of-use.

The fix is to let the database handle atomicity. A UNIQUE constraint on the idempotency key column means only one insert can ever succeed. The second one fails, and your code catches that conflict:

INSERT INTO payments (idempotency_key, amount, customer_id, status)
VALUES ('ord_123_pay', 4999, 'cus_abc', 'completed')
ON CONFLICT (idempotency_key) DO NOTHING;

One statement. Atomic. No race condition. The database's locking mechanism handles concurrent access for you. If you're building anything that processes payments, webhook events, or queue messages, this pattern should be your default.

And yeah, retry strategies (exponential backoff, jitter, circuit breakers) and distributed transactions are their own topics. They work alongside idempotency but don't replace it. Separate posts for those.

If your API sits behind an API gateway, the gateway might handle retries automatically. Another reason your handlers need to be idempotent even when you don't think you're retrying.