Idempotency Keys Explained: How to Make API Retries Safe (With Node.js and Redis Code)

A customer taps "Pay" on a slow mobile connection. The request reaches your server, the card is charged, and then the response gets lost on the way back. The app sees a timeout and does what every well-behaved client does: it retries.

Now the customer has been charged twice.

Nothing in that story is a bug in the usual sense. The server worked. The client worked. The network did what networks do. The problem is that the client cannot tell the difference between "the request never arrived" and "the request succeeded but I never heard back." Retrying is the only sensible move, so the server has to make retrying safe.

That is what idempotency keys are for. This post covers how they work, the race condition most first implementations miss, and working Node.js code you can adapt.

What idempotency actually means

An operation is idempotent if doing it once has the same effect as doing it many times.

Some HTTP methods are idempotent by design:

• GET /orders/42 reads data. Call it a hundred times and nothing changes.
• PUT /users/7 with a full body sets the user to that state. Repeating it gives the same state.
• DELETE /sessions/abc removes the session. The second call finds nothing to remove, and the end state is the same.

POST is the odd one out. POST /payments creates a new payment every time it runs. Two calls, two payments. The same goes for sending an email, placing an order, or issuing a refund.

You can't make these operations naturally idempotent, so you add a mechanism that lets the server recognise a repeat.

How idempotency keys work

The idea is simple:

1. The client generates a unique ID (a UUID is fine) for each logical operation, not for each HTTP attempt.
2. It sends that ID in a header, usually Idempotency-Key.
3. The server checks whether it has seen that key before.
4. If not, it processes the request and stores the response against the key.
5. If it has, it skips the work and returns the stored response.

POST /payments HTTP/1.1
Content-Type: application/json
Idempotency-Key: 7f3c2a9e-1b4d-4c8a-9e2f-5a6b7c8d9e0f

{ "amount": 4999, "currency": "INR", "orderId": "ord_812" }

The first request charges the card. Every retry with the same key gets the same response back, and the card is charged once.

The naive version and its race condition

Most first attempts look like this:

app.post("/payments", async (req, res) => {
  const key = req.get("Idempotency-Key");

  const cached = await redis.get(`idem:${key}`);
  if (cached) return res.json(JSON.parse(cached));

  const payment = await chargeCard(req.body); // slow
  await redis.set(`idem:${key}`, JSON.stringify(payment), "EX", 86400);

  res.json(payment);
});

It handles the easy case: a retry that arrives after the first request has finished.

It fails the hard case. Suppose the retry arrives while the first request is still inside chargeCard. Both requests run the get, both find nothing, and both charge the card. The check and the write are two separate steps, and anything can happen between them.

This is not rare. Aggressive client timeouts, double-clicks, and load balancer retries all produce two copies of a request within milliseconds of each other.

A safer Express middleware with Redis

The fix is to claim the key atomically before doing any work. Redis SET with the NX option does this: it writes only if the key doesn't exist, and tells you whether it won.

The middleware below also does two more things. It scopes keys to the user, so one customer can't collide with another. And it stores a hash of the request, so the same key reused with a different body is rejected instead of silently returning the wrong response.

const crypto = require("crypto");
const Redis = require("ioredis");

const redis = new Redis();

const LOCK_SECONDS = 60;          // how long a request may stay "processing"
const RESULT_SECONDS = 60 * 60 * 24; // how long a finished result is kept

function fingerprint(req) {
  return crypto
    .createHash("sha256")
    .update(req.method + req.originalUrl + JSON.stringify(req.body))
    .digest("hex");
}

function idempotency() {
  return async (req, res, next) => {
    const key = req.get("Idempotency-Key");
    if (!key) {
      return res
        .status(400)
        .json({ error: "Idempotency-Key header is required" });
    }

    const redisKey = `idem:${req.user.id}:${key}`;
    const hash = fingerprint(req);

    // Atomic claim: only one request can win this.
    const claimed = await redis.set(
      redisKey,
      JSON.stringify({ status: "processing", hash }),
      "EX",
      LOCK_SECONDS,
      "NX"
    );

    if (claimed !== "OK") {
      const raw = await redis.get(redisKey);
      const saved = raw ? JSON.parse(raw) : null;

      if (!saved || saved.status === "processing") {
        return res
          .status(409)
          .json({ error: "Request is already in progress. Retry shortly." });
      }

      if (saved.hash !== hash) {
        return res.status(422).json({
          error: "Idempotency-Key was already used with a different request",
        });
      }

      // Finished earlier: replay the stored response.
      return res.status(saved.code).json(saved.body);
    }

    // We own the key. Capture the response when the handler sends it.
    const sendJson = res.json.bind(res);
    res.json = (body) => {
      if (res.statusCode >= 500) {
        // Server failed: release the key so the client can retry.
        redis.del(redisKey).catch(() => {});
      } else {
        redis
          .set(
            redisKey,
            JSON.stringify({ status: "done", hash, code: res.statusCode, body }),
            "EX",
            RESULT_SECONDS
          )
          .catch(() => {});
      }
      return sendJson(body);
    };

    next();
  };
}

app.post("/payments", authenticate, idempotency(), async (req, res) => {
  const payment = await chargeCard(req.body);
  res.status(201).json(payment);
});

What each status code tells the client:

• 201 (or whatever the handler returned): done, whether this was the first attempt or a replay.
• 409: the original request is still running. Wait and retry with the same key.
• 422: the key was reused for a different request. This is a client bug, so don't retry.
• 5xx: the attempt failed and the key was released. Safe to retry.

When Redis is not enough: the database version

The middleware has one gap. The business write (the payment row) and the idempotency record live in two different systems. If the process crashes after the payment commits but before Redis is updated, the lock expires and a retry runs the handler again.

For most endpoints that window is acceptable. For money, it isn't. The stronger pattern is to store the key in the same database, in the same transaction, as the work itself:

CREATE TABLE idempotency_keys (
  user_id       BIGINT      NOT NULL,
  idem_key      TEXT        NOT NULL,
  request_hash  TEXT        NOT NULL,
  response_code INT,
  response_body JSONB,
  created_at    TIMESTAMPTZ NOT NULL DEFAULT now(),
  PRIMARY KEY (user_id, idem_key)
);

Inside one transaction, insert the key row, do the work, and save the response. The primary key makes the claim atomic: a second insert with the same key fails. And because everything commits or rolls back together, you can never end up with a payment that has no idempotency record.

If your handler calls an external provider such as a payment gateway, pass your key through to them as well. Most payment APIs accept an idempotency key of their own, which protects the one step your database transaction can't cover.

The client side: retrying with the same key

The server can only recognise a retry if the client sends the same key. Generate it once per operation, outside the retry loop:

async function createPayment(payload) {
  const key = crypto.randomUUID(); // once per operation, NOT per attempt

  for (let attempt = 0; attempt < 4; attempt++) {
    try {
      const res = await fetch("https://api.example.com/payments", {
        method: "POST",
        headers: {
          "Content-Type": "application/json",
          "Idempotency-Key": key,
        },
        body: JSON.stringify(payload),
      });

      // Anything except "still processing" or a server error is final.
      if (res.status !== 409 && res.status < 500) {
        return res.json();
      }
    } catch (err) {
      // Network error or timeout: fall through and retry.
    }

    // Exponential backoff: 0.5s, 1s, 2s, 4s
    await new Promise((r) => setTimeout(r, 2 ** attempt * 500));
  }

  throw new Error("Payment request failed after retries");
}

In a web or mobile app, create the key when the user opens the checkout screen or taps the button, and keep it until the operation finishes. That way a double-tap or an app restart mid-request still reuses the same key.

Common mistakes

Generating a new key on every retry. This is the most common one, and it defeats the whole mechanism. Each attempt looks like a brand-new request.

Check-then-set without atomicity. A separate GET followed by SET leaves the race condition shown earlier. Use SET NX or a unique constraint.

Not scoping keys to the user. A global key namespace lets one client replay another client's response if keys collide or are guessed.

Ignoring the request body. If you don't compare a request hash, a client that reuses a key by mistake gets back a response for a completely different request.

Caching server errors. If you store a 500, the client can never recover with that key. Store successes and client errors; release the key on server errors.

No expiry. Keys kept forever grow without limit. 24 hours is a common retention window, since retries after that are almost never the same logical operation.

A lock that outlives nothing. If the "processing" marker has no TTL and the process crashes, the key is stuck in 409 forever. Always give the lock an expiry longer than your slowest request.

Checklist before you ship

• Every non-idempotent write endpoint (payments, orders, emails, refunds) requires an Idempotency-Key.
• The key is claimed atomically before any work starts.
• Keys are scoped per user or per API client.
• The request hash is stored and compared on replay.
• In-progress requests return 409, mismatched bodies return 422.
• Server errors release the key; successes are stored with a TTL.
• Clients generate the key once per operation and reuse it on every retry.
• For money, the key is written in the same transaction as the business data.

Idempotency is one of those things nobody notices when it works and everybody notices when it doesn't. It costs a few dozen lines of code up front, and it removes a whole class of duplicate-charge and duplicate-order bugs that are painful to clean up afterwards.

If you're building payment flows, order systems, or any API where a retry must not repeat the work, the team at Prism Infoways designs and builds backends with this kind of reliability built in from the start.

How do you handle retries in your APIs? Share your approach in the comments.