DEV Community

Cover image for Stop Charging Customers Twice - Idempotency Keys in Express for Beginners
FredAbod
FredAbod

Posted on Edited on

Stop Charging Customers Twice - Idempotency Keys in Express for Beginners

So you added payments to your API. It works. The test card goes through. Postman is green. You are feeling like a fintech engineer. 😎

Then one evening the network gets slow.

Your server calls the payment provider (the company that actually charges the card for you, like Stripe or Paystack). The provider charges the card. The answer is on its way back... and your request times out first.

Your code sees an error. So it does the "responsible" thing and retries.

The customer gets two debit alerts. 😭

And the customer is not sending you a nice message.

Shocked

If you have ever stood at a POS stand in Lagos, you already know this story. You want to withdraw ₦5,000. The agent slots your card. The machine thinks... and thinks... "Transaction timed out". The agent says "Oga, let's try again". Machine prints a slip. Then your phone buzzes. Debit alert. Buzz again. Debit alert. πŸ‘€

The machine did not fail. It was just late with the answer. The second try was a brand new withdrawal.

That is exactly the bug we are fixing today. The fix is a small thing called an idempotency key, and it is one of the most important habits in payment code.

At Prooval I built the payment flows on Stripe, Paystack and Fincra, and making them idempotent was a big part of that work. Today you'll see why.

What does "idempotent" even mean? πŸ€”

Big word, simple idea: doing it twice has the same effect as doing it once.

  • Pressing the lift button 10 times still brings one lift. Idempotent. βœ…
  • Sending a ₦5,000 transfer 10 times sends ₦50,000. Not idempotent. ❌

Creating a payment is the second kind. So we have to make it safe to retry.

An idempotency key is a unique string your server sends along with the request (usually in an Idempotency-Key header). The provider remembers the key. If the same key arrives again, it doesn't charge again. It just replays the first answer.

Back to the POS stand: imagine every withdrawal had a slip number written down before the agent touched the machine. If the machine hangs, the agent retries with the same slip number, and the bank says "I've seen slip 0042 already, here's what happened." One withdrawal. One debit alert. Peace. πŸ₯°

The trick (and where most people get it wrong) is when you create that key, where you keep it, and which errors are worth retrying with it.

Prerequisites

  • Node.js 20 or newer (I used Node 22)
  • Basic Express
  • A little TypeScript helps. If you only know JavaScript, read things like type Charge = { ... } as "a description of what this object looks like". They disappear when the code is compiled.
  • curl (or Postman if you prefer clicking)

No real payment account needed. We'll build a tiny fake payment provider that behaves like the real ones on the important part: same key + same body = same answer. I'm being honest here: it is NOT Stripe or Paystack, it's a pretend provider so you can break things safely on your laptop.

Quick cheat sheet: status codes πŸ”’

Every HTTP reply comes with a number that says how it went. You already know 200 (OK) and 404 (not found). Here are the others you'll see today:

  • 201 Created: like 200, but "I created something new" (here: a charge). βœ…
  • 4xx (400 to 499): "No, there's a problem with your request." For example 402 (payment not accepted), 409 (conflict: "busy with this one"), 422 ("this doesn't match what you sent before"), 429 ("too many requests, slow down").
  • 5xx (500 to 599): "Something broke on my side." 500 is the classic one. 502 usually means "I asked someone else and didn't get a good answer."

Don't memorise them. Just remember: 2xx good, 4xx a clear no, 5xx something broke.

Let's Dive Right In 😁😁

Here's the plan:

 Customer        Our Checkout API             Fake Provider
    |                  |                            |
    |-- POST /pay ---->|                            |
    |                  |-- POST /v1/charges ------->|  charges card πŸ’Έ
    |                  |   Idempotency-Key: K1      |  ...answer is slow 🐒
    |                  |   (times out after 1s)     |
    |                  |                            |
    |                  |-- retry, SAME key K1 ----->|  "seen K1 before"
    |                  |<-- 201 (replayed) ---------|  no new charge βœ…
    |<-- paid ---------|                            |
Enter fullscreen mode Exit fullscreen mode

Set up the project:

mkdir idempotency-keys-demo && cd idempotency-keys-demo
npm init -y
npm install express@5
npm install -D typescript @types/express @types/node@22
npm pkg set type=module scripts.build="tsc" scripts.start="node dist/server.js"
Enter fullscreen mode Exit fullscreen mode

Folder structure:

idempotency-keys-demo/
β”œβ”€β”€ src/
β”‚   β”œβ”€β”€ fakeProvider.ts
β”‚   β”œβ”€β”€ paymentClient.ts
β”‚   β”œβ”€β”€ orders.ts
β”‚   β”œβ”€β”€ app.ts
β”‚   └── server.ts
β”œβ”€β”€ demo.sh
β”œβ”€β”€ tsconfig.json
└── package.json
Enter fullscreen mode Exit fullscreen mode

Step 1: Tell TypeScript to be strict β€” tsconfig.json

{
  "compilerOptions": {
    "target": "ES2022",
    "module": "NodeNext",
    "moduleResolution": "NodeNext",
    "outDir": "dist",
    "rootDir": "src",
    "strict": true,
    "esModuleInterop": true,
    "skipLibCheck": true,
    "types": ["node"]
  },
  "include": ["src"]
}
Enter fullscreen mode Exit fullscreen mode

strict: true because we are touching money. No "any" vibes here. πŸ˜…

Step 2: The pretend bank β€” src/fakeProvider.ts

This file plays the role of the payment provider. It does one thing: create charges, and remember idempotency keys. It also has two "break it on purpose" switches for testing.

import express from "express";
import { createHash, randomUUID } from "node:crypto";

// A tiny pretend payment provider. NOT Stripe, NOT Paystack.
// It only copies the idea: same Idempotency-Key + same body = same answer.

type Charge = { id: string; orderId: string; amount: number; currency: string };
type SavedResult = { bodyHash: string; status: number; body: unknown };

const charges: Charge[] = [];
const idempotencyStore = new Map<string, SavedResult>();
let slowResponsesLeft = 0;
let crashesLeft = 0;
let crashAfterCharging = true;

const sleep = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms));
const hashBody = (body: unknown) =>
  createHash("sha256").update(JSON.stringify(body)).digest("hex");

export const provider = express();
provider.use(express.json());

provider.post("/v1/charges", async (req, res) => {
  const key = req.get("Idempotency-Key");
  const bodyHash = hashBody(req.body);

  if (key) {
    const saved = idempotencyStore.get(key);
    if (saved && saved.bodyHash !== bodyHash) {
      res.status(422).json({ error: "Idempotency-Key was already used with a different body" });
      return;
    }
    if (saved) {
      res.set("Idempotent-Replayed", "true").status(saved.status).json(saved.body);
      return;
    }
  }

  const { orderId, amount, currency } = req.body as Omit<Charge, "id">;
  const charge: Charge = { id: `ch_${randomUUID().slice(0, 8)}`, orderId, amount, currency };
  const crash = crashesLeft > 0;
  // The money moves on a normal request, or when we crash AFTER charging
  const moneyMoves = !crash || crashAfterCharging;
  if (moneyMoves) charges.push(charge); // πŸ’Έ money has moved

  if (crash) {
    // The "server error" part: something broke (maybe after the money moved, maybe before).
    // Like Stripe, we SAVE this 500 under the key, so retries get the same 500.
    crashesLeft--;
    const error = { error: "Internal server error" };
    if (key) idempotencyStore.set(key, { bodyHash, status: 500, body: error });
    res.status(500).json(error);
    return;
  }

  if (key) idempotencyStore.set(key, { bodyHash, status: 201, body: charge });

  if (slowResponsesLeft > 0) {
    slowResponsesLeft--;
    await sleep(3000); // the "network is bad" part: charged, but the answer is late
  }
  res.status(201).json(charge);
});

// Look up charges, optionally for one order (like searching by your orderId in metadata)
provider.get("/v1/charges", (req, res) => {
  const orderId = req.query.orderId;
  const found = orderId ? charges.filter((c) => c.orderId === orderId) : charges;
  res.json({ count: found.length, charges: found });
});

// Test helpers so we can break things on purpose
provider.post("/__test/slow", (req, res) => {
  slowResponsesLeft = Number(req.query.times ?? 1);
  res.json({ slowResponsesLeft });
});

provider.post("/__test/crash", (req, res) => {
  crashesLeft = Number(req.query.times ?? 1);
  crashAfterCharging = req.query.charged !== "no";
  res.json({ crashesLeft, crashAfterCharging });
});
Enter fullscreen mode Exit fullscreen mode

Four important parts, don't skip them:

  • Seen this key + same body? Replay the saved answer and add Idempotent-Replayed: true. Stripe sends that exact header on replayed responses.
  • hashBody turns the request body into a short fingerprint (a "hash"). Same body, same fingerprint. It's an easy way to check "is this the exact same request as before?"
  • Seen this key + different body? Refuse. Someone is reusing a key for a different payment. Stripe also errors in this case; the IETF draft for the Idempotency-Key header suggests 422, so that's what our fake uses.
  • The slow mode charges first, then waits 3 seconds before answering. That is our POS machine: money gone, answer late.
  • The crash mode answers with a 500 (a "server error": something broke on the provider's side). Sometimes the money already moved, sometimes it didn't, and from the outside you can't tell which. And the provider saves that 500 under the key. Stripe does exactly this: its docs say retries with the same key return the saved response, "including 500 errors". Remember this one. It's the bug the reader caught. πŸ‘€

Step 3: The agent who retries β€” src/paymentClient.ts

This is the code your server uses to call the provider. It has a timeout and it retries. But it does NOT retry everything. Different answers mean different things.

const PROVIDER_URL = process.env.PROVIDER_URL ?? "http://localhost:4000";
const TIMEOUT_MS = 1000;
const MAX_ATTEMPTS = 3;

export type ChargeParams = { orderId: string; amount: number; currency: string };
export type Charge = ChargeParams & { id: string };

// "I don't know if money moved." Never charge again until you've checked.
export class PaymentUnknownError extends Error {
  name = "PaymentUnknownError";
}
// "No, and nothing was charged." Safe to try again later with a NEW key.
export class PaymentRejectedError extends Error {
  name = "PaymentRejectedError";
}

const sleep = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms));

// 409 = "this key is still being processed", 429 = "too many requests, slow down".
// In both cases nothing was saved under the key yet, so retrying with the SAME key is safe.
const isRetryableStatus = (status: number) => status === 409 || status === 429;

export async function createCharge(
  params: ChargeParams,
  getKey: () => string // where the Idempotency-Key comes from
): Promise<Charge> {
  for (let attempt = 1; attempt <= MAX_ATTEMPTS; attempt++) {
    const key = getKey();
    try {
      const res = await fetch(`${PROVIDER_URL}/v1/charges`, {
        method: "POST",
        headers: { "Content-Type": "application/json", "Idempotency-Key": key },
        body: JSON.stringify(params),
        signal: AbortSignal.timeout(TIMEOUT_MS),
      });
      const replayed = res.headers.get("Idempotent-Replayed") === "true";
      console.log(`attempt ${attempt} key=${key.slice(0, 8)} -> ${res.status}${replayed ? " (replayed)" : ""}`);

      if (res.ok) return (await res.json()) as Charge;

      if (res.status >= 500) {
        // A 5xx is saved under the key: retrying with it only replays this same error.
        throw new PaymentUnknownError(`Provider returned ${res.status}`);
      }
      if (!isRetryableStatus(res.status)) {
        throw new PaymentRejectedError(`Provider said no: ${res.status} ${await res.text()}`);
      }
    } catch (err) {
      if (err instanceof PaymentUnknownError || err instanceof PaymentRejectedError) throw err;
      // Timeout or network error: we don't know what happened, so retry with the same key
      console.log(`attempt ${attempt} key=${key.slice(0, 8)} -> ${(err as Error).name}`);
    }
    await sleep(200 * 2 ** attempt); // 400ms, 800ms, ...
  }
  throw new PaymentUnknownError(`No clear answer after ${MAX_ATTEMPTS} attempts`);
}

// Ask the provider: "did any charge go through for this order?"
export async function findChargeForOrder(orderId: string): Promise<Charge | undefined> {
  const res = await fetch(`${PROVIDER_URL}/v1/charges?orderId=${encodeURIComponent(orderId)}`, {
    signal: AbortSignal.timeout(TIMEOUT_MS),
  });
  if (!res.ok) throw new PaymentUnknownError(`Lookup failed: ${res.status}`);
  const { charges } = (await res.json()) as { charges: Charge[] };
  return charges[0];
}
Enter fullscreen mode Exit fullscreen mode

Here's how it reads each answer from the provider. HTTP status codes are just numbers the server sends back to say how things went:

What came back What it means What we do
201 Charged. πŸŽ‰ Done
No answer (timeout or network error) We don't know if money moved Retry with the same key
409 Conflict "I'm still working on that same key" Wait, retry with the same key
429 Too Many Requests "Slow down" Wait, retry with the same key
5xx (500, 502, 503...) Something broke. Money may have moved. The error is saved under the key Stop. Mark it "unknown"
Other 4xx (like 402 card declined, 422 key reused) A clear "no". Nothing was charged Stop. Customer can try again later

A few more things:

  • getKey is a tiny function that hands us the key for each attempt. () => order.paymentKey gives back the same saved key every time. () => randomUUID() makes a brand-new random key every time. That one difference is what this whole article is about.
  • AbortSignal.timeout(1000) gives up waiting after 1 second and throws a TimeoutError. Our fake provider's slow mode waits 3 seconds, so the first attempt always times out. That's how we fake a bad network.
  • res.ok is true for any 2xx reply.
  • Retries wait a bit longer each time: 200 * 2 ** attempt means 200 Γ— 2 to the power of the attempt number, so 400ms, then 800ms. That's called backoff: we don't hammer a provider that's already struggling.
  • Why stop on a 5xx? Because the provider saved that 500 under our key. Retrying with the same key just replays the same 500. That's what my first version did: three attempts, three identical 500s, zero progress. πŸ€¦πŸ½β€β™‚οΈ And retrying with a new key is worse: if the money already moved, that's a second charge.
  • If we never get a clear answer after 3 attempts, that's also "unknown".
  • We call getKey() on every attempt. Whether that returns a new key or the same key is the whole difference between a happy customer and an angry one. Hold that thought.

Step 4: Write the slip number down first β€” src/orders.ts

This file does one thing: keep orders. And every order gets its payment key the moment it is created, before anyone tries to pay.

import { randomUUID } from "node:crypto";

export type Order = {
  id: string;
  amount: number; // minor units, e.g. kobo
  currency: string;
  status: "pending" | "paid" | "payment_unknown";
  paymentKey: string; // the Idempotency-Key for paying THIS order
  chargeId?: string;
};

// In-memory "database". In real life this is a table with a unique paymentKey column.
const orders = new Map<string, Order>();

export function createOrder(amount: number, currency: string): Order {
  const order: Order = {
    id: `ord_${randomUUID().slice(0, 8)}`,
    amount,
    currency,
    status: "pending",
    paymentKey: randomUUID(), // created and saved BEFORE any payment attempt
  };
  orders.set(order.id, order);
  return order;
}

export const findOrder = (id: string) => orders.get(id);

export function markPaid(order: Order, chargeId: string) {
  order.status = "paid";
  order.chargeId = chargeId;
}

export function markUnknown(order: Order) {
  order.status = "payment_unknown"; // money MAY have moved. Check before charging again.
}

// Only call this once you KNOW nothing was charged with the old key
export function resetForNewAttempt(order: Order) {
  order.status = "pending";
  order.paymentKey = randomUUID();
}
Enter fullscreen mode Exit fullscreen mode

Three statuses now:

  • pending: not paid yet, safe to try.
  • paid: done. πŸŽ‰
  • payment_unknown: money may have moved. The order is locked until we check.

resetForNewAttempt gives the order a fresh key. We only call it when we know nothing was charged with the old one. That's the golden rule: a new key only after you've confirmed the old one charged nothing.

That paymentKey line is the slip number. πŸ“

I'm using an in-memory Map so you don't need a database to follow along. In production, save the key and status in your real database (a column on the order or a payments table). If the key only lives in memory and your server restarts in the middle of a retry, you lose the slip and you're back to square one. Stripe's docs say it plainly: if you retry across separate calls or process restarts, generate and persist (save somewhere permanent, like your database) a key for the operation and send it on every attempt.

Amounts are in minor units (kobo, so ₦5,000 is 500000), because floats and money don't mix. That's a whole story for another day. πŸ˜‹

Step 5: Two checkout doors, one is broken β€” src/app.ts

Now the Express app. I'm keeping a broken route on purpose so you can see the bug with your own eyes, plus a new "check what happened" route. (The imports end in .js even though our files are .ts. That's because TypeScript turns each .ts file into a .js file in dist/, and that's what Node actually runs.)

import express from "express";
import { randomUUID } from "node:crypto";
import {
  createCharge,
  findChargeForOrder,
  PaymentRejectedError,
  PaymentUnknownError,
} from "./paymentClient.js";
import { createOrder, findOrder, markPaid, markUnknown, resetForNewAttempt } from "./orders.js";

export const app = express();
app.use(express.json());

app.post("/orders", (req, res) => {
  const { amount, currency } = req.body as { amount: number; currency: string };
  if (!Number.isInteger(amount) || amount <= 0 || typeof currency !== "string") {
    res.status(400).json({ error: "amount (integer, minor units) and currency are required" });
    return;
  }
  res.status(201).json(createOrder(amount, currency));
});

// ❌ The bug: a brand-new key on every attempt
app.post("/orders/:id/pay-naive", async (req, res) => {
  const order = findOrder(req.params.id);
  if (!order) {
    res.status(404).json({ error: "Order not found" });
    return;
  }
  const charge = await createCharge(
    { orderId: order.id, amount: order.amount, currency: order.currency },
    () => randomUUID()
  );
  markPaid(order, charge.id);
  res.json(order);
});

// βœ… The fix: one key per order, saved before the first attempt, reused every time
app.post("/orders/:id/pay", async (req, res) => {
  const order = findOrder(req.params.id);
  if (!order) {
    res.status(404).json({ error: "Order not found" });
    return;
  }
  if (order.status === "paid") {
    res.json(order); // already paid, nothing to do
    return;
  }
  if (order.status === "payment_unknown") {
    res.status(409).json({ error: "We're still checking your last payment. Not charging again." });
    return;
  }
  try {
    const charge = await createCharge(
      { orderId: order.id, amount: order.amount, currency: order.currency },
      () => order.paymentKey
    );
    markPaid(order, charge.id);
    res.json(order);
  } catch (err) {
    if (err instanceof PaymentUnknownError) {
      markUnknown(order);
      res.status(502).json({ error: "Payment status unknown. Do not charge again; check with the provider.", order });
      return;
    }
    if (err instanceof PaymentRejectedError) {
      resetForNewAttempt(order); // nothing was charged, so a fresh key is safe
      res.status(402).json({ error: "Payment was not accepted.", order });
      return;
    }
    throw err;
  }
});

// πŸ”Ž Find out what really happened, then unlock the order
app.post("/orders/:id/reconcile", async (req, res) => {
  const order = findOrder(req.params.id);
  if (!order) {
    res.status(404).json({ error: "Order not found" });
    return;
  }
  if (order.status !== "payment_unknown") {
    res.json(order); // nothing to check
    return;
  }
  const charge = await findChargeForOrder(order.id);
  if (charge) {
    markPaid(order, charge.id); // the money DID move. Don't charge again.
  } else {
    resetForNewAttempt(order); // confirmed: nothing went through. New key, safe to retry.
  }
  res.json(order);
});

app.get("/orders/:id", (req, res) => {
  const order = findOrder(req.params.id);
  if (!order) {
    res.status(404).json({ error: "Order not found" });
    return;
  }
  res.json(order);
});

app.use((err: Error, _req: express.Request, res: express.Response, _next: express.NextFunction) => {
  console.error(err);
  res.status(502).json({ error: "Payment status unknown. Do not charge again; check with the provider." });
});
Enter fullscreen mode Exit fullscreen mode

Here's the whole journey of an order:

 POST /pay ──► 201 ──────────────────────► paid βœ…
         β”œβ”€β”€β–Ί clear "no" (4xx) ──────────► pending + NEW key
         └──► 5xx / no clear answer ─────► payment_unknown πŸ”’
                                              β”‚
                                   POST /reconcile (check)
                                              β”‚
                         β”œβ”€β”€ charge found ────► paid βœ… (no 2nd charge)
                         └── nothing found ───► pending + NEW key
Enter fullscreen mode Exit fullscreen mode

Spot the important bits:

  • pay-naive passes () => randomUUID(). New slip number on every retry. Looks safe. Is not. πŸ™ˆ
  • pay passes () => order.paymentKey. Same slip number every time.
  • pay checks status === "paid" first, so a double-click on the Pay button doesn't even reach the provider.
  • pay refuses to charge a payment_unknown order. The customer gets "we're still checking" instead of a surprise second debit.
  • /reconcile is the "check what really happened" step. In payments, reconcile just means "compare your records with the provider's and make them match".
  • Our own API answers with codes too: 502 ("the payment company didn't give us a clear answer"), 402 ("payment not accepted") and 409 ("we're busy checking this order, try later").
  • /reconcile It asks the provider "is there a charge for this order?" If yes: paid. If no: now it's safe to make a new key.

In this demo, /reconcile searches the fake provider by orderId. With real Stripe, Stripe's docs say to work it out by fetching the payment, checking the request in the Dashboard, and using webhooks (Stripe calling your server when something happens). They also recommend putting your own ID in the payment's metadata (extra info you attach to it), so when a webhook arrives you know which order it belongs to. In a real app, your webhook handler would do what /reconcile does here.

(We return paymentKey in the JSON only so the demo can show it. In a real app, keep it on the server.)

Step 6: Start both servers β€” src/server.ts

import { app } from "./app.js";
import { provider } from "./fakeProvider.js";

provider.listen(4000, () => console.log("πŸ’³ Fake provider on http://localhost:4000"));
app.listen(3000, () => console.log("πŸ›’ Checkout API on http://localhost:3000"));
Enter fullscreen mode Exit fullscreen mode

Build and run:

npx tsc --strict
npm start
Enter fullscreen mode Exit fullscreen mode

Step 7: Let's break it on purpose β€” demo.sh

Open a second terminal. This script slows down and crashes the fake provider on purpose, and pays orders both ways.

Don't worry about the node -pe ... bits: they just pull one field (like id) out of the JSON reply. > /dev/null hides output we don't need, and head -1 shows only the first line (the status code).

#!/usr/bin/env bash
# Run `npm start` in another terminal first.
API=http://localhost:3000
PROVIDER=http://localhost:4000
new_order() { curl -s -X POST $API/orders -H "Content-Type: application/json" -d '{"amount":500000,"currency":"NGN"}' | node -pe 'JSON.parse(require("fs").readFileSync(0)).id'; }
charges_for() { echo "Charges for $1: $(curl -s "$PROVIDER/v1/charges?orderId=$1" | node -pe 'JSON.parse(require("fs").readFileSync(0)).count')"; }

echo "== 1. Naive payment (new key every attempt), slow network on first try =="
ORDER=$(new_order)
curl -s -X POST "$PROVIDER/__test/slow?times=1" > /dev/null
curl -s -X POST $API/orders/$ORDER/pay-naive; echo
charges_for $ORDER

echo "== 2. Fixed payment (one key per order), same slow network =="
ORDER=$(new_order)
curl -s -X POST "$PROVIDER/__test/slow?times=1" > /dev/null
curl -s -X POST $API/orders/$ORDER/pay; echo
charges_for $ORDER

echo "== 3. User double-clicks Pay =="
curl -s -X POST $API/orders/$ORDER/pay; echo
charges_for $ORDER

echo "== 4. Same key, different amount (someone reused a key) =="
KEY=$(curl -s $API/orders/$ORDER | node -pe 'JSON.parse(require("fs").readFileSync(0)).paymentKey')
curl -s -i -X POST $PROVIDER/v1/charges -H "Content-Type: application/json" -H "Idempotency-Key: $KEY" \
  -d "{\"orderId\":\"$ORDER\",\"amount\":999,\"currency\":\"NGN\"}" | head -1

echo "== 5. Unknown order =="
curl -s -X POST $API/orders/ord_nope/pay; echo

echo "== 6. Provider charges the card, THEN crashes with a 500 =="
ORDER=$(new_order)
curl -s -X POST "$PROVIDER/__test/crash?times=1" > /dev/null
curl -s -X POST $API/orders/$ORDER/pay; echo
echo "Customer clicks Pay again:"
curl -s -X POST $API/orders/$ORDER/pay; echo
echo "We check what really happened:"
curl -s -X POST $API/orders/$ORDER/reconcile; echo
charges_for $ORDER

echo "== 7. Provider crashes with a 500 BEFORE charging =="
ORDER=$(new_order)
curl -s -X POST "$PROVIDER/__test/crash?times=1&charged=no" > /dev/null
curl -s -X POST $API/orders/$ORDER/pay; echo
echo "We check what really happened:"
curl -s -X POST $API/orders/$ORDER/reconcile; echo
echo "Customer clicks Pay again (new key now):"
curl -s -X POST $API/orders/$ORDER/pay; echo
charges_for $ORDER
Enter fullscreen mode Exit fullscreen mode
chmod +x demo.sh   # allow the script to run
./demo.sh
Enter fullscreen mode Exit fullscreen mode

Here's what I got (your IDs will be different):

== 1. Naive payment (new key every attempt), slow network on first try ==
{"id":"ord_0893d289","amount":500000,"currency":"NGN","status":"paid","paymentKey":"24a89b7d-fb0f-4674-a60d-95dd675c5eaf","chargeId":"ch_c0239405"}
Charges for ord_0893d289: 2
== 2. Fixed payment (one key per order), same slow network ==
{"id":"ord_304c6a08","amount":500000,"currency":"NGN","status":"paid","paymentKey":"7458cd8a-0281-4fb1-b377-d7bc101a95bc","chargeId":"ch_d5e08189"}
Charges for ord_304c6a08: 1
== 3. User double-clicks Pay ==
{"id":"ord_304c6a08","amount":500000,"currency":"NGN","status":"paid","paymentKey":"7458cd8a-0281-4fb1-b377-d7bc101a95bc","chargeId":"ch_d5e08189"}
Charges for ord_304c6a08: 1
== 4. Same key, different amount (someone reused a key) ==
HTTP/1.1 422 Unprocessable Entity
== 5. Unknown order ==
{"error":"Order not found"}
== 6. Provider charges the card, THEN crashes with a 500 ==
{"error":"Payment status unknown. Do not charge again; check with the provider.","order":{"id":"ord_2967c55f","amount":500000,"currency":"NGN","status":"payment_unknown","paymentKey":"a97e8cef-1273-4f0a-9349-a8cc6b9d37df"}}
Customer clicks Pay again:
{"error":"We're still checking your last payment. Not charging again."}
We check what really happened:
{"id":"ord_2967c55f","amount":500000,"currency":"NGN","status":"paid","paymentKey":"a97e8cef-1273-4f0a-9349-a8cc6b9d37df","chargeId":"ch_dabe9b6b"}
Charges for ord_2967c55f: 1
== 7. Provider crashes with a 500 BEFORE charging ==
{"error":"Payment status unknown. Do not charge again; check with the provider.","order":{"id":"ord_b335bfe7","amount":500000,"currency":"NGN","status":"payment_unknown","paymentKey":"3dfbf9cb-9c20-4ad1-be7b-0085c3019368"}}
We check what really happened:
{"id":"ord_b335bfe7","amount":500000,"currency":"NGN","status":"pending","paymentKey":"ce716ffa-7761-4ae7-9b97-cc7fbca51150"}
Customer clicks Pay again (new key now):
{"id":"ord_b335bfe7","amount":500000,"currency":"NGN","status":"paid","paymentKey":"ce716ffa-7761-4ae7-9b97-cc7fbca51150","chargeId":"ch_a295c95a"}
Charges for ord_b335bfe7: 1
Enter fullscreen mode Exit fullscreen mode

And the server terminal tells the full story:

attempt 1 key=1bbe2ef6 -> TimeoutError
attempt 2 key=6588fd45 -> 201
attempt 1 key=7458cd8a -> TimeoutError
attempt 2 key=7458cd8a -> 201 (replayed)
attempt 1 key=a97e8cef -> 500
attempt 1 key=3dfbf9cb -> 500
attempt 1 key=ce716ffa -> 201
Enter fullscreen mode Exit fullscreen mode

Look at the naive one (scenario 1). Look at it. πŸ‘€

The API happily says "status":"paid", everything looks fine... and the customer was charged twice for one ₦5,000 order. Two different keys, two charges. (Notice the keys in the server log don't match the paymentKey printed for that order. That's the bug: pay-naive ignores the saved key and makes up new ones.) The first one timed out on our side, but the provider had already done the work.

The fixed one (scenario 2): same key on both attempts, the provider says "replayed", one charge. πŸŽ‰

celebrate

Now the 500s:

  • Scenario 6 (charged, then crashed): one attempt, no pointless retries. The order goes to payment_unknown. The customer clicks Pay again and gets "we're still checking" instead of a second charge. Reconcile finds the charge, the order becomes paid. One charge.
  • Scenario 7 (crashed before charging): reconcile finds nothing, so the order gets a new key. The next Pay goes through. Still one charge.

In my first version, scenario 6 would retry the same key three times, get the same saved 500 three times, and every later click would hit the same wall for as long as the provider remembers the key. 😭

Bonus test: if the provider is completely down, pay tries 3 times with the same key (network errors are worth retrying), then marks the order payment_unknown and returns 502. No guessing, no surprise second charge later.

Before vs after

Before (pay-naive) After (pay)
Key created On every attempt Once, when the order is created
Key stored Nowhere Saved with the order
Slow network + retry Two charges 😭 One charge, second answer replayed
Double-click on Pay Another charge "Already paid", provider not called
Provider returns a 500 Next click makes a new key, maybe a second charge Order locked, we check first
New key Every attempt Only after confirming nothing was charged
Server restart mid-retry New key, new charge Same key (if saved in a real DB)

Why we do it this way 🧐

1. The key belongs to the business action, not the HTTP request.
"Pay for order ord_123" is one action. It might take three HTTP attempts. All three must carry the same key. A new key means you are telling the provider "this is a new payment". And the provider believes you.

2. A timeout is not a failure.
It's an "I don't know". The money may have moved. Stripe's docs say it directly: when a connection fails you might not know whether the request was executed, so retry with the same key and the same parameters.

3. A 500 is also "I don't know", but you can't retry your way out of it.
Stripe's docs say to treat a 500 as indeterminate (could have gone either way): the payment might have happened even though you got an error. They save that 500 under the key, so the same key replays it. And they say not to retry with a new key until you've worked out what actually happened. So: stop, mark it unknown, check, and only then decide. Compare that with a 409: Stripe says it means another request with the same key is still running and nothing has been saved yet, so waiting and retrying with the same key is exactly right.

4. Keys don't live forever.
Stripe keeps keys for at least 24 hours and may remove them after that. If you still don't know what happened after that window, Stripe's advice is to stop retrying (with the old key or a new one) and check: look up the payment, check your dashboard, and let webhooks tell you the final state.

5. Same key, same body. Always.
If you change the amount and reuse the key, Stripe errors instead of charging. That's a feature: it stops one key from being used for two different things.

6. Use random keys, nothing personal.
Stripe suggests V4 UUIDs (random IDs like 3e82095b-2c0f-4d8c-a3c6-b8838d773137, exactly what randomUUID() gives you) or another string random enough that two can't clash, up to 255 characters, and warns against putting things like email addresses in them. crypto.randomUUID() is built into Node, no package needed.

7. If you use an official SDK, read what it already does.
An SDK is the official library you install to talk to a provider (like npm install stripe). The Stripe Node SDK retries failed requests once by default (maxNetworkRetries) and adds idempotency keys to those automatic retries. You can also pass your own key as a request option:

await stripe.paymentIntents.create(
  { amount: 500000, currency: "ngn" },
  { idempotencyKey: order.paymentKey }
);
Enter fullscreen mode Exit fullscreen mode

(I tested this snippet with stripe@23.1.0 pointed at a small local mock server instead of real Stripe, and the mock received our exact key in the Idempotency-Key header. With a real test key it goes to Stripe.) The SDK's built-in key only protects retries inside one call. If your own code retries, or a job re-runs after a crash, you still need your own saved key.

8. Not every provider does it the same way.
Idempotency-Key is the header Stripe uses, and there's a draft from the IETF (the group that writes internet standards) trying to standardise it (it lists Stripe, Adyen and Worldpay among the implementers, and notes PayPal uses its own PayPal-Request-Id header). Before you ship, check your provider's docs for their exact header, how long they keep keys, what they save (do they save errors too?), and what they return on a repeat. If your provider has no idempotency support at all, your saved order status + "check before charging again" + webhooks become your safety net.

thinking hard

Recap (say it with me) πŸ—£οΈ

  1. A timeout means "I don't know", not "it failed".
  2. Create the idempotency key once per business action (one order = one key).
  3. Save it before the first attempt.
  4. Reuse it on every retry. Never generate a fresh one inside the retry loop.
  5. Same key always goes with the same body.
  6. Retry timeouts, network errors, 409 and 429 with the same key, with backoff.
  7. On a 5xx, stop. Mark the order "unknown" and check what really happened (webhook or look it up).
  8. Only make a new key once you've confirmed the old one charged nothing.

Next time the POS machine hangs, you'll smile, because now you know what the bank should be doing behind the scenes. 😁😁

smug nud

GitHub repo: [https://github.com/FredAbod/stop-charging-customers-twice-idempotency-keys-code]

I hope this was helpful. Don't forget to drop a like and follow πŸ˜‹πŸ˜‹

Thank you for taking the time to read πŸ₯°πŸ₯°

See you on the next one πŸ€—

waving bye

Top comments (2)

Collapse
 
sunnatulla_dev profile image
Sunnatulla •

One thing that bites with Stripe specifically: isRetryableStatus puts 5xx in the same bucket as 409/429, but those behave very differently once the key is reused.

For API v1 Stripe saves the response of the first request under the key as soon as the endpoint starts executing, and their docs say retries with the same key return that saved response, including 500 errors. So attempts 2 and 3 in createCharge can never succeed, they just replay the cached 500. And because pay reuses order.paymentKey, every later click on Pay replays it too, for as long as Stripe keeps the key (24h+). The order is stuck in pending, and the customer may or may not have been charged.

I copied your retry loop and pointed it at a small mock that follows that rule (first attempt charges, then returns 500):

attempt 1 -> 500 replayed=no
attempt 2 -> 500 replayed=true
attempt 3 -> 500 replayed=true
click 1: Provider returned 500 after 2846ms, charges=1
attempt 1 -> 500 replayed=true
...
click 2: Provider returned 500 after 2819ms, charges=1
Enter fullscreen mode Exit fullscreen mode

409 is the opposite case. That's "same key is still executing", nothing is saved, so retrying it is right. What I'd do: keep retrying timeouts/409/429 with the same key, but on a 5xx stop and mark the order something like payment_unknown. Then reconcile, either from the webhook or by looking up the PaymentIntent (put orderId in metadata so you can find it). Only mint a new key once you've confirmed nothing went through. Stripe's error-handling page says pretty much this: treat a 500 as indeterminate and don't retry with a new key until you've resolved it.

Your 502 "status unknown, don't charge again" message is already the right instinct, it just needs a state behind it so the next click doesn't hit the same wall.

Collapse
 
fredabod profile image
FredAbod •

You’re right, and thanks for actually running it. πŸ™

I treated a 5xx like a timeout, but with Stripe the 500 is saved under the key. So retrying with the same key just gets the same 500 back, and every later click on Pay hits that wall too.

I’m updating the article and code to:
β†’ keep retrying timeouts, 409 and 429 with the same key
β†’ on a 5xx, stop and mark the order payment_unknown
β†’ check what really happened (webhook, or look up the PaymentIntent by orderId in metadata)
β†’ only make a new key once I’ve confirmed nothing went through

Really appreciate this. This is exactly the kind of catch that makes the article better for everyone. πŸ€—