Building an Idempotent REST API with Request Keys and Safe Retries

#frontend #webdev

Building an Idempotent REST API with Request Keys and Safe Retries

When a client times out or a network blip happens, the same request may be sent twice. An idempotent API lets you survive that without creating duplicate orders, payments, or side effects.

Why this matters

A plain POST endpoint can be dangerous when a retry happens after the server already processed the first request. The practical fix is to give each logical operation a stable request key and make the server return the same result for repeated submissions of that key.

This tutorial shows a realistic pattern for creating an POST /orders endpoint that is safe to retry. We will use:

An Idempotency-Key header.
A database table to record request status and response data.
A transaction that creates the order only once.
Cached responses for duplicate requests.

Design overview

The flow is simple:

The client generates a unique idempotency key for one logical action.
The server checks whether that key was already used.
If not, the server processes the request and stores the outcome.
If yes, the server returns the original response instead of running the action again.

This is especially useful for payments, checkout flows, account creation, and any API that may be retried by mobile clients, gateways, or background jobs. Idempotency is one of the easiest ways to make a distributed system feel reliable.

Database model

Use a table to track the request key and the response you want to replay:

CREATE TABLE idempotency_keys (
  key TEXT PRIMARY KEY,
  request_hash TEXT NOT NULL,
  status TEXT NOT NULL,
  response_code INT,
  response_body JSONB,
  created_at TIMESTAMP NOT NULL DEFAULT NOW()
);

CREATE TABLE orders (
  id SERIAL PRIMARY KEY,
  customer_id INT NOT NULL,
  amount_cents INT NOT NULL,
  status TEXT NOT NULL,
  created_at TIMESTAMP NOT NULL DEFAULT NOW()
);

The request_hash helps you detect when the same key is reused for a different payload. That protects you from accidental key collisions or buggy clients.

Server implementation

Below is a compact Node.js example using Express and PostgreSQL. It stores the key first, then creates the order, then saves the response for future retries.

import express from "express";
import crypto from "crypto";
import pg from "pg";

const app = express();
app.use(express.json());

const pool = new pg.Pool({
  connectionString: process.env.DATABASE_URL,
});

function hashBody(body) {
  return crypto.createHash("sha256").update(JSON.stringify(body)).digest("hex");
}

app.post("/orders", async (req, res) => {
  const idempotencyKey = req.header("Idempotency-Key");
  if (!idempotencyKey) {
    return res.status(400).json({ error: "Idempotency-Key header is required" });
  }

  const requestHash = hashBody(req.body);

  const client = await pool.connect();
  try {
    await client.query("BEGIN");

    const existing = await client.query(
      "SELECT * FROM idempotency_keys WHERE key = $1 FOR UPDATE",
      [idempotencyKey]
    );

    if (existing.rows.length > 0) {
      const row = existing.rows;

      if (row.request_hash !== requestHash) {
        await client.query("ROLLBACK");
        return res.status(409).json({
          error: "Idempotency key reused with a different request body",
        });
      }

      await client.query("COMMIT");
      return res.status(row.response_code).json(row.response_body);
    }

    const orderResult = await client.query(
      "INSERT INTO orders (customer_id, amount_cents, status) VALUES ($1, $2, $3) RETURNING id, customer_id, amount_cents, status, created_at",
      [req.body.customerId, req.body.amountCents, "created"]
    );

    const order = orderResult.rows;
    const responseBody = {
      orderId: order.id,
      status: order.status,
      amountCents: order.amount_cents,
    };

    await client.query(
      `INSERT INTO idempotency_keys (key, request_hash, status, response_code, response_body)
       VALUES ($1, $2, $3, $4, $5)`,
      [idempotencyKey, requestHash, "completed", 201, responseBody]
    );

    await client.query("COMMIT");
    return res.status(201).json(responseBody);
  } catch (err) {
    await client.query("ROLLBACK");
    throw err;
  } finally {
    client.release();
  }
});

app.listen(3000);

This version gives you a replayable response, but it still has one important weakness: if the process crashes after creating the order and before storing the key record, a duplicate request could slip through. The next section addresses that.

Safer transaction flow

A stronger approach is to insert the idempotency row first with a processing state, then complete the business action, then update the row with the final response. That way, the row exists even if the handler fails halfway through.

Here is the improved flow:

Lock or reserve the idempotency key.
If it already exists and is complete, replay the response.
If it already exists and is processing, return 409 or 202.
If it does not exist, create it in processing.
Run the business logic.
Update the row to completed with the final response.

Example SQL-friendly pseudocode:

await client.query("BEGIN");

const inserted = await client.query(
  `INSERT INTO idempotency_keys (key, request_hash, status)
   VALUES ($1, $2, 'processing')
   ON CONFLICT (key) DO NOTHING
   RETURNING key`,
  [idempotencyKey, requestHash]
);

if (inserted.rows.length === 0) {
  const existing = await client.query(
    "SELECT * FROM idempotency_keys WHERE key = $1",
    [idempotencyKey]
  );

  if (existing.rows.request_hash !== requestHash) {
    await client.query("ROLLBACK");
    return res.status(409).json({ error: "Key reused for different request" });
  }

  if (existing.rows.status === "completed") {
    await client.query("COMMIT");
    return res.status(existing.rows.response_code).json(existing.rows.response_body);
  }

  await client.query("ROLLBACK");
  return res.status(202).json({ status: "Request already processing" });
}

That pattern is often easier to reason about under retries because the key record becomes the source of truth for the request lifecycle.

Client usage

The client should generate one key per logical attempt, not one key per HTTP retry. A UUID is usually enough.

async function createOrder() {
  const response = await fetch("/orders", {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      "Idempotency-Key": crypto.randomUUID(),
    },
    body: JSON.stringify({
      customerId: 42,
      amountCents: 2599,
    }),
  });

  const data = await response.json();
  console.log(data);
}

If the request times out, the client can safely retry with the same Idempotency-Key. That lets the server return the same outcome instead of creating a second order.

Common mistakes

A few mistakes show up repeatedly in real systems:

Reusing the same key for different payloads.
Storing only a success flag and not the response body.
Letting keys expire too quickly for real retry windows.
Applying idempotency only to the database write but not to downstream side effects.
Returning a new order ID on every retry, which defeats the whole point.

If your endpoint publishes messages or sends emails, make those side effects idempotent too. Otherwise the database may stay consistent while the user still receives duplicate notifications. That concern is closely related to reliable event publishing patterns like the transactional outbox.

Testing the behavior

Test three cases before you ship:

First request succeeds and creates one record.
Same request with same key returns the original response.
Same key with different payload returns 409 Conflict.

Example test sketch:

test("repeated request returns same response", async () => {
  const key = "test-key-123";

  const first = await request(app)
    .post("/orders")
    .set("Idempotency-Key", key)
    .send({ customerId: 42, amountCents: 2599 });

  const second = await request(app)
    .post("/orders")
    .set("Idempotency-Key", key)
    .send({ customerId: 42, amountCents: 2599 });

  expect(first.status).toBe(201);
  expect(second.status).toBe(201);
  expect(second.body.orderId).toBe(first.body.orderId);
});

A good test suite should also simulate timeouts, process restarts, and concurrent duplicate submissions. Those are the exact situations idempotency is meant to handle.

Production checklist

Before rollout, make sure you have:

A unique index on the idempotency key column.
A clear expiry policy for old keys.
Payload hashing for conflict detection.
Stored response data for replay.
Monitoring for conflict and replay rates.
Documentation telling clients to reuse keys across retries.

For payment-like operations, consider making the idempotency key part of your public API contract. That makes retries predictable and keeps the behavior visible to both frontend and backend teams.

Next step

Once this is in place, extend the same idea to downstream jobs, message publishing, and webhook handlers. That gives you a consistent retry story across the whole request chain, not just one endpoint.

Rizwan Saleem | https://rizwansaleem.co