How to Add Request Deduplication to a Node.js API in an Afternoon

You don't need a redesign to make an existing Node.js API safe against duplicate requests. If you've got a database with unique constraints and a few hours, you can add a working idempotency layer to one endpoint today and roll it out to the rest as time allows. Here's the whole process, step by step, using nothing more exotic than Express and whatever relational database you're already running.

Step 1: Decide Which Endpoints Actually Need This

Not every endpoint is worth protecting. A GET request is naturally safe to repeat. A POST that only reads data or triggers something harmless to run twice doesn't need dedup logic either. The endpoints that matter are the ones with real side effects: creating an order, charging a card, sending a notification, provisioning a resource. Start with the one endpoint where a duplicate would actually cause a visible problem, ideally your highest-traffic mutating endpoint, rather than trying to cover everything on day one.

Step 2: Add the Dedup Table

A single table handles this for most APIs:

CREATE TABLE idempotency_keys (
  key TEXT PRIMARY KEY,
  request_hash TEXT NOT NULL,
  response_status INT,
  response_body JSONB,
  created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);

The key column is the client-generated idempotency key, and its primary key constraint is what makes the whole thing race-safe. request_hash lets you detect when a client reuses a key with a different payload, which should be treated as a conflict rather than a legitimate retry. response_status and response_body let you replay the exact original response on a repeat request instead of just returning a generic "already processed" message.

Step 3: Write the Middleware

const crypto = require('crypto');

async function idempotencyMiddleware(req, res, next) {
  const key = req.header('Idempotency-Key');
  if (!key) return next();

  const requestHash = crypto
    .createHash('sha256')
    .update(JSON.stringify(req.body))
    .digest('hex');

  try {
    await db.query(
      'INSERT INTO idempotency_keys (key, request_hash) VALUES ($1, $2)',
      [key, requestHash]
    );
  } catch (err) {
    if (err.code === '23505') {
      const existing = await db.query(
        'SELECT request_hash, response_status, response_body FROM idempotency_keys WHERE key = $1',
        [key]
      );
      const row = existing.rows[0];

      if (row.request_hash !== requestHash) {
        return res.status(409).json({ error: 'idempotency_key_conflict' });
      }
      if (row.response_status) {
        return res.status(row.response_status).json(row.response_body);
      }
      return res.status(409).json({ error: 'request_in_progress' });
    }
    throw err;
  }

  req.idempotencyKey = key;
  next();
}

The insert happens before any real work runs. If it succeeds, this is a genuinely new request and processing continues normally. If it fails with a unique violation (23505 is PostgreSQL's constraint violation code), one of two things is true: this exact request was already made, and you should replay the stored response, or a different request reused the same key, and you should reject it with a 409. If the row exists but has no stored response yet, another request with the same key is currently mid-flight, which the client should treat as a retryable in-progress state.

Step 4: Store the Response After Processing

The middleware alone doesn't complete the pattern. After your route handler finishes, you need to write the response back to the same row:

async function finalizeIdempotency(req, status, body) {
  if (!req.idempotencyKey) return;
  await db.query(
    'UPDATE idempotency_keys SET response_status = $1, response_body = $2 WHERE key = $3',
    [status, body, req.idempotencyKey]
  );
}

Call this right before sending the response in your route handler, so the stored row reflects exactly what the client received the first time. This is the piece that makes a retry return the original result instead of just a generic acknowledgment.

Step 5: Wire It Into One Route First

app.post('/orders', idempotencyMiddleware, async (req, res) => {
  const order = await createOrder(req.body);
  await finalizeIdempotency(req, 201, order);
  res.status(201).json(order);
});

Keep the first rollout narrow. One route, tested thoroughly, gives you confidence in the pattern before you copy it across the rest of your mutating endpoints, and it lets you catch integration issues, like a client library that doesn't consistently send the header, without that mistake reaching every endpoint at once.

Step 6: Test the Race Condition, Not Just the Happy Path

A sequential test, send a request, then send the identical request again and check that the response matches, confirms the basic mechanism works. It doesn't confirm the part that actually matters under load: two requests with the same key arriving close enough together to race.

const [res1, res2] = await Promise.all([
  request(app).post('/orders').set('Idempotency-Key', 'test-key-1').send(payload),
  request(app).post('/orders').set('Idempotency-Key', 'test-key-1').send(payload),
]);

const successCount = [res1, res2].filter(r => r.status === 201).length;
expect(successCount).toBe(1);

If both requests return 201, the unique constraint isn't actually preventing concurrent processing, which usually means the insert is happening after the real work instead of before it. This is the single most valuable test in the whole implementation, because it's the one that catches the mistake that looks correct in every other test.

Optional: Adding a Fast Path With Redis

If your API handles enough traffic that a relational lookup on every request becomes a noticeable cost, a common pattern is to put a fast, in-memory check in front of the database. Redis works well here: attempt a SET key value NX (set-if-not-exists) before touching the database at all. If the Redis set succeeds, proceed to the database insert as the durable record. If it fails, you can often skip the database round trip entirely and return a cached response straight from Redis. This isn't a replacement for the database's unique constraint, since Redis alone doesn't give you the same durability guarantee across a restart, but it's a reasonable optimization once the relational check starts showing up in your latency numbers.

A Mistake This Pattern Specifically Prevents

The most common mistake teams make when they build idempotency logic without a database constraint is checking for an existing key, then doing the work, then writing the key, as three separate steps in application code. That ordering looks correct in a single-threaded test and fails the moment two requests with the same key run concurrently, which is a textbook race condition: both checks run before either write completes, and both proceed to do the real work. The insert-first ordering in Step 3 above isn't a stylistic preference. It's the specific detail that closes this exact race, because the database's constraint check and the write happen as one atomic operation instead of two separate steps your application code has to coordinate.

Step 7: Roll Out to Remaining Endpoints

Once the pattern is proven on one route, adding it to others is mostly copy and wire, since the middleware and the finalize call don't change per endpoint. The main judgment call at each new endpoint is whether it genuinely needs protection, per Step 1, rather than adding the overhead everywhere by default.

How Long This Actually Takes

The steps above are genuinely small in isolation: one migration, one middleware function, one finalize call, one route wired up, one concurrency test. Most of the real time in an afternoon rollout goes into deciding which endpoint to start with and verifying the concurrency test actually fails before your fix and passes after it, not into writing the code itself. If you're doing this for the first time, budget an hour for the schema and middleware, another hour for wiring and manual testing against one route, and the rest for the concurrency test and a second pass at whichever endpoint you pick next.

What You Haven't Solved Yet

This afternoon's work covers the core client-initiated case. It doesn't cover webhook deliveries, which typically arrive with their own event ID rather than a client-supplied key, though the same table and constraint pattern applies with a small adjustment. It also doesn't cover retention: without a cleanup job, the idempotency_keys table grows indefinitely. A scheduled job that deletes rows older than your realistic retry window, a day or two is generous for most clients, keeps lookups fast long term.

For the fuller version of this pattern, including how to scope keys correctly across different operations, handle the conflict case in more detail, and decide on a retention window, 137Foundry has a longer breakdown at How to Design an Idempotency Key Strategy So Retried API Requests Never Double-Process. If you'd rather have someone build this into an existing API rather than doing it endpoint by endpoint yourself, 137Foundry works on exactly this kind of backend reliability work. The PostgreSQL unique constraint used here is standard across most relational databases, so the same approach ports cleanly if you're on a different engine than the one in these examples.