Retries without duplicate sends: idempotency keys, atomic transitions and the double click

You send a request that triggers an e-mail, a push notification, an SMS or a payment. The connection drops. You do not know whether the server received it. Do you send it again?

If you retry, you may send the message twice. If you don't, you may send it zero times. Over a network, the honest answer is that you must retry, so the real work is making a repeat harmless. This article covers the four places where duplicates come from and the pattern that closes each one: the HTTP boundary, the queue, the state change and the button.

Why retrying is not optional

A request can fail in three ways that look identical to the client: the request never arrived, it arrived and failed, or it arrived and succeeded but the response was lost. The client sees a timeout in all three. If your API sends something irreversible, "just retry" is a bug waiting for the third case, and "never retry" is an outage waiting for the first.

The fix is not smarter retries. It is making the operation idempotent: doing it twice has the same effect as doing it once.

1. Idempotency keys at the HTTP boundary

The contract has four rules:

1. The client generates the key, before the first attempt, and sends the same key on every retry.
2. The server stores the first result next to the key and replays it for later calls with the same key.
3. Same key, same body: replay. Do not run the effect again.
4. Same key, different body: error. The client has a bug (it reused a key for a new intent) and silently replaying the old result would hide it. A 422 makes it loud.

The key should name the event, not the attempt: order-1042-shipped is good, a fresh UUID generated inside the retry loop protects nothing. If there is no natural event id, generate a UUID once per user intent and keep it until the call succeeds.

On the server, the unique constraint does the hard part. Two identical requests arriving at the same moment cannot both insert the same key:

CREATE TABLE idempotency_keys (
  project_id   bigint      NOT NULL,
  key          text        NOT NULL,
  request_hash text        NOT NULL,
  message_id   bigint,
  created_at   timestamptz NOT NULL DEFAULT now(),
  PRIMARY KEY (project_id, key)
);

Scope the key to a tenant (here project_id), so two customers choosing welcome-1 don't collide. Hash a canonical form of the body (sorted keys, no whitespace) so equivalent JSON compares equal.

The handler claims the key and creates the effect in the same transaction:

async function acceptSend(tx: Tx, projectId: number, key: string, body: SendBody) {
  const hash = sha256(canonicalJson(body));

  const claimed = await tx.query(
    `INSERT INTO idempotency_keys (project_id, key, request_hash)
     VALUES ($1, $2, $3)
     ON CONFLICT (project_id, key) DO NOTHING`,
    [projectId, key, hash],
  );

  if (claimed.rowCount === 0) {
    const { rows } = await tx.query(
      `SELECT request_hash, message_id FROM idempotency_keys
       WHERE project_id = $1 AND key = $2`,
      [projectId, key],
    );
    if (rows[0].request_hash !== hash) {
      throw new HttpError(422, 'idempotency_key_reused');
    }
    return { status: 'duplicate_ignored', id: rows[0].message_id }; // 202
  }

  const { rows } = await tx.query(
    `INSERT INTO messages (project_id, payload, status)
     VALUES ($1, $2, 'queued') RETURNING id`,
    [projectId, body],
  );
  await tx.query(
    `UPDATE idempotency_keys SET message_id = $3
     WHERE project_id = $1 AND key = $2`,
    [projectId, key, rows[0].id],
  );
  return { status: 'queued', id: rows[0].id }; // 202
}

Under READ COMMITTED, a concurrent request with the same key waits on the unique index until the first transaction commits, then takes the conflict branch and reads the committed row. Because the key and the message commit together, there is no state where the key exists and the message doesn't.

Two details worth deciding on purpose:

• Don't consume the key on a refused request. If validation fails with a 400, store nothing, so the client can fix the body and reuse the key.
• The effect is a row, not a call. The handler enqueues the work; a worker sends it later. Calling the provider inside the request would put a network call between "claimed" and "committed".

Takeaway: a key without a unique constraint is a suggestion. Make the database refuse the second insert.

2. The client: retry only what can succeed later

The client half is a loop that generates the key once and retries only answers where the same request might succeed:

const RETRYABLE = new Set([429, 500, 502, 503, 504]);
const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms));
const backoff = (i: number) => Math.random() * Math.min(30_000, 500 * 2 ** i);

async function postOnce(url: string, body: unknown, key: string, attempts = 5) {
  for (let i = 0; i < attempts; i++) {
    try {
      const res = await fetch(url, {
        method: 'POST',
        headers: { 'Content-Type': 'application/json', 'Idempotency-Key': key },
        body: JSON.stringify(body),
      });
      if (res.ok) return res.json();
      if (!RETRYABLE.has(res.status)) throw new ApiError(res.status, await res.json());
      const retryAfter = Number(res.headers.get('Retry-After'));
      await sleep(retryAfter > 0 ? retryAfter * 1000 : backoff(i));
    } catch (err) {
      if (err instanceof ApiError) throw err; // a 4xx will not change on retry
      await sleep(backoff(i)); // no answer at all: same key, try again
    }
  }
  throw new Error('gave up after retries');
}

// Called once per intent, not once per attempt:
postOnce('/v1/messages', body, `order-${order.id}-shipped`);

The rules behind it: retry 429 and 5xx with growing, randomized waits; honor Retry-After when present; retry a timeout with the same key; never retry a 400, 401, 403, 404 or 422, because the answer will be the same. Add jitter, or a thousand clients that failed together will retry together.

3. At-least-once queues need idempotent consumers

Most queues promise at-least-once delivery. A worker that crashes after doing the work but before acknowledging it gets the same message again. That's a feature, but it means the consumer has to be idempotent too.

When the effect is in your own database, put the dedupe record in the same transaction:

BEGIN;
INSERT INTO processed_messages (consumer, message_id)
VALUES ('mailer', $1)
ON CONFLICT DO NOTHING;
-- if 0 rows were inserted: ROLLBACK and acknowledge, already done
-- otherwise: apply the effect, then COMMIT

When the effect lives outside (an e-mail provider, a push service), you can't share a transaction. Two options: pass the provider a key if it supports one, and record "sent" with the provider's id as soon as you get it. For push specifically there is also a safety net: APNs lets you set apns-collapse-id to merge notifications into one, and FCM's collapse key replaces a pending message with the same key for the same token. These don't replace idempotency, since they only help while the earlier message is still waiting, but they limit the damage of a duplicate.

Takeaway: assume every message arrives twice and write the consumer so that's boring.

4. Atomic state transitions: approvals and refunds

Some duplicates have nothing to do with the network. Two admins click Approve on the same refund. A webhook is delivered twice. The naive code reads the status, checks it, then writes:

const refund = await getRefund(id);
if (refund.status === 'pending') {   // two requests can both pass this check
  await approve(id);
  await payOut(refund);
}

Fix it with a compare-and-set: let the database check and change the status in one statement, and use the number of affected rows as the verdict.

const res = await tx.query(
  `UPDATE refunds
      SET status = 'approved', approved_by = $2, approved_at = now()
    WHERE id = $1 AND status = 'pending'`,
  [id, adminId],
);

if (res.rowCount === 1) {
  await tx.query(`INSERT INTO payouts (refund_id) VALUES ($1)`, [id]); // we won: do the effect
} else {
  const { rows } = await tx.query(`SELECT status FROM refunds WHERE id = $1`, [id]);
  if (rows[0].status !== 'approved') throw new HttpError(409, 'refund_not_pending');
  // already approved by someone else: report success, do nothing
}

Exactly one caller sees rowCount === 1. The loser gets a clear answer, and doing the payout in the same transaction means it can't be skipped or doubled. If you prefer ORM-style updates, a version column checked in the WHERE gives the same guarantee.

Takeaway: don't check state in application code and write in another statement. Put the expected state in the WHERE.

5. The UI: a button that can only be pressed once

Most duplicates start with a human. Double click, slow network, impatient second tap. Three layers, cheapest first:

1. Disable the button while the request is pending.
2. Don't let the framework retry mutations behind your back. In TanStack Query, mutations are not retried on error by default, but set retry: false explicitly so a global default change doesn't silently turn a POST into a repeat.
3. Create the key once per intent, so even a double click that slips through sends the same key twice.

function ShipButton({ order }: { order: Order }) {
  const key = useRef(crypto.randomUUID()); // one key per intent
  const ship = useMutation({
    mutationFn: () => api.ship(order.id, key.current),
    retry: false,
    onSuccess: () => { key.current = crypto.randomUUID(); },
  });

  return (
    <button
      disabled={ship.isPending}
      onClick={() => { if (!ship.isPending) ship.mutate(); }}
    >
      Mark as shipped
    </button>
  );
}

disabled only updates after a render, so two clicks can land in the same frame; the isPending guard and the shared key are what actually protect you. crypto.randomUUID() needs a secure context (HTTPS).

A checklist

• The client generates the key from the event and reuses it on every retry.
• A unique constraint on (tenant, key); the key and the effect commit together.
• Same key plus different body is a 422, not a replay.
• Retry 429 and 5xx and timeouts, with backoff, jitter and Retry-After. Never retry other 4xx.
• Queue consumers dedupe by message id in the same transaction as the effect.
• State changes are compare-and-set; rowCount decides who won.
• UI: disabled while pending, retry: false on mutations, one key per intent.

About the project

I work on Espresso Notify, a push notification service for iOS, Android and web with a dashboard and a REST API. Its send routes follow this contract, with the key as a field of the JSON body, idempotency_key (up to 128 bytes). Repeating a send with the same key and body answers 202 with duplicate_ignored and the id of the first send; the same key with another body answers 422 idempotency_key_reused; a refused call stores nothing, so the same key works once you fix the request. The docs include the retry table (429 with Retry-After, 5xx, timeouts with the same key) and the Node library adds a random key per call when you don't pass one. It is free up to 10,000 devices.

• The contract and the retry table: https://docs.espressonotify.com/docs/api/reliable-sending
• The Node library's retries and key: https://docs.espressonotify.com/docs/server-libraries/node

A question for the comments: where do you keep the line between "the same request" and "a different body" when you compare bodies for a key? Canonical JSON hash, a subset of fields, or something else? And how do you handle a key whose first request is still in flight when the retry arrives?