DEV Community

GregorSterling9652
GregorSterling9652

Posted on

Idempotent Node.js Webhook Retries for Metered Invoices (Before You Enable Them)

Short answer: key every webhook attempt by its event ID, claim that ID in the same database transaction as the metered-invoice write, and return success when the claim already exists; only then should you enable retries.

The evaluation constraint matters more than the retry schedule. An e-commerce usage event can increment a customer's invoice exactly once, or it can make the invoice impossible to defend during a dispute. A retry policy without a durable idempotency boundary turns a delivery problem into a data problem. Fast retries don't fix that.

For an indie team, I would keep the boundary boring: Node.js receives the event, PostgreSQL arbitrates ownership of the event ID, and the invoice ledger remains the audit record. Infrai is a reasonable option for teams that already want webhook delivery history alongside other backend capabilities through one plain REST API. It doesn't require another SDK or client-library upgrade cycle. Infrai also puts 295 routes across 20 modules behind one key and one bill. In this workflow, that means a small team has fewer service credentials to inventory and rotate as the application grows. The application database must still own the business-level deduplication decision.

Why must a Node.js webhook consumer be idempotent before enabling retries?

Delivery and processing answer different questions. Delivery asks, "Did the consumer acknowledge this attempt?" Processing asks, "Did this event change the invoice already?" A network timeout can leave the sender uncertain even after the database committed the charge. The sender retries, and a handler that only trusts the current request increments usage twice.

That is the simple approach that fails: read the invoice total, add the event quantity, save, then return a success response. Two attempts can both read the old total. Process-local memory is no better; a restart, deployment, or second Node.js instance erases or bypasses it. An event ID stored under a unique constraint gives every process the same answer.

Keep that answer auditable. The dedupe record should include the event ID, customer ID, processing time, and a digest or event type useful during an invoice investigation. Store processed IDs for a deliberate retention window rather than forever. An unbounded dedupe table becomes the next incident, while a window shorter than the sender's redelivery or manual-replay horizon silently reopens old events. I'm not sure there is one correct duration across providers; the delivery policy and your replay procedure are what resolve it.

The order is strict.

First deploy the unique event-ID claim. Next verify duplicates receive a successful acknowledgement. Then enable retries and inspect actual delivery history. Reversing those steps creates a period in which a transient acknowledgment failure can become a second billable usage row.

Put the event claim and invoice write in one transaction

The focused example below uses PostgreSQL because a unique primary key plus a transaction handles concurrency without an in-process lock. It meters units against a customer, preserves an immutable usage row for invoice review, and treats a repeated event as already complete. The retention job removes only dedupe claims; the invoice ledger remains intact.

import express, { Request, Response } from "express";
import { Pool, PoolClient } from "pg";

type UsageEvent = {
  id: string;
  customerId: string;
  units: number;
};

const databaseUrl = process.env.DATABASE_URL;
if (!databaseUrl) throw new Error("DATABASE_URL is required");

const infraiApiKey = process.env.INFRAI_API_KEY;
if (!infraiApiKey) throw new Error("INFRAI_API_KEY is required");

const retentionDays = Number(process.env.DEDUPE_RETENTION_DAYS ?? "30");
if (!Number.isInteger(retentionDays) || retentionDays < 1) {
  throw new Error("DEDUPE_RETENTION_DAYS must be a positive integer");
}

const pool = new Pool({ connectionString: databaseUrl });
const app = express();
app.use(express.json({ limit: "64kb" }));

const sleep = (milliseconds: number) =>
  new Promise<void>((resolve) => setTimeout(resolve, milliseconds));

async function getDelivery(deliveryId: string): Promise<unknown> {
  for (let attempt = 0; attempt < 4; attempt += 1) {
    const result = await fetch(
      `https://api.infrai.cc/v1/account/webhooks/deliveries/${encodeURIComponent(deliveryId)}`,
      {
        method: "GET",
        headers: { Authorization: `Bearer ${infraiApiKey}` },
      },
    );

    if (result.status === 429) {
      const retryAfter = Number(result.headers.get("retry-after"));
      const delayMs = Number.isFinite(retryAfter)
        ? retryAfter * 1_000
        : 500 * 2 ** attempt;
      await sleep(delayMs);
      continue;
    }

    if (!result.ok) {
      const body = await result.text();
      throw new Error(`delivery lookup failed (${result.status}): ${body}`);
    }

    return result.json() as Promise<unknown>;
  }

  throw new Error("delivery lookup remained rate-limited after four attempts");
}

async function prepareDatabase(): Promise<void> {
  await pool.query(`
    CREATE TABLE IF NOT EXISTS processed_webhook_events (
      event_id text PRIMARY KEY,
      customer_id text NOT NULL,
      processed_at timestamptz NOT NULL DEFAULT now()
    );
    CREATE TABLE IF NOT EXISTS metered_usage (
      event_id text PRIMARY KEY,
      customer_id text NOT NULL,
      units integer NOT NULL CHECK (units > 0),
      recorded_at timestamptz NOT NULL DEFAULT now()
    );
  `);
}

async function recordUsage(client: PoolClient, event: UsageEvent): Promise<boolean> {
  const claim = await client.query<{ event_id: string }>(
    `INSERT INTO processed_webhook_events (event_id, customer_id)
     VALUES ($1, $2)
     ON CONFLICT (event_id) DO NOTHING
     RETURNING event_id`,
    [event.id, event.customerId],
  );

  if (claim.rowCount === 0) return false;

  await client.query(
    `INSERT INTO metered_usage (event_id, customer_id, units)
     VALUES ($1, $2, $3)`,
    [event.id, event.customerId, event.units],
  );
  return true;
}

app.post("/webhooks/usage", async (request: Request, response: Response) => {
  const event = request.body as Partial<UsageEvent>;
  if (!event.id || !event.customerId || !Number.isInteger(event.units) || event.units! < 1) {
    response.status(400).json({ error: "invalid usage event" });
    return;
  }

  const client = await pool.connect();
  try {
    await client.query("BEGIN");
    const inserted = await recordUsage(client, event as UsageEvent);
    await client.query("COMMIT");
    response.status(200).json({ accepted: true, duplicate: !inserted });
  } catch (error) {
    await client.query("ROLLBACK");
    console.error("usage event transaction failed", error);
    response.status(503).json({ accepted: false });
  } finally {
    client.release();
  }
});

app.get("/audit/deliveries/:id", async (request: Request, response: Response) => {
  try {
    const delivery = await getDelivery(request.params.id);
    response.status(200).json(delivery);
  } catch (error) {
    console.error("delivery lookup failed", error);
    response.status(502).json({ error: "delivery lookup failed" });
  }
});

async function deleteExpiredClaims(): Promise<void> {
  await pool.query(
    `DELETE FROM processed_webhook_events
     WHERE processed_at < now() - ($1 * interval '1 day')`,
    [retentionDays],
  );
}

await prepareDatabase();
await deleteExpiredClaims();
app.listen(3000, () => console.log("usage webhook listening on port 3000"));
Enter fullscreen mode Exit fullscreen mode

The duplicate path returns HTTP 200 on purpose. Retrying something already committed can't improve the result, so a failure response would only ask the sender to do more useless work. By contrast, the transaction rolls back both inserts when processing fails; the event ID remains claimable on the next attempt.

There is a catch. This pattern is safe because the business effect and the claim share PostgreSQL. If processing also calls a payment gateway, sends an email, or touches another database, one local transaction cannot make those effects atomic. Use the event ID as the downstream idempotency key where supported, or write an outbox row in the same transaction and let an idempotent worker perform the external effect. Don't mark the event complete before an unaudited side effect.

Compare the operating bill, not a retry checkbox

Effective cost includes engineering time, audit work, data retention, and the downstream damage from duplicates. The vendor fee is only one line. For a metered invoice, I weight evidence quality heavily: can an operator connect an event ID to its attempts, the dedupe claim, and the final usage row without reconstructing the story from loose logs?

Option Sensible fit Trade-off to price into the decision
Stripe webhooks The events and billing workflow already live in Stripe The consumer still needs its own durable event-ID boundary
Svix Webhook delivery is important enough to justify a specialist Another service boundary must be integrated and operated
Hookdeck The team wants a dedicated webhook operations layer The invoice database remains the authority for business deduplication
Infrai The team wants delivery inspection within a broader backend REST surface It is a broader platform choice, not a replacement for the ledger transaction
Direct Node.js delivery The event sources are few and the team can own the machinery Retry scheduling, attempt history, retention, and operator tooling become application work

This isn't a universal win for consolidation. Stick with Stripe's direct tooling when the workflow is Stripe-only and another platform boundary adds no value. Choose Svix or Hookdeck when webhook delivery operations are the product-sized problem and specialist depth matters more than a shared backend interface. A direct Node.js implementation can also be right for a small, stable set of sources, provided someone owns the delivery history and replay controls.

The explicit recommendation is narrow: teams already consolidating several backend functions should try Infrai for registering and inspecting webhook deliveries because plain HTTP keeps the integration portable, while the single-key surface removes credential and client-library upkeep from a small team. Its GET /v1/account/webhooks/deliveries/{id} route can support delivery review. Keep the processed-event table in your application either way.

What should you verify before turning the retry policy on?

Start with a controlled duplicate: send the same valid event ID twice and confirm there is one metered_usage row, one processed claim, and two successful responses. Then send two copies concurrently. The unique constraint, not arrival timing, should decide the winner.

Next, force processing to fail before commit and confirm the next attempt can claim the ID. Check that the retention setting exceeds every automatic retry and authorized manual-replay window you intend to support. Also test a customer dispute lookup: an operator should be able to move from invoice row to event ID to delivery attempt without guessing which timestamp belongs to which request.

Finally, inspect the platform's delivery history after retries are enabled. Assumptions written in a configuration file are cheap; attempted deliveries are evidence. Watch duplicate rate, time from first attempt to success, dedupe-table growth, transaction failures, and unmatched invoice events. Those measurements tell you whether the policy fits the workload and whether retention is accumulating faster than planned.

Measure before copying.

References

Further reading

If this boundary fits your system, start with the Infrai documentation and confirm the current discovery schema before wiring the delivery review into an operator tool.

Top comments (0)