- Book: Event-Driven Architecture Pocket Guide
- Also by me: Thinking in Go (2-book series) — Complete Guide to Go Programming + Hexagonal Architecture in Go
- My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools
- Me: xgabriel.com | GitHub
Your webhook receiver restarts during a deploy. For about a minute, every request gets a connection refused. Two senders hit it in that minute: Stripe, with an invoice.paid, and GitHub, with a push to the main branch.
The Stripe event comes back. Stripe's docs say it "attempts to deliver events to your destination for up to three days with an exponential back off in live mode." The GitHub event does not. GitHub's docs say it plainly: "GitHub does not automatically redeliver failed webhook deliveries." The same page says a server that is down or "takes longer than 10 seconds to respond" gets the delivery recorded as a failure.
So one sender will hand you the same event many times, and the other will hand it to you once and forget about it. A receiver that serves both has to be safe against duplicates and has to go and fetch what it missed. This post builds that receiver in TypeScript on Node.js, Express 5 and PostgreSQL: one SQL table and five short files.
Two senders, two failure contracts
Put the two contracts side by side before writing code, because every design choice below falls out of them.
GitHub wants a 2xx "within 10 seconds of receiving a webhook delivery", per its best-practices page, and recommends a queue so you can acknowledge first and work later. Each delivery carries an X-GitHub-Delivery header, and the same page notes that on a redelivery "the X-GitHub-Delivery header will be the same as in the original delivery." That GUID is your dedupe key. You can redeliver deliveries from the past 3 days, by hand or through the REST API. Nothing happens unless you ask.
Stripe tells you to "quickly return a successful status code (2xx) before any complex logic that could cause a timeout" (webhooks guide). It retries on its own, and in a sandbox it retries only "three times over the course of a few hours." It also warns that endpoints "might occasionally receive the same event more than once" and that it "doesn't guarantee the delivery of events in the order that they're generated." The fix it recommends is to log the event IDs you've processed. A manual resend from the Dashboard does not cancel the automatic retries either, so a resend can produce yet another copy.
That gives the receiver three jobs. Verify the signature, write the event down under its unique ID, and answer 200. All real work moves to a worker that reads the table. A separate scheduled job covers the gap GitHub leaves.
Same mailbox, two senders: one keeps knocking for three days, the other knocks once.
Verify the signature on the bytes you received
Both providers sign with HMAC SHA-256, and both sign the raw request body. GitHub's validation guide says the X-Hub-Signature-256 header "always starts with sha256=", followed by the hex digest, and asks you to make sure "the payload and headers are not modified before verification." Stripe's Stripe-Signature header holds a t= timestamp and one or more v1= signatures; the signed string is the timestamp, a ., then the body (manual verification). Stripe's libraries reject timestamps more than 5 minutes off by default, and each retry gets a fresh timestamp and signature.
If a JSON parser touches the body first, re-serializing it changes the bytes and the HMAC fails. So the routes below take the body as a Buffer, and verification runs before anything parses it.
// verify.ts
import { createHmac, timingSafeEqual } from "node:crypto";
function sameHex(a: string, b: string): boolean {
const x = Buffer.from(a, "hex");
const y = Buffer.from(b, "hex");
return x.length > 0 && x.length === y.length
&& timingSafeEqual(x, y);
}
export function verifyGitHub(
raw: Buffer, header: string | undefined, secret: string,
): boolean {
if (!header?.startsWith("sha256=")) return false;
const want = createHmac("sha256", secret)
.update(raw).digest("hex");
return sameHex(header.slice(7), want);
}
timingSafeEqual is the constant-time comparison both providers ask for. It throws on buffers of different lengths, which is why sameHex checks length first. Stripe's header needs a little parsing before the same comparison can run:
// verify.ts (continued)
const TOLERANCE_S = 300; // matches Stripe's library default
export function verifyStripe(
raw: Buffer, header: string | undefined, secret: string,
): boolean {
if (!header) return false;
let t = "";
const v1: string[] = [];
for (const part of header.split(",")) {
const [k, v = ""] = part.split("=", 2);
if (k === "t") t = v;
else if (k === "v1") v1.push(v);
}
const ts = Number(t);
if (!t || !Number.isInteger(ts) || v1.length === 0) return false;
if (Math.abs(Date.now() / 1000 - ts) > TOLERANCE_S) return false;
const want = createHmac("sha256", secret)
.update(`${t}.`).update(raw).digest("hex");
return v1.some((sig) => sameHex(sig, want));
}
The checker only looks at v1 entries: Stripe's docs say to ignore every other scheme to prevent downgrade attacks, and during a secret roll there can be several v1 signatures, so any match passes. In production you can swap verifyStripe for stripe.webhooks.constructEvent from the official library. I hand-rolled it here so both schemes sit in one file.
The inbox table: one unique key, duplicates become no-ops
The table is the queue. The unique constraint on (source, delivery_id) is what turns Stripe's retries, your manual resends and GitHub's redeliveries into harmless repeats.
CREATE TABLE webhook_inbox (
id bigserial PRIMARY KEY,
source text NOT NULL,
delivery_id text NOT NULL,
event_type text NOT NULL,
payload jsonb NOT NULL,
received_at timestamptz NOT NULL DEFAULT now(),
next_attempt_at timestamptz NOT NULL DEFAULT now(),
attempts int NOT NULL DEFAULT 0,
last_error text,
processed_at timestamptz,
UNIQUE (source, delivery_id)
);
CREATE INDEX webhook_inbox_due
ON webhook_inbox (next_attempt_at)
WHERE processed_at IS NULL;
For GitHub the key is the X-GitHub-Delivery GUID. For Stripe it is the event's id (the evt_... value), not anything in the signature header, since that header changes on every attempt. The insert uses ON CONFLICT DO NOTHING, which Postgres documents as skipping the row instead of raising a unique violation. A duplicate delivery gets a 200 and changes nothing.
The unique key does the dedupe. A second copy of an event hits the constraint and goes nowhere.
A receiver that answers in milliseconds
The receiver does one insert per request and nothing else. Two small helpers first:
// db.ts
import pg from "pg";
export function env(name: string): string {
const v = process.env[name];
if (!v) throw new Error(`${name} is not set`);
return v;
}
export const pool = new pg.Pool(); // PGHOST, PGUSER... from env
env fails at boot on a missing secret. An empty string is a valid HMAC key, so a receiver that defaults the secret to "" would accept signatures anyone can compute.
The dependencies are express, pg (node-postgres) and their @types packages. The files are ES modules ("type": "module" in package.json), and two of them use top-level await, so run them with tsx or compile them with tsc using "module": "nodenext".
// server.ts
import express from "express";
import { env, pool } from "./db.js";
import { verifyGitHub, verifyStripe } from "./verify.js";
const GH_SECRET = env("GITHUB_WEBHOOK_SECRET");
const STRIPE_SECRET = env("STRIPE_WEBHOOK_SECRET");
const raw = express.raw({ type: "application/json", limit: "25mb" });
const app = express();
async function enqueue(
source: string, id: string, type: string, body: Buffer,
): Promise<void> {
await pool.query(
`INSERT INTO webhook_inbox
(source, delivery_id, event_type, payload)
VALUES ($1, $2, $3, $4::jsonb)
ON CONFLICT (source, delivery_id) DO NOTHING`,
[source, id, type, body.toString("utf8")],
);
}
enqueue is the only database call the receiver makes. express.raw only reads bodies whose content type matches, so set your GitHub webhook's content type to application/json; with form encoding the route below answers 415. The 25 MB limit matches GitHub's payload cap.
// server.ts (continued)
app.post("/webhooks/github", raw, async (req, res) => {
const body = req.body;
if (!Buffer.isBuffer(body)) { res.sendStatus(415); return; }
const sig = req.get("x-hub-signature-256");
if (!verifyGitHub(body, sig, GH_SECRET)) {
res.sendStatus(401); return;
}
const id = req.get("x-github-delivery");
const type = req.get("x-github-event");
if (!id || !type) { res.sendStatus(400); return; }
await enqueue("github", id, type, body);
res.sendStatus(200);
});
app.post("/webhooks/stripe", raw, async (req, res) => {
const body = req.body;
if (!Buffer.isBuffer(body)) { res.sendStatus(415); return; }
const sig = req.get("stripe-signature");
if (!verifyStripe(body, sig, STRIPE_SECRET)) {
res.sendStatus(400); return;
}
const event = JSON.parse(body.toString("utf8"));
if (typeof event.id !== "string") {
res.sendStatus(400); return;
}
await enqueue("stripe", event.id, String(event.type), body);
res.sendStatus(200);
});
app.listen(Number(process.env.PORT ?? 3000));
The Buffer.isBuffer check matters on Express 5, where req.body is undefined when no parser ran, per the migration guide. The same guide says a rejected promise inside an async handler is forwarded to the error handler. If Postgres is down, enqueue throws, the sender gets a 500, and the delivery counts as failed. That is the behavior you want: Stripe will retry it, and the job in the last section will ask GitHub to send it again.
The worker drains the inbox
The worker claims one due row with FOR UPDATE SKIP LOCKED. The Postgres docs say it "can be used to avoid lock contention with multiple consumers accessing a queue-like table," so you can run several workers without them grabbing the same event.
// worker.ts
import { setTimeout as sleep } from "node:timers/promises";
import type { PoolClient } from "pg";
import { pool } from "./db.js";
type Row = {
id: string; source: string; event_type: string; payload: unknown;
};
async function handle(db: PoolClient, row: Row): Promise<void> {
// route on row.source + row.event_type; write through `db`
}
async function claimOne(): Promise<boolean> {
const db = await pool.connect();
try {
await db.query("BEGIN");
const { rows } = await db.query<Row>(
`SELECT id, source, event_type, payload
FROM webhook_inbox
WHERE processed_at IS NULL AND attempts < 8
AND next_attempt_at <= now()
ORDER BY next_attempt_at
LIMIT 1
FOR UPDATE SKIP LOCKED`);
const row = rows[0];
if (!row) { await db.query("COMMIT"); return false; }
At this point the transaction holds a lock on one row, and any other worker running the same query skips past it. The rest of claimOne does the work and records the outcome:
// worker.ts (continued, inside claimOne)
try {
await db.query("SAVEPOINT work");
await handle(db, row);
await db.query(`UPDATE webhook_inbox
SET processed_at = now() WHERE id = $1`, [row.id]);
} catch (err) {
await db.query("ROLLBACK TO SAVEPOINT work");
await db.query(`UPDATE webhook_inbox
SET attempts = attempts + 1, last_error = $2,
next_attempt_at = now()
+ interval '30 seconds' * 2 ^ attempts
WHERE id = $1`, [row.id, String(err)]);
}
await db.query("COMMIT");
return true;
} catch (err) {
await db.query("ROLLBACK");
throw err;
} finally {
db.release();
}
}
for (;;) {
if (!(await claimOne())) await sleep(1000);
}
The savepoint is what makes failure handling work. If handle runs a SQL statement that errors, Postgres aborts the transaction, and without the savepoint the attempts update would fail too. With it, the handler's writes roll back, the failure gets recorded, and the row comes back after 30 s, 1 min, 2 min and so on. After eight attempts it stops being picked up; WHERE processed_at IS NULL AND attempts >= 8 is your dead-letter view.
Database writes made through db commit together with processed_at, so a crash between the two can't half-apply an event. Calls to other systems (an email, an API) are outside that transaction and can run twice, so give them their own idempotency key. Keep handlers short too, since the row lock is held while they run.
Two Stripe rules still apply inside handle. Events arrive out of order, so re-fetch the object from Stripe's API instead of trusting the order of rows. And Stripe says that "in some cases, two separate Event objects are generated"; those have different event IDs, so the docs suggest matching on data.object ID plus event.type.
GitHub keeps the failed deliveries for three days. Something on your side has to go and ask for them.
The job GitHub won't run for you
GitHub's failed-deliveries guide describes the fix: a scheduled script that lists recent deliveries, finds the ones whose status isn't OK, and calls the redeliver endpoint. Its example for repository webhooks groups deliveries by guid, skips any GUID with a successful attempt, and redelivers one ID per failed event. Here is the same logic in plain fetch, with the two endpoints from the REST reference: GET .../hooks/{hook_id}/deliveries and POST .../deliveries/{delivery_id}/attempts, which answers 202.
// redeliver-github.ts
import { env } from "./db.js";
const API = "https://api.github.com";
const HOOK = `${API}/repos/${env("GH_REPO")}`
+ `/hooks/${env("GH_HOOK_ID")}`;
const LOOKBACK_MS = 24 * 60 * 60 * 1000;
const headers = {
Accept: "application/vnd.github+json",
Authorization: `Bearer ${env("GH_TOKEN")}`,
"X-GitHub-Api-Version": "2026-03-10",
};
type Delivery = {
id: number; guid: string;
delivered_at: string; status_code: number;
};
function nextPage(link: string | null): string | null {
const m = link?.match(/<([^>]+)>;\s*rel="next"/);
return m?.[1] ?? null;
}
The list endpoint returns at most 100 deliveries per page and puts the next-page cursor in the Link header, which nextPage reads. The loop stops at the first delivery older than the lookback window.
// redeliver-github.ts (continued)
const since = Date.now() - LOOKBACK_MS;
const byGuid = new Map<string, Delivery[]>();
let url: string | null = `${HOOK}/deliveries?per_page=100`;
pages: while (url) {
const res = await fetch(url, { headers });
if (!res.ok) throw new Error(`list failed: ${res.status}`);
for (const d of (await res.json()) as Delivery[]) {
if (Date.parse(d.delivered_at) < since) break pages;
byGuid.set(d.guid, [...(byGuid.get(d.guid) ?? []), d]);
}
url = nextPage(res.headers.get("link"));
}
const ok = (d: Delivery) =>
d.status_code >= 200 && d.status_code < 300;
for (const [guid, tries] of byGuid) {
const first = tries[0];
if (!first || tries.some(ok)) continue;
const res = await fetch(
`${HOOK}/deliveries/${first.id}/attempts`,
{ method: "POST", headers },
);
console.log(guid, res.status); // 202 = queued
}
Run it on a schedule. GitHub's example uses a GitHub Actions cron every six hours; hourly is fine too. The early break assumes the list comes back newest first, the same assumption GitHub's example makes. The token needs write access to repository webhooks: GitHub's example uses a fine-grained token with that permission (or a classic token with repo), and notes the built-in GITHUB_TOKEN in Actions "does not have sufficient permissions to redeliver webhooks." The 2026-03-10 header pins one of the two supported API versions.
GitHub's own example stores the last run time in a repository variable. This one uses a fixed 24-hour window instead. It keeps no state, at the cost of redelivering a persistently failing event on every run. That is harmless here, because the inbox ignores the copies. If the job itself was down, raise LOOKBACK_MS to cover the outage, up to GitHub's 3-day redelivery limit. Organization and GitHub App webhooks have their own deliveries endpoints, so the same pattern works there.
On the Stripe side there is nothing to schedule for the first three days. For an outage longer than that, the Stripe CLI can resend an event up to 30 days after it was created, and the inbox dedupes whatever comes back twice.
If this was useful
The inbox table is the receiving-side twin of the outbox: write the fact down in the same database first, let a worker do the work later, and make every handler safe to run twice. The Event-Driven Architecture Pocket Guide is where I go further into that family of patterns, with the outbox, sagas and CQRS, and the retry and ordering traps that show up once more than two systems are talking to each other.
Top comments (0)