TL;DR: If a webhook signature check starts failing after deploy, the body was probably parsed by middleware before verification. Check the signature against the exact bytes received, before any JSON middleware runs. Give each signing secret a non-secret key ID, accept old and new keys only during a bounded rotation window, and record which key verified each event. For a property-management system, that turns a leaked-key drill from a hopeful secret replacement into a 30-minute procedure with evidence.
The common post-deploy failure has a narrow cause: the sender signed one byte sequence, while the server checked another. express.json() consumes the stream and replaces req.body with a JavaScript value. Serializing that value again is not a recovery strategy; whitespace, escaping, duplicate keys, or number representation may differ even when the JSON means the same thing.
How should a webhook signature check handle parsed middleware after deploy?
Middleware order is executable security policy. A local route may receive a Buffer, then fail in production because an application-level express.json() was registered first. By the time the verification handler runs, the original bytes are gone. No choice of hash library fixes that boundary.
This can be confusing because ordinary JSON tests use compact fixtures. The trap appears when a sender changes insignificant formatting, a proxy path reaches a different Express stack, or a refactor moves the global parser above the webhook route. Imagine a valid payment event arriving as pretty-printed JSON: a parser turns it into the same object as the compact fixture, so every business assertion passes, but rebuilding the JSON removes the original spaces and produces different HMAC input. The payload still parses. The MAC does not match. That split result is useful evidence: parsing succeeded while byte-level authentication failed.
Do not sign JSON.stringify(req.body). Preserve the body.
In the property-management example below, a lease.payment_recorded event updates a resident ledger. That is sensitive enough to reject on uncertainty: no valid signature means no ledger mutation, no acknowledgement that implies success, and no verbose response that reveals which candidate key almost matched.
Put the byte boundary in code
The example defines a small signing contract: the sender places Unix time in x-webhook-timestamp, a key identifier in x-webhook-key-id, and a lowercase hex HMAC-SHA-256 in x-webhook-signature. The signed message is the ASCII timestamp, one period, then the untouched request bytes. Your actual sender's specification is authoritative; header names and message construction are protocol details, not interchangeable conventions.
import express, { Request, Response } from "express";
import { createHmac, timingSafeEqual } from "node:crypto";
const app = express();
const MAX_AGE_SECONDS = 300;
type SigningKey = { id: string; secret: Buffer };
// Populate this ring from a secret manager during startup or rotation.
const signingKeys: SigningKey[] = [
{ id: "primary-42", secret: Buffer.from(process.env.WEBHOOK_SECRET!, "utf8") },
];
function verify(req: Request): { ok: boolean; keyId?: string; reason?: string } {
if (!Buffer.isBuffer(req.body)) return { ok: false, reason: "body_not_raw" };
const timestampText = req.header("x-webhook-timestamp");
const suppliedHex = req.header("x-webhook-signature");
const requestedKeyId = req.header("x-webhook-key-id");
if (!timestampText || !suppliedHex || !requestedKeyId) {
return { ok: false, reason: "missing_header" };
}
const timestamp = Number(timestampText);
const now = Math.floor(Date.now() / 1000);
if (!Number.isInteger(timestamp) || Math.abs(now - timestamp) > MAX_AGE_SECONDS) {
return { ok: false, reason: "stale_timestamp" };
}
if (!/^[0-9a-f]{64}$/.test(suppliedHex)) {
return { ok: false, reason: "malformed_signature" };
}
const key = signingKeys.find((candidate) => candidate.id === requestedKeyId);
if (!key) return { ok: false, reason: "unknown_key_id" };
const signed = Buffer.concat([
Buffer.from(`${timestampText}.`, "ascii"),
req.body,
]);
const expected = createHmac("sha256", key.secret).update(signed).digest();
const supplied = Buffer.from(suppliedHex, "hex");
return timingSafeEqual(expected, supplied)
? { ok: true, keyId: key.id }
: { ok: false, reason: "signature_mismatch" };
}
app.post(
"/webhooks/lease-events",
express.raw({ type: "application/json", limit: "256kb" }),
(req: Request, res: Response) => {
const result = verify(req);
const receivedAt = new Date().toISOString();
if (!result.ok) {
console.warn(JSON.stringify({
action: "webhook.verify",
outcome: "rejected",
reason: result.reason,
receivedAt,
}));
res.sendStatus(401);
return;
}
let event: { id?: string; type?: string; propertyId?: string };
try {
event = JSON.parse(req.body.toString("utf8"));
} catch {
res.sendStatus(400);
return;
}
console.info(JSON.stringify({
action: "webhook.verify",
outcome: "accepted",
keyId: result.keyId,
eventId: event.id,
eventType: event.type,
propertyId: event.propertyId,
receivedAt,
}));
res.sendStatus(204);
},
);
// Other JSON endpoints can use parsed bodies after the raw webhook route.
app.use(express.json());
app.listen(3000);
The length and hexadecimal checks happen before timingSafeEqual because Node requires equal-length buffers. The five-minute freshness window limits acceptance of old signed messages, but it does not provide full replay protection. Store each accepted event ID with an expiry and reject a duplicate before applying the ledger change. Make the ledger write idempotent as well; delivery retries are normal even when no attacker is present.
A 256 KB route limit is a deliberate resource boundary, not a universal recommendation. Set it just above the largest legitimate event observed in contract tests. A global multi-megabyte limit spends memory on a path that should carry small event envelopes.
This approach has limits. A raw-body route consumes memory proportional to the accepted payload and is a poor fit for large streamed deliveries; for those, choose a protocol and framework path that authenticates a stream incrementally. Route ordering is also less attractive in an application whose framework owns all body parsing. In that case, use the framework's documented raw-body capture hook and prove with an integration test that the captured buffer is the pre-parse input. The trade-off is clear: route-specific raw parsing is easy to audit, while a global capture hook reduces routing constraints but retains a copy of every matching request unless its scope is kept tight.
I'd keep that scope narrow.
Run the leaked-key drill as a state transition
Start the timer when the incident owner marks primary-42 exposed. Create a replacement through the approved secret system, assign it a new key ID, and configure the sender to sign with it. During a short, declared overlap, the verifier may hold both keys and select by key ID. This avoids trying every secret and makes the audit record unambiguous.
Then send three controlled events: one signed by the new key, one by the retiring key, and one with an invalid signature. The expected evidence is specific. The new key succeeds, the retiring key succeeds only while the overlap is open, and the invalid event receives 401 without touching the resident ledger. After the sender has switched, remove the retiring key and repeat its test; it must now fail.
Keep the rotation boring.
The audit trail should answer who authorized the rotation, when each key became active or inactive, which key ID verified an event, and whether downstream processing committed. It should not contain the secret, the signature value, or the raw resident payload. OWASP recommends lifecycle management, least privilege, rotation, revocation, and auditing for secrets; the drill exercises those controls as one chain rather than treating rotation as a console action.
Auditability changes the design choices
A single secret with no identifier is easy to start with, but weak during response: an accepted event cannot be attributed to the old or new credential. A key ring with explicit IDs costs a little configuration and gives deterministic selection, bounded overlap, and useful evidence. That trade is worthwhile for access auditing.
Likewise, logging every body makes debugging feel faster while creating a second sensitive-data store. Structured metadata is enough for the verification decision. Correlate eventId, propertyId, keyId, outcome, reason, and receipt time, then protect and retain those logs according to the same access policy used for other security records.
Tests should cover byte-level behavior, not just parsed objects. Sign fixtures containing spaces, escaped Unicode, and alternate key order; assert that the original bytes pass and a one-byte change fails. Add an integration test that boots the production middleware order. Finally, exercise stale timestamps, malformed hex, unknown key IDs, duplicate event IDs, oversized bodies, and JSON that is authentic but invalid for the domain. Authentication happens before parsing, while schema validation and authorization happen after it.
For deployment, expose counts for accepted and rejected verification outcomes, grouped by low-cardinality reason and key ID. Alert on a sudden rejection increase after a release or rotation, but keep event IDs out of metric labels. Watch latency too: secret lookup should not become a network round trip on every delivery. Load the active ring through a controlled refresh path, constrain access to the webhook process, and erase retired material from memory when the overlap closes.
The operational finish line
Before declaring the drill complete, confirm the route receives a Buffer, the production parser order matches the test, and the replacement key verifies a known fixture. Confirm the old key has been removed after the overlap, a replayed event cannot repeat a ledger mutation, and rejection logs contain a stable reason without payload or credential material. An operator who did not perform the rotation should be able to reconstruct its authorization, timing, key transition, test outcomes, and downstream effect from the records alone.
That is the decision rule: preserve bytes first, rotate by explicit key ID, and make every security transition independently auditable. The signature check is only one line of defense. The drill proves the whole path.
Top comments (0)