Short answer: webhook signature verification should authenticate the exact raw body in a route-specific ingress handler, then parse JSON and record marketplace usage; prefer that boundary over application-wide byte capture unless every route genuinely needs it.
This is an ordering decision with a security consequence. A marketplace invoice can be corrected after a malformed event is rejected, but accepting an event whose authenticity was checked against a different representation is much harder to unwind. The invariant is strict: the bytes covered by the sender's signature must be the same bytes supplied to the verifier. Object key order, whitespace, escaping, and numeric rendering can all change when a payload becomes an object and is serialized again, even when its apparent data is unchanged.
Bytes first.
For a metered account platform, I would isolate webhook authentication from the usage ledger and isolate credentials by sender or tenant wherever the operational model permits. One shared signing secret is convenient, yet its compromise expands from one customer's events to every customer's invoice input. The catch is that per-customer secrets create rotation, lookup, and audit work; a small deployment with one trusted sender may rationally keep one secret, provided its blast radius is documented rather than ignored.
What invariants define the webhook boundary?
The handler owns four invariants. It retains the raw body unchanged, resolves the intended verification key without trusting unverified business fields, compares a computed message authentication code in constant time, and permits JSON parsing only after authentication succeeds. Timestamp or replay controls belong at this same boundary when the sender's signing contract defines them, but their precise canonical message must come from that contract. Guessing whether a timestamp, delimiter, or header is signed is worse than omitting sample code for it.
The failure boundaries should be equally explicit. Missing or malformed authentication metadata gets a 401; a signature mismatch gets a 401; authenticated bytes that aren't valid JSON get a 400; a valid duplicate returns the ledger's already-recorded result rather than adding usage twice. That last rule is not signature verification. It is idempotency, and conflating the two leaves invoices exposed to legitimate retries.
Credential selection needs care because customer_id inside the body is still untrusted at verification time. Select a key from authenticated transport context or a sender identifier in the signed header contract, then require the verified payload's customer to match the resolved account. Never parse the body early merely to discover which secret should verify it. If the only available tenant selector is inside the unsigned payload, the protocol cannot securely support per-tenant key selection; use a different authenticated selector or accept and document a shared-key boundary.
Keep the boundary small.
The ledger behind it should enforce a uniqueness key such as (sender_id, event_id) and write the usage delta plus receipt metadata in one transaction. A valid signature proves possession of a key and integrity under the defined signing scheme; it doesn't prove that a new event hasn't been delivered before, that its unit count is sensible, or that it belongs in the current billing period. Those are separate checks, with separate alerts.
How should Node.js Express middleware preserve the raw body for webhook signature verification?
Mount a route-scoped raw parser before the general JSON parser. In Express, express.raw({ type: "application/json" }) produces a Buffer for the matching request, so the webhook route can authenticate it before calling JSON.parse; mount express.json() afterward for ordinary routes. If a global JSON parser runs first, changing the later handler can't recover the original byte stream.
Middleware order is the control surface. The intended sequence is raw bytes -> header validation -> key lookup -> signature comparison -> JSON parse -> schema validation -> idempotent ledger write. Do not turn the raw body into text before verification unless the signing specification explicitly defines a text encoding and canonicalization procedure. HMAC operates on bytes, and the receiver must implement the sender's exact contract.
All executable code below is Python because the critical path is easier to inspect without framework lifecycle details. It models the same boundary an Express route must preserve: body is a byte string supplied by the route-scoped raw parser, while parsing happens only after compare_digest succeeds.
import hashlib
import hmac
import json
from dataclasses import dataclass
from typing import Any
@dataclass(frozen=True)
class VerifiedUsageEvent:
sender_id: str
event_id: str
customer_id: str
units: int
def verify_and_decode(
body: bytes,
signature_header: str,
sender_id: str,
secret_by_sender: dict[str, bytes],
) -> VerifiedUsageEvent:
prefix = "v1="
if not signature_header.startswith(prefix):
raise PermissionError("missing supported signature version")
supplied_hex = signature_header[len(prefix):]
try:
supplied = bytes.fromhex(supplied_hex)
except ValueError as exc:
raise PermissionError("malformed signature") from exc
secret = secret_by_sender.get(sender_id)
if secret is None:
raise PermissionError("unknown sender")
expected = hmac.digest(secret, body, hashlib.sha256)
if not hmac.compare_digest(expected, supplied):
raise PermissionError("signature mismatch")
try:
payload: dict[str, Any] = json.loads(body)
except (UnicodeDecodeError, json.JSONDecodeError) as exc:
raise ValueError("authenticated body is not valid JSON") from exc
return VerifiedUsageEvent(
sender_id=sender_id,
event_id=str(payload["event_id"]),
customer_id=str(payload["customer_id"]),
units=int(payload["units"]),
)
The v1= envelope in this example is an application protocol choice, not a universal webhook format. In a real integration, copy the sender's documented algorithm, encoding, signed-message construction, and version negotiation exactly. I'm not sure a generic replay window can be prescribed responsibly: clock tolerance, retry duration, and delayed delivery semantics vary, so the sender's protocol and the marketplace's duplicate-retention requirement must settle it.
There is another sharp edge. Constant-time comparison functions generally expect compatible types and lengths; decode the supplied representation deliberately and treat bad hex as failed authentication. Don't log the supplied signature, computed digest, or secret. Log a request correlation ID, the resolved sender identity, a coarse rejection reason, and the outcome instead.
Route-scoped bytes versus global capture
Both designs can preserve the signed representation, but they create different ownership and failure modes.
| Decision | Route-scoped raw body | Global raw-body capture |
|---|---|---|
| Best fit | A few signed webhook endpoints | Many endpoints governed by one byte-level policy |
| Parser order | Explicit at the webhook route | Centralized and easy to apply too broadly |
| Memory exposure | Limited to matching requests | Every captured request can retain an extra body copy |
| Accidental use | Ordinary handlers receive parsed objects | Unrelated handlers may start depending on raw bytes |
| Operational catch | New webhook routes must opt in correctly | Body-size and content-type policy affect a wider surface |
| Preferred choice here | Yes: invoice ingestion has a narrow trust boundary | Only if byte authentication is truly platform-wide |
For this marketplace, route-scoped handling wins because a mistake affects one ingress class rather than every JSON endpoint. Set a body-size limit appropriate to the sender contract before buffering, reject unexpected content types, and keep the handler free of decompression or mutation that isn't specified by the signing protocol. The exact limit is workload-dependent; evidence from observed legitimate payload sizes and the sender's documented maximum should set it, with margin and an alert near the threshold.
Global capture remains valid when a gateway authenticates every downstream request over the original representation, or when a framework adapter centrally guarantees raw-byte availability without changing handler semantics. It is not suitable when most routes have no byte-signature requirement and teams can quietly begin treating a retained buffer as ordinary request state. Broader convenience means broader memory and policy blast radius.
Credential blast radius belongs in the architecture record
A correct parser order cannot compensate for a secret shared too widely. Store signing secrets in a managed secret system, encrypt them at rest, restrict read access to the verifier, rotate them under a documented process, and avoid putting them in source code or routine logs. OWASP's secrets guidance also emphasizes lifecycle concerns such as creation, rotation, revocation, expiration, and auditing; verification code is only one consumer within that lifecycle.
Three credential layouts deserve an explicit decision:
- One platform-wide key minimizes lookup and rotation plumbing, but one disclosure can authorize forged usage across the entire sender population.
- One key per external sender confines compromise to that integration and usually maps cleanly to an authenticated sender identifier.
- One key per marketplace customer offers the narrowest customer blast radius, but only if the protocol exposes a trustworthy pre-verification selector and the team can operate many rotations without orphaning deliveries.
The middle option is often the defensible starting point, not a law. Stick with a shared key when there is exactly one sender, its trust boundary already covers all tenants, and extra key granularity would be fictional isolation. Move toward per-customer keys when customers control independent senders or when contractual isolation requires a single customer's credential incident to remain local. During rotation, accept old and new key identifiers only for a bounded overlap defined by the protocol; record which key version authenticated an event, never the secret itself.
This is where storage architecture enters the security argument. A secret lookup failure must not degrade into an unsigned ledger write, and a retry after a successful verification must not double the units. The verifier should fail closed, while the transactional ledger should make duplicate acceptance harmless. Monitor rejection counts by sender, duplicate rates, payload-size percentiles, verification latency, and ledger conflicts. Sudden signature failures can indicate stale rotation state or hostile traffic, but the metric alone cannot distinguish them, so retain correlation metadata without retaining secrets.
Testing the rejected path and documenting its valid use
Test byte preservation with payload pairs that parse to equivalent objects but have different bytes: reordered keys, inserted spaces, escaped Unicode, and 1 versus 1.0. Sign one byte sequence and submit the other. Verification must fail before schema or ledger code runs. Then test malformed hex, an unknown sender, validly signed invalid JSON, duplicate event_id values, a body above the configured limit, and concurrent delivery of the same event.
The rejected design for this system is “parse globally, reserialize, then verify.” Its failure is structural: serialization creates a new representation, and no amount of careful object comparison proves it matches the signed bytes. A canonical JSON signing standard could make parsed-and-canonicalized verification valid, but only when both parties explicitly implement that standard and its exact rules. It isn't a retrofit to an opaque-body HMAC contract.
Global raw-body capture is not rejected everywhere. It remains a reasonable choice for a dedicated webhook process where all routes share the same authentication boundary, request limits are centralized, and no ordinary application endpoints inherit the buffer. That is the condition boundary. For a mixed account platform, use route-scoped raw bytes, verify first, parse second, and let a uniqueness constraint protect the invoice ledger from authenticated retries.
Top comments (0)