Short answer: put signature verification on the raw request body, compare the HMAC in constant time, then parse JSON and dispatch an idempotent job. Keep the raw bytes available only for that boundary; do not pass them through the rest of the application.\n\nA webhook verifier has one non-negotiable input: the exact bytes the sender signed. The practical choice is to capture those bytes before Express (or any JSON parser) turns them into an object. Parse first and you can get a valid-looking payload with an invalid signature, which is the kind of failure that becomes a replay or delivery incident instead of a clean 401.\n\nI have been paged for missed jobs and duplicate deliveries. The lesson is unglamorous: authentication and delivery semantics are separate controls. A correct signature proves who signed these bytes. It does not prove that this event has not already been processed.
Why does raw body order matter in Express middleware?
JSON has more than one valid spelling. Whitespace, key order, escaped characters, and a trailing newline can all change the byte sequence while representing the same object. express.json() intentionally discards those spelling details. Re-serializing the parsed object produces a new string, so an HMAC over that string is not the sender's HMAC.
The safe pipeline is narrow:
- Read the request bytes.
- Verify the signature against those bytes and a key selected for the tenant.
- Parse JSON only after verification succeeds.
- Validate the event schema, timestamp, and event ID.
- Enqueue work behind an idempotency check.
In a Node.js service, that means the route that receives the webhook must run a raw-body parser (or capture hook) before the global JSON parser. Mounting express.json() globally and hoping a later handler can recover the original bytes is the pitfall. It cannot.
The same boundary is visible in this small Go handler. It reads once, verifies once, and only then decodes JSON:
package webhook
import (
"crypto/hmac"
"crypto/sha256"
"crypto/subtle"
"encoding/hex"
"encoding/json"
"io"
"net/http"
)
type Event struct {
ID string `json:"id"`
Type string `json:"type"`
}
func VerifyAndDecode(w http.ResponseWriter, r *http.Request, secret []byte) {
raw, err := io.ReadAll(io.LimitReader(r.Body, 1<<20))
if err != nil {
http.Error(w, "bad request", http.StatusBadRequest)
return
}
provided, err := hex.DecodeString(r.Header.Get("X-Signature-SHA256"))
if err != nil {
http.Error(w, "unauthorized", http.StatusUnauthorized)
return
}
mac := hmac.New(sha256.New, secret)
mac.Write(raw)
expected := mac.Sum(nil)
if len(provided) != len(expected) || subtle.ConstantTimeCompare(provided, expected) != 1 {
http.Error(w, "unauthorized", http.StatusUnauthorized)
return
}
var event Event
if err := json.Unmarshal(raw, &event); err != nil || event.ID == "" {
http.Error(w, "invalid event", http.StatusBadRequest)
return
}
w.WriteHeader(http.StatusAccepted)
}
The 1 MiB limit is an example guardrail, not a universal policy. Set it from your provider contract and measure rejected payloads before tightening it. The important invariant is the ordering, not the number.
The incident pattern: valid JSON, rejected signature
A common postmortem starts with a harmless refactor: a team adds express.json() to the application entry point. A webhook route still receives JSON, so smoke tests pass. Production signatures fail because the parser normalized the body first. Retries pile up, operators see a stream of 401 responses, and someone is tempted to disable verification.
Do not disable it. Return 401 for an invalid signature and 400 for malformed authenticated JSON. Log a request ID, tenant ID, event ID when available, and a reason code, but never log the secret or the full payload. A bounded hash of the raw body can help correlate retries without exposing student data.
Signature headers also need a replay window. Require a sender timestamp, reject values outside a small clock-skew allowance, and store the event ID (or a digest) with a uniqueness constraint before enqueuing work. A signature check without replay protection authenticates an old request just as well as a new one.
How should a tenant-scoped key limit blast radius?
The key should identify one tenant and one purpose, with the smallest permission set that can accept the webhook. Keep verification secrets in a managed secret store, rotate them with an overlap period, and make the active key version observable. OWASP's Secrets Management Cheat Sheet recommends lifecycle controls such as rotation, access policy, and auditing; those controls matter more than a clever header format.
For an edtech platform, a credential leak for tenant school-204 must not authorize events for school-205. Resolve the tenant from a trusted key identifier, load only that tenant's secret, and enforce the tenant again when writing the event record. Never accept a tenant ID solely from the JSON body.
There is a trade-off here. Per-tenant keys increase rotation and lookup work, and a shared key is operationally simpler. Shared keys are not suitable when one tenant's compromise must have a tightly bounded blast radius; use per-tenant material in that case. Conversely, for an internal queue where the producer and consumer already share a private network and an authenticated identity layer, a signed webhook may add complexity without reducing meaningful risk.
Testing and operations that keep the fix in place
Write a fixture test with two byte strings that decode to the same JSON value but differ in whitespace. One must verify and the other must fail. Add a middleware-order test that sends a signed request through the real Express stack, not just a handler unit test. Include duplicate event IDs, stale timestamps, oversized bodies, malformed signatures, and a rotated-key overlap case.
Watch counters for accepted events, 400s, 401s, replay rejections, and queue deduplication. Alert on a sudden 401 ratio by tenant; that catches a bad rotation or route change before a classroom integration silently stops syncing. Keep response timing and status behavior boring and documented.
Your mileage may vary on the exact timestamp window and body limit. Those values depend on sender retry policy, clock quality, and payload size. The decision should be recorded beside the runbook, with an owner and a review date.
Top comments (0)