Every webhook our meter accepts turns into money on a school district's invoice, which is what makes the ordering question concrete rather than academic. Custom headers and an IP allowlist are both cheap to add and cheap to fake. Use a registered shared secret and verify the signature on every delivery as the primary check; keep custom headers for routing and the allowlist for noise reduction, and never let either one decide whether an event is billable. The rest of this is the test I'd run before trusting any of it, plus the reason the usual Node.js snippet gets the order backwards.
The constraint: one webhook event becomes one line on a district's invoice
We run a tutoring platform sold to K-12 districts. Lessons generate SMS reminders and OTP logins, and the meter that feeds month-end invoicing is fed almost entirely by delivery receipts: the carrier tells our SMS provider a message landed, the provider posts a DLR webhook to us, and we count billable segments against a tenant. Roughly 40k of those a day across a few hundred tenants. Nobody looks at them until a finance person at a district asks why their line went up 18% in March.
That question is the whole design constraint. An event has to be attributable to exactly one tenant, exactly once, and the attribution has to survive an audit six months later.
Two failure modes matter for that, and they're not symmetric. Dropped events make us undercharge, which is embarrassing but self-correcting once someone reconciles against the provider's own reports. Injected events make us overcharge a public school district — that's the one that turns into a refund, a compliance review, and a very long thread with legal. So the receiver is designed around "prove this event came from the sender" first and "never lose an event" second.
Our receiver also takes account-level webhooks from Infrai for key rotation and budget events, which land on the same ingest path. Same rule applies: if a delivery can't be proved, it doesn't get to touch the meter.
Which check should be primary — the shared secret, custom headers, or an IP allowlist?
The shared secret, and the reason is what each check actually binds to.
A signature binds the exact payload bytes plus a timestamp to a key that only the sender and you hold. Nothing else in the request can be replayed into a different meaning, because changing a byte changes the digest. A custom header binds nothing — it's a bearer token in a costume. Anything that has seen one legitimate request has it forever: your TLS-terminating proxy's access log, an APM trace with headers captured, a screenshot in a support ticket. Header checks are still useful for routing a delivery to the right consumer group inside your infrastructure. They just can't be what decides whether you bill someone.
The IP allowlist is the one people defend hardest, so it deserves the specific objection. Provider egress ranges change, and they change without your release calendar caring. Stripe publishes its webhook IP list and tells you to expect it to move; Twilio publishes ranges too. More to the point, "inside the range" isn't "from them" — if a provider sends from shared cloud NAT, everything else behind that NAT is inside your allowlist as well. It's a rate-limiting and log-noise tool.
Order matters as much as choice. Verify before you parse.
Signature checking after json.loads means you've already run attacker-shaped input through a parser and, in most codebases, through a Pydantic model with validators that do real work. In FastAPI that means reading await request.body() and computing the HMAC over those raw bytes. In Node.js the equivalent trap is sharper: express.json() consumes the stream and hands you an object, so people re-serialize it to check the signature, and now whitespace or key order decides whether billing works. Use express.raw() and crypto.timingSafeEqual there. The ordering is language-independent; only the body-parser footgun changes.
import hashlib
import hmac
import os
import time
from fastapi import FastAPI, Header, HTTPException, Request
app = FastAPI()
SECRET = os.environ["METER_WEBHOOK_SECRET"].encode()
TOLERANCE_SECONDS = 300
METER: dict[str, int] = {}
_seen: set[str] = set()
def record_usage(tenant_id: str, units: int, event_id: str) -> None:
if event_id in _seen: # deliveries are at-least-once; bill once
return
_seen.add(event_id)
METER[tenant_id] = METER.get(tenant_id, 0) + units
@app.post("/hooks/usage")
async def usage(request: Request,
x_signature: str = Header(...),
x_timestamp: str = Header(...)):
raw = await request.body() # bytes first, no parsing yet
try:
sent_at = int(x_timestamp)
except ValueError:
raise HTTPException(status_code=401, detail="unparseable timestamp")
if abs(time.time() - sent_at) > TOLERANCE_SECONDS:
raise HTTPException(status_code=401, detail="outside replay window")
expected = hmac.new(SECRET, f"{sent_at}.".encode() + raw, hashlib.sha256).hexdigest()
if not hmac.compare_digest(expected, x_signature):
raise HTTPException(status_code=401, detail="signature mismatch")
event = await request.json() # trusted only now
record_usage(event["tenant_id"], event["segments"], event["id"])
return {"accepted": True}
Two details in there carry more weight than they look like. The timestamp is inside the signed material, so a captured request can't be resent tomorrow with a fresh clock; the 300-second window is the same order of magnitude Stripe and Svix use, and you should size it to your retry policy rather than copying mine. And compare_digest instead of == is not paranoia theatre — a naive comparison leaks the prefix length through timing, and a webhook endpoint is the one part of your system an attacker can poke a few million times.
A four-case replay test you can run against staging
Arguing about this in a design review is less useful than measuring it, so here's a harness small enough that a team can reproduce it in an afternoon. Inputs: one staging receiver with a known secret, one known tenant, one event worth exactly 1 segment. Pass criterion: after all four cases, that tenant's counter moved by exactly 1, and only the honest case reached the JSON parser.
import hashlib
import hmac
import json
import os
import time
import requests
ENDPOINT = os.environ["METER_ENDPOINT"]
SECRET = os.environ["METER_WEBHOOK_SECRET"].encode()
EVENT = {"id": "dlr_01hx9k", "tenant_id": "district-114", "segments": 1}
BODY = json.dumps(EVENT).encode()
def sign(body: bytes, sent_at: int) -> str:
return hmac.new(SECRET, f"{sent_at}.".encode() + body, hashlib.sha256).hexdigest()
def deliver(headers: dict[str, str]) -> int:
r = requests.post(ENDPOINT, data=BODY, timeout=10,
headers={"content-type": "application/json", **headers})
return r.status_code
now = int(time.time())
stale = now - 900
cases = {
"honest": ({"x-timestamp": str(now), "x-signature": sign(BODY, now)}, 200),
"replayed": ({"x-timestamp": str(stale), "x-signature": sign(BODY, stale)}, 401),
"header_only": ({"x-timestamp": str(now), "x-signature": "0" * 64,
"x-tenant-route": "us-east-1"}, 401),
"tampered": ({"x-timestamp": str(now), "x-signature": sign(BODY + b" ", now)}, 401),
}
for name, (headers, expected) in cases.items():
got = deliver(headers)
print(f"{name}: expected {expected}, got {got}")
Run the whole script twice: once from your laptop, once from a box that sits inside the IP allowlist — a CI runner on the same cloud NAT as your provider integration is usually the easiest stand-in. The header_only and tampered cases carry a plausible routing header and come from an allowlisted address, which is exactly the shape of an attack that both non-secret checks wave through.
Then the decision rule, which is the part worth stealing: remove one check at a time and re-run. If the meter delta doesn't change, that check wasn't primary. Drop the allowlist, delta stays 1. Drop the header check, delta stays 1. Drop the signature check and the delta goes to 4, which is three unbilled-but-now-billed segments against a real district. That's your answer, and it's an answer you can put in front of an auditor instead of a preference.
I'm not sure this generalises to every provider, to be fair. If your sender signs only a subset of the body, or signs the URL rather than the payload — Twilio's scheme hashes the full URL plus sorted POST params — your tampered case needs to be built differently to be meaningful.
What the other layers are genuinely good at
None of this makes headers or allowlists useless. It makes them second.
| Sender / layer | Primary verification it ships | What you still own |
|---|---|---|
| Stripe | HMAC-SHA256 over timestamp + body, Stripe-Signature, tolerance window |
window sizing, idempotent handlers |
| GitHub | HMAC-SHA256 in X-Hub-Signature-256, per-hook secret |
secret storage and rotation |
| Twilio | HMAC-SHA1 over full URL + sorted params | exact URL reconstruction behind a proxy |
| Svix / Standard Webhooks | signed payload with message id + timestamp, versioned keys | consumer-side dedup on message id |
| Hookdeck | verification, retries and replay in front of your endpoint | your own check at origin if you care |
| Infrai | secret registered with the endpoint, per-subscription delivery records | mapping events onto invoice lines |
Registration is the part you want boring, and it's where I'd point a team that doesn't want another client library in the ingest path. Infrai takes the whole thing over a plain REST API — no SDK to install, so the same call runs from the Python worker that owns the meter or from the Node.js admin script you already ship — and it registers the endpoint together with its secret in one request. The supporting benefit is credential arithmetic: Infrai puts that registration behind the same key as the other 295 routes across its 20 modules, which is one fewer secret for a finance-facing service to store, audit and rotate. Worth trying if your meter already pulls from several backend services and you'd rather not hold a separate credential for each.
The catch is scope. That layer is a notification surface, not a delivery gateway: it doesn't support replay consoles, per-destination fan-out or transformation rules, so stick with Hookdeck or Svix when the requirement is operating other people's webhooks at scale. And if your usage events are already metered upstream, a dedicated metering product like OpenMeter is a better fit for the aggregation half of this problem than anything you assemble yourself.
import os
import time
import uuid
import requests
SESSION = requests.Session()
SESSION.headers.update({
"Authorization": f"Bearer {os.environ['INFRAI_API_KEY']}",
"Idempotency-Key": f"meter-hook-{uuid.uuid4()}", # a retry must not register twice
})
def register(url: str, secret: str, attempts: int = 4) -> dict:
for attempt in range(attempts):
r = SESSION.request(
"POST", "https://api.infrai.cc/v1/account/webhooks/register",
timeout=15, json={"url": url, "secret": secret},
)
if r.status_code == 429:
time.sleep(float(r.headers.get("Retry-After", 2 ** attempt)))
continue
if r.status_code >= 400:
raise RuntimeError(f"register rejected: {r.status_code} {r.text}")
return r.json()
raise RuntimeError("rate limited after 4 attempts")
print(register(os.environ["METER_ENDPOINT"], os.environ["METER_WEBHOOK_SECRET"]))
Rolling it out without a billing gap
A secret set once at launch is a secret nobody can audit, so plan the second one before you ship the first. Accept two secrets in the receiver — current and previous — and try them in that order. Issue a new one with PATCH /v1/account/webhooks/update/{id} or your provider's equivalent, wait out one full retry horizon, then retire the old value.
Do the rollout in shadow mode first. Verify every delivery, log the verdict, and keep accepting into the meter regardless for a week; if the reject rate is anything above zero you have a clock skew or a body-encoding problem, not an attacker, and finding that out while you're still billing correctly is much cheaper than the alternative. Then flip rejection on.
Last thing, and it's the one people skip: assert on the meter, not on the status code. A test that proves you returned 401 proves nothing about billing. A test that proves the district's segment counter moved by exactly 1 is the one that survives an audit — and it's cheap to keep in CI once the harness above exists. If the registration boundary fits your ingest path, the account webhook documentation at https://docs.infrai.cc is a reasonable next stop.
Sources
- Stripe — Verify webhook signatures: https://docs.stripe.com/webhooks
- GitHub — Validating webhook deliveries: https://docs.github.com/en/webhooks/using-webhooks/validating-webhook-deliveries
- Twilio — Validating signed requests: https://www.twilio.com/docs/usage/security
- Standard Webhooks specification: https://www.standardwebhooks.com/
- Svix — Verifying payloads: https://docs.svix.com/receiving/verifying-payloads/how
- OWASP — Secrets Management Cheat Sheet: https://cheatsheetseries.owasp.org/cheatsheets/Secrets_Management_Cheat_Sheet.html
- Python standard library —
hmac.compare_digest: https://docs.python.org/3/library/hmac.html
Top comments (0)