DEV Community

ULNIT
ULNIT

Posted on

I Launched on Product Hunt. Four Hours Later, Someone Was Replaying My Checkout Webhook.

The launch itself went fine. Traffic spike, a few hundred signups, dopamine. Then at around hour four, my monitoring pinged me with something that made my stomach drop: the same webhook event, delivered to my server 61 times, from 9 different IPs.

It was a checkout.completed event from my payment processor. Signed. Valid. And completely fake.

Whoever sent it had captured one real webhook from somewhere — maybe their own purchase, maybe a leaked example — and was replaying it at my endpoint, hoping my server would just... believe it. And the worst part? For the first two days of my launch, it would have. I'd shipped the signature verification and nothing else. No timestamp check, no idempotency, no rate limit on the webhook route.

The attacker's version of the replay was slightly malformed (they'd re-signed it with a guessed secret, so HMAC verification actually rejected it — my one piece of luck). But it forced me to spend that evening auditing every unauthenticated door into my system. Here's what I found, what I fixed, and the exact patterns I now refuse to launch without.

Why webhooks are the softest target on launch day

Launch day is the one day your attack surface gets crowdsourced attention. Thousands of people hit your site, and a few of them are curious in ways you don't want. Authenticated routes are mostly fine — they're behind your login. But webhooks are, by design, endpoints anyone on the internet can POST to. Your payment processor needs to reach them without logging in.

That makes them the classic "door that's unlocked because the delivery guy needs to get in." And most solo founders (me, pre-launch) treat them like internal plumbing instead of what they are: a public API that can credit accounts, mark orders paid, or trigger fulfillment.

If an attacker can get your server to accept a fake payment.completed, they get your product for free. If they can replay a real one, they might get it multiple times — duplicate credits, duplicate provisioning, duplicate emails to a customer who now thinks they bought three subscriptions.

The four checks every webhook endpoint needs

1. Verify the signature (table stakes — and do it in constant time)

Every serious payment processor signs webhooks with an HMAC of the raw body. You must verify against the raw request body, not a parsed-and-re-serialized JSON object — key ordering and whitespace differences will make valid signatures fail and, worse, tempt you to "fix" it by skipping verification.

import hmac, hashlib

def verify_signature(raw_body: bytes, signature: str, secret: bytes) -> bool:
    expected = hmac.new(secret, raw_body, hashlib.sha256).hexdigest()
    # constant-time compare — never use == on signatures
    return hmac.compare_digest(expected, signature)
Enter fullscreen mode Exit fullscreen mode

Two mistakes I've seen (and made) here: comparing signatures with == (timing attack, low risk but free to fix), and verifying a JSON round-trip instead of the raw bytes (breaks randomly, gets "fixed" by deleting the check).

2. Reject stale timestamps (this kills replay)

A valid signature proves the sender knows your secret — it says nothing about when the event was created. Replay is only possible because old events stay valid forever. Almost every processor includes a timestamp in the signed payload or header. Check it:

import time

def reject_stale(timestamp: int, tolerance_seconds: int = 300) -> bool:
    return abs(time.time() - timestamp) <= tolerance_seconds
Enter fullscreen mode Exit fullscreen mode

Five minutes of tolerance is plenty. Now a captured webhook is only useful for 300 seconds, which turns "replay forever" into "race the clock" — and attackers don't like races they usually lose.

3. Store event IDs and make processing idempotent

Even with 1 and 2, legitimate processors will deliver the same event more than once — retries after timeouts are normal. Your handler must be idempotent:

def handle_webhook(event: dict) -> None:
    event_id = event["id"]
    if redis.set(f"webhook:{event_id}", "1", nx=True, ex=86400 * 7):
        process_event(event)  # first time only
    # else: already processed, return 200 anyway
Enter fullscreen mode Exit fullscreen mode

The SET ... NX pattern is atomic — no race between check and insert, even with multiple workers. Note the subtlety: you return 200 for duplicates, not an error. Error responses trigger more retries, and a retry storm against a handler that errors is how you DDoS yourself on launch day.

This one also saved me from a bug I'd already shipped: my fulfillment code credited the user's account inside the webhook handler. Before idempotency keys, a single retried delivery would have double-credited. I found it in my logs before an attacker did — 48 hours after launch, a processor timeout caused a retry, and one early customer got two months of access for the price of one. I caught it because the numbers didn't reconcile, refunded myself the embarrassment, and shipped the Redis guard the same night.

4. Rate-limit the route like it's public — because it is

Per-IP rate limiting on the webhook endpoint costs ten lines and turns "61 requests from 9 IPs in an hour" into "9 IPs politely told to go away." Something like:

from flask_limiter import Limiter
limiter = Limiter(app=app, default_limits=[])

@app.route("/webhooks/stripe", methods=["POST"])
@limiter.limit("30/minute", methods=["POST"])
def webhook():
    ...
Enter fullscreen mode Exit fullscreen mode

Legitimate processors deliver in bursts, not floods — 30/min per IP is generous. Scanners and replayers hit that ceiling instantly and show up in your rate-limit logs, which doubles as a free early-warning system.

The honest failure section: what I got wrong first

I want to be straight about the sequence, because the "four checks" list above makes me sound more competent than I was.

Version 1 (pre-launch): no signature check at all. I told myself I'd "add it before real traffic." Classic.

Version 2 (launch morning): signature check added at 6 AM, raw-body bug included — it rejected legitimate webhooks about 30% of the time because I was verifying re-serialized JSON. My first three paying customers' events bounced. The processor retried, and luck (retry + eventual success) covered for a bug that was silently losing revenue events.

Version 3 (that evening, post-replay-attack): raw body fixed, timestamps checked, idempotency keys in. The double-credit incident above happened after this — because I'd assumed my processor "probably never retries." It does. Assumptions about third-party behavior are bugs with a delivery date.

The pattern in all three: I treated the webhook as plumbing until the internet treated it as a target. The fixes weren't hard — none of this took more than a few hours total. The cost was entirely in doing them reactively, with an attacker's timing instead of mine.

The 15-minute pre-launch webhook drill

If you're launching something with payments soon, here's the compressed version of what I learned the slow way:

  1. curl -X POST your own webhook endpoint with a garbage body. If it returns anything other than 4xx, stop and fix that first.
  2. Grab a real signed payload from your processor's dashboard/test mode. Replay it twice from your terminal. The second one must be a no-op (same 200, no duplicate side effects).
  3. Replay it with the timestamp edited. Must be rejected.
  4. Hammer it with 60 requests in a minute from one IP. Rate limiter must fire.
  5. Check your logs: can you distinguish "duplicate rejected" from "invalid signature" from "rate limited"? If they all look the same, your future incident response will be guesswork.

Five tests, fifteen minutes, and it covers the exact attack I got hit with — plus the two bugs I hit myself.

The full checklist + scripts are in Ship Safe — The Launch-Day Security Kit — code LAUNCH90 at checkout makes it $1.50.

Launch day should be stressful because of traffic, not because of trespassers.

Top comments (0)