DEV Community

Cover image for You can't code-review a webhook retry
FetchSandbox
FetchSandbox

Posted on • Originally published at fetchsandbox.com

You can't code-review a webhook retry

#webdev #stripe #ai #testing

My coding agent wrote a Stripe webhook handler a few weeks ago. Signature check, event type check, fulfillment call, clean 200. The diff read like something I'd write on a good day. I approved it in about ninety seconds.

app.post("/webhooks/stripe", async (req, res) => {
  const event = verifySignature(req);
  if (event.type === "charge.succeeded") {
    await creditCustomer(event.data.object);
  }
  res.sendStatus(200);
});
Enter fullscreen mode Exit fullscreen mode

Every line is correct. And the first time Stripe delivers that event twice — which it's allowed to do, and eventually will — the handler credits the customer twice.

Reading can't catch it

The bug isn't on any line, so no amount of reading finds it. Each statement is fine on its own. What's wrong is an assumption living between the lines: that every event arrives exactly once. Stripe promises at-least-once delivery. Retries and duplicates are documented, normal behavior.

Code review checks the diff against the traffic you can imagine. I imagined one clean charge.succeeded. So did the agent — the quickstart code it learned from imagined one too.

CI passed for the same reason

My tests replayed the events I thought of. The fixture file had exactly one copy of each event, because I wrote the fixture file. Nobody writes the test where the same event lands twice nine minutes apart. The suite was my assumptions checking my assumptions.

I spent three years on the developer platform of a very large payments company. This exact bug got past good reviewers more times than I can count. It was never a talent problem. Reading is the wrong tool for this class of bug.

The review that actually works

What caught it was running the failure. Deliver the event, deliver it again, watch the customer get credited twice. Add the dedupe on the event ID, run it again, watch it hold. Ten minutes, and the review went from "looks right" to "proved right."

That loop — reproduce the failure, fix it, rerun, keep the receipt — is what I built FetchSandbox to run from your IDE over MCP. The sandbox fires the real charge lifecycle, including the duplicate delivery your fixture file doesn't have, so your agent proves the handler before your customers do.

If you'd rather judge it on real code than my word, there's a webhook-dedupe bug planted in a Stripe app in our open playground, waiting to be caught: github.com/fetchsandbox/playground.

Top comments (0)