DEV Community

Pritom Mazumdar
Pritom Mazumdar

Posted on

Carbon Layer v0.6 : Webhook resilience testing for payment handlers (idempotency, out-of-order, signature verification)

#ai #webdev #python #opensource

New release of Carbon Layer : the open source chaos engineering tool for payment flows.

v0.5 added multi-provider support (Razorpay, Stripe, Cashfree, Juspay). v0.6 focuses on a different problem: how resilient is your webhook handler?

The problem

Most webhook handlers are tested against the happy path : one event, correct signature, delivered in order. Production is different:

  • Payment gateways retry failed deliveries, so your handler gets the same webhook 2-5 times
  • Webhook delivery order is not guaranteed — payment.captured can arrive before payment.authorized
  • If your handler doesn't verify signatures, anyone can forge webhook events

These are the bugs that don't show up in staging.

What's new in v0.6

Idempotency testing : Fire each webhook N times and see if your handler processes it once or N times:

carbon run dispute-spike --provider mock \
  --webhook-url http://localhost:8000/webhooks \
  --webhook-repeat 5
Enter fullscreen mode Exit fullscreen mode

Out-of-order delivery : Randomize or reverse webhook delivery order:

carbon run dispute-spike --provider mock \
  --webhook-url http://localhost:8000/webhooks \
  --webhook-order random
Enter fullscreen mode Exit fullscreen mode

Signature verification : Send webhooks with missing, corrupted, or wrong-secret signatures:

carbon run dispute-spike --provider mock \
  --webhook-url http://localhost:8000/webhooks \
  --webhook-signature missing
Enter fullscreen mode Exit fullscreen mode

Webhook replay : Re-fire webhooks from any previous run. Useful for regression testing:

carbon replay <run_id> --webhook-url http://localhost:8000/webhooks
Enter fullscreen mode Exit fullscreen mode

CI/CD exit codes : Exit with code 1 if any webhook returned 5xx or timed out:

carbon run dispute-spike --provider mock \
  --webhook-url http://localhost:8000/webhooks \
  --ci
Enter fullscreen mode Exit fullscreen mode

4 new scenarios

  • upi-timeout : UPI payments stuck without terminal status
  • vpa-not-found : Invalid UPI VPA failures
  • mandate-rejection : UPI autopay mandate rejections
  • settlement-delay : Refunds on captured-but-unsettled payments

That brings us to 11 scenarios total.

Quick start

pip install carbon-layer
carbon run dispute-spike --provider mock --webhook-url http://localhost:8000/webhooks
Enter fullscreen mode Exit fullscreen mode

No database setup, no gateway credentials. 11 scenarios, 5 providers, webhook resilience testing. Apache 2.0.

GitHub: github.com/Pritom14/carbon-layer

We're building a hosted version with dashboards, scheduled runs, and compliance reports. Join the waitlist: pritom14.github.io/carbon-layer/waitlist

Top comments (0)