We fired 150 dispute webhooks at a payment service. 12 handlers crashed. Here's what we built.

#stripe #python #testing #webdev

Every company processing payments tests the happy path.

Payment succeeds, order gets fulfilled, customer gets a confirmation email. That’s the flow that gets reviewed, tested in staging, and monitored in production.

What doesn’t get tested is everything else.

Dispute spikes. Refund storms. Gateway errors that leave orders stuck. Webhook sequences your handlers were never built to handle at volume.

These are the failure modes that show up in production usually at the worst possible time.

The problem is not that engineering teams don’t care.
It’s that the tools to test this don’t exist.

Why you can’t test this in Razorpay’s sandbox

Razorpay’s test API cannot create disputes.

Disputes are raised by banks and card networks, not merchants. There is no POST /disputes endpoint.

Even if you could trigger disputes manually, you can’t fire 150 of them in 10 seconds on a test account. Razorpay would rate limit you. And you can’t control the timing or sequence of webhook events in any sandbox.

So the failure mode you most need to test is the one the provider doesn’t let you simulate.

Teams ship, cross their fingers, and find out what breaks when customers find it first.

What we built

Carbon Layer is an open-source chaos engineering tool for payment flows.

You run a scenario dispute spike, refund storm, payment decline spike and it fires Razorpay format webhook events directly at your endpoint.

Same JSON shape.
Same headers.
Same HMAC-SHA256 signature as real Razorpay webhooks.

Your server can’t tell the difference.

pip install carbon-layer

carbon run dispute-spike \
  --provider mock \
  --webhook-url http://localhost:8000/webhooks/razorpay

No Razorpay account needed.
No sandbox credentials.
No rate limits.

The report

The report shows exactly what happened:

Webhook Delivery Summary
Target: http://localhost:8000/webhooks/razorpay

Event Type                 Sent    2xx    4xx    5xx    Timeout
payment.captured            100     98      0      1          1
payment.dispute.created     150    135      0     12          3
refund.processed             50     49      0      1          0

Total                       300    282      0     14          4

14 events your handler didn’t process correctly.

In production, each unhandled dispute is a chargeback the merchant loses by default.

Scenarios available

• dispute-spike — 150 disputes on captured payments
• payment-decline-spike — Simulates a 30% payment failure rate
• refund-storm — Mass refunds across captured payments
• flash-sale — High-volume order and payment flow
• gateway-error-burst — Intermittent gateway failures
• min-amount — Minimum paise transactions
• max-amount — Large-value transactions

All scenarios work with the mock adapter.
No external account required.

If you use Razorpay

You can also run scenarios against your actual Razorpay test account:

carbon run dispute-spike \
  --provider razorpay \
  --api-key rzp_test_xxx \
  --api-secret yyy \
  --webhook-url https://your-staging-app.com/webhooks/razorpay