Most webhook discussions focus on sending events.
But if you’re on the receiving side like billing, orders, CI, internal automation, the real pain starts when something goes wrong and you don’t know it did.
A few failure modes I’ve seen repeatedly in production:
- Your endpoint times out during a deploy
- A brief network blip drops an event
- Rate limits kick in upstream
- Retries happen… but you can’t see them
- An event fails once and disappears forever
The worst part isn’t the failure, but it’s finding out from a customer there's a problem.
What ended up working for us was treating inbound webhooks like any other critical async system:
- Log every delivery attempt (request + response)
- Retry with backoff + jitter
- Send unrecoverable events to a DLQ with a reason
- Allow safe, idempotent replays
- Make all of it visible in one place
We eventually built a small service around this so our app never talks directly to third-party webhook senders anymore and they go through a reliability layer first.
This is just a quick overview here. But if this is a problem you’ve run into then try it out for free:
Genuinely curious how others are handling webhook reliability on the receiving side. Especially at low to medium scale where “just build it yourself” is often suggested.
Top comments (0)