I spent a whole night looking for money that was never missing

I once spent an entire night looking for money that was never missing.

It was tunyt's second event. By morning I had accounted for every rupee, and the problem I'd chased all night turned out to be hiding inside a fix I'd already made.

Decision 1: let the webhook drive everything

When someone pays, Razorpay sends our server a webhook, an HTTP request saying the payment succeeded. We made that the trigger for everything after a payment:

1. create the ticket,
2. send the QR code,
3. increase the number of tickets sold,
4. add the amount to the organiser's dashboard.

I'd defend most of that today. A browser can't be trusted to say it paid, because anyone can fake a success screen. The webhook comes from Razorpay's servers to ours, which makes it the one confirmation you can believe.

The weakness was that it was the only path. One event in, four effects out, and nothing downstream ever checked.

The night it broke

Three people bought tickets and never got them. The money left their accounts. On our side there was no ticket, no QR code and no email. I traced it back and found that Razorpay had disabled our webhook. Payments were arriving and our system never heard about them.

Decision 2: people first, then the books

I had two jobs, and they weren't equal. One was to understand what had happened. The other was that three people had paid for an event and had nothing to show at the door. That one had a deadline. The investigation didn't.

So I turned the webhook back on and emailed all three their QR tickets myself, by hand. They were getting in, whatever else was true.

I'd make that call again. It's also the call that set up the rest of my night.

Then the money didn't add up

While making sure I'd covered everyone, I went through the payments, and Razorpay showed more money than our dashboard did. It looked like three payments had arrived with no tickets behind them at all.

Three people with no tickets I understood. Three payments with no people was worse. Where had that money come from, and where was it on our side?

Decision 3: go through it line by line

I knew very little about reconciliation then. I didn't know how much data I could pull straight from Razorpay, or that I could store it and compare it with our own records using a short script. I wasn't good with Excel either.

So I did the only thing I knew how to do. I exported everything to CSV and went through it by hand. This is the payment ID. This is the order string. Does it exist on our side? Next line. It's slow in the way that makes you start doubting your own spreadsheet.

The twist

By morning I had the answer, and it was almost funny. There were no extra payments. They were the same three people.

When the webhook stopped firing, none of the four steps ran. I'd fixed step 2 by hand, because step 2 is the one people can see. Steps 3 and 4 had never happened. The ticket count didn't go up and the amount never reached the dashboard. The money was all there. Our records just hadn't caught up, and my manual fix had made the customers whole while leaving the books exactly as wrong as before.

What a webhook actually promises

A webhook is a notification, not a record. Providers try to deliver them and retry when they fail, but the guarantees are looser than they look. One can arrive late, or twice, or not at all, and it can be switched off on the provider's side without anything in your app noticing.

A design where one notification triggers four effects, and nothing ever double-checks, doesn't fail loudly. It drifts. Your records slowly stop matching reality, and nobody finds out until the numbers stop adding up.

Decision 4: trust the provider, check ourselves

So we stopped relying on the webhook alone. We pull payment data from Razorpay, store it in our own database, and compare the two with a script. If money arrived and our side doesn't fully reflect it, the script finds it, and neither an organiser squinting at a dashboard nor an attendee at the door has to. The check now runs automatically, with AI in the loop to flag anything that doesn't match.

If I were designing it today I'd go one step further and make every post-payment step safe to run twice, keyed by the payment ID. A missed webhook would then mean "run it again", and I wouldn't be fixing it by hand at night.

What I keep from that night is that a manual fix only repairs what you can see. I had completed one step of a four-step transaction, and the other three don't complain when they're skipped.

The money was there the whole time. What was missing was our record of it, and I'd only repaired the part that hurt.