Over the past few weeks, I’ve helped 3 developers get refunds from OpenAI and Anthropic after their API keys leaked.
Every one of them found out the same way: from the bill. Sometimes a day or two after the damage was done.
Here’s what I learned, and what I’m building because of it.
What I learned
1. Billing alerts are too slow and too broad.
Provider billing data usually lags by hours. And even when an alert fires, it tells you how much you spent, not what caused it. Was it a leaked key? A retry loop? One user hammering your app? You have to dig through logs to find out.
2. The first few minutes matter most.
The people who got the most money back were the ones who noticed fast, revoked the key, and collected evidence before contacting support. Panic usually makes things worse.
3. Not every spike is a leak.
Sometimes it’s real growth, sometimes a bug, sometimes one user abusing a free tier. Each one needs a different response, so treating every spike as an emergency just trains you to ignore alerts.
What I’m building: Brello
Brello tracks your AI API spend per user and per feature, in real time, and alerts you in minutes when something looks off.
See which users and features actually drive your costs
Alerts to your webhook (Slack, Discord, etc.) in minutes, not the next day
Tells you whether it looks like a leak or just a spike, and what to do next
Suggests where a cheaper model could do the job
Warns you when a model you use gets deprecated or repriced
Works with OpenAI, Anthropic, and Gemini.
Why an SDK and not a proxy
A lot of tools in this space sit between your app and the provider as a proxy. I didn’t want that: it means your prompts and keys pass through someone else’s server, and if the proxy goes down, your app goes down with it.
So Brello is a drop-in SDK wrapper that runs inside your app:
Requests go straight to the provider
Your key never leaves your app
It collects usage metadata only, never prompts or responses
Data is sent in the background, so if Brello is down, your app keeps running
The SDK will be open source, so you can check exactly what it sends.
I’m posting progress updates on the Brello blog too 😃
Closed beta
I’m opening a closed beta around Nov 20 for 10 people running AI features in production.
Free during the beta
3 months of Pro free after launch
I’ll personally help you set it up
Apply here: https://brello.app/beta
Even if you’re not interested in the beta, I’d love to hear: how do you keep an eye on your AI API spend today? Billing alerts, your own logging, or not at all?
Top comments (0)