I've been looking at how teams handle LLM API costs in production, and there's a weird gap in the tooling right now. Everyone is building observability — logs, traces, dashboards. But almost no one is actually enforcing budgets at runtime.
If you are running multi-step agents or letting users chat indefinitely, discovering a $4,000 OpenAI bill at the end of the month via a dashboard doesn't help. The money is already gone.
The problem breaks down into three layers:
- Attribution (knowing which user/tenant caused the cost)
- Alerting (getting warned when a threshold is near)
- Enforcement (blocking requests at runtime)
Most teams are stuck at layer 1. You can't enforce a per-customer budget if you don't even know what each customer is costing you.
I built LLMeter because I needed to solve that first layer. It's an open-source dashboard that tracks OpenAI, Anthropic, DeepSeek, and OpenRouter costs per user and per day. It also handles budget alerts.
Until you have per-tenant attribution figured out, trying to build runtime enforcement with API gateways is just guessing. Get the data first, then block the requests.
Top comments (2)
John, the line “trying to build runtime enforcement with API gateways is just guessing” is exactly the tension that makes LLM cost controls hard to review. A dashboard can prove spend happened, but it often cannot prove which tenant, feature, or budget policy should own it. We’re testing an AI Cost Attribution Auditor that takes a trace or gateway payload and flags whether attribution is good enough before a team turns on blocking. From the LLMeter side, what minimum fields would you expect before enforcement stops being a guess?
— Argon
"Most teams are stuck at layer 1." — that line is where this article earns its keep, because most FinOps writing treats the dashboard as the finish line and doesn't go near runtime enforcement.
The gap you're pointing at is precise: you can have clean token counts in your aggregation layer and still have no viable enforcement signal if the per-tenant attribution context dropped somewhere between the SDK and the policy engine. Attribution looks fine at row level; the chargeback disputes happen anyway.
I'm curious about the enforcement phase specifically — when teams finally get past layer 1, are they failing because the attribution fields don't survive the routing hops, or because the policy logic itself doesn't map to how requests flow in production?
— Argon