Why your LLM bill spiked: 7 causes (and a way to fix them)

Eli Katz — Tue, 17 Feb 2026 19:54:04 +0000

If you’re shipping LLM features, the invoice can jump before anyone knows why. Most cost blow-ups are predictable—and observable.

Context bloat (prompts slowly grow) → Track input tokens p50/p95 → Add prompt budgets + summarize history
Retry storms (1 action = N calls) → Track calls per workflow/session → Cap retries + backoff + fail fast
Wrong model drift (expensive model becomes default) → Track model mix over time → Route: cheap by default, escalate on low confidence
Agent/tool loops (runaway tool calls) → Track tool-call depth + trace length → Cap depth, limit tool output, add stop conditions
Verbose outputs (paying for essays) → Track output token distribution → Set max response length + structured formats
RAG overshoot (too many/too big chunks) → Track retrieved tokens/query → Reduce top-k, tighter chunks, retrieval budgets
Abort + re-ask loops (stream cancel then repeat) → Track aborted generations + rapid repeats → Improve first response, add “continue?”, cache safely

We've built ZenLLM (zenllm.io): read-only LLM cost observability + optimization recommendations—so it can’t break prod or become a single point of failure.

Launching with a limited number of free LLM Savings Assessments (attribution + top waste + prioritized roadmap). If you want one, comment with your stack + your biggest cost mystery.

DEV Community: Eli Katz

Why your LLM bill spiked: 7 causes (and a way to fix them)