LLM pricing looks simple until a real application starts making repeated calls, sending long prompts, and carrying conversation history forward.
Before shipping a feature, I use a small estimation model:
1. Separate input and output tokens
Providers usually price input and generated tokens differently. The basic estimate is:
cost = (input tokens × input rate + output tokens × output rate) / 1,000,000
Then multiply that result by requests per user and expected active users.
2. Model repeated context explicitly
A chat request may resend system instructions and conversation history. A 1,000-token user message can therefore produce a request several times larger than expected.
For a rough forecast, I calculate three scenarios:
- Lean: short prompts, limited history, compact output
- Expected: normal production usage
- Heavy: long context, retries, tools, and verbose output
3. Add operational overhead
The token bill is not the whole cost. I also reserve room for:
- failed requests and retries
- evaluation and staging traffic
- embeddings or reranking
- moderation calls
- currency changes and provider price updates
4. Check whether self-hosting really helps
For local models, token prices are replaced by GPU memory and infrastructure constraints. VRAM requirements depend on parameter count, precision, context length, KV cache, and runtime overhead.
I built two free browser-based tools to make these estimates faster:
The calculations run in the browser and do not require an account. They are planning estimates, so current provider pricing and real workload measurements should remain the final source of truth.
What variables have surprised you most in production LLM costs?
Top comments (0)