Your AI feature is in production. Users are asking questions, and answers arrive on time. A month later, the product gets a new document: its operating bill.
Token charges are easy to spot. The costs created by long context, knowledge retrieval, storage, engineering tools, and quality checks are easier to miss.
I’m Anton Fokin, CEO Qtim. We build AI chatbots and RAG systems, and our teams use Claude and ChatGPT with Codex. After launch, we separate customer-facing AI costs from the tools used by the delivery team. For the product itself, I track the cost of one useful outcome: an answer backed by the right source, a completed action in another system, or a handoff to a person.
A single model call rarely completes the business task.
The bill starts before the model writes its first word
Model providers charge for tokens, the small units into which text is split. Input and output have separate prices. Cached input has another rate, and Claude also charges for writing a prompt prefix to the cache.
The user’s question is only one part of the input. Before answering, the model may receive a system instruction, tool descriptions, examples, conversation history, and passages from a knowledge base. Output tokens cover what the model generates. The exact numbers should come from the usage field in the API response and be stored for every request.
A practical first formula is:
Cost per response = regular input + cache reads + cache writes + output + tool calls.
Product usage and team tools need separate cost buckets. The table below shows what belongs in each one.
Press enter or click to view image in full size
11,000 input tokens: context multiplies every request
Context is everything the model receives before it answers. In a retrieval-augmented generation (RAG) system, the user’s question becomes a search query. Search returns several passages, and the application adds them to the instruction sent to Claude. The conversation history grows with every message as well.
In the example below, each model call carries 11,000 input tokens. A stable prefix takes 8,000 tokens. The user’s question, chat history, and retrieved documents add another 3,000. The product assembles these pieces separately, but the provider bills them as one input stream.
A large context window shows the model’s technical limit. The invoice depends on the amount of context actually sent. If the application attaches twenty pages of policy text to every request, the provider charges for those pages again as regular or cached input. Extra passages can also make the answer worse by burying the useful paragraph among similar instructions and obsolete versions.
Three controls keep this in check. First, limit the number and size of passages returned by retrieval. Second, summarize older messages into a compact conversation state. Third, cache the stable prefix that contains the system instruction and tool definitions.
With Claude, cache pricing depends on the operation and the entry’s lifetime. A short-lived cache works well when requests arrive close together. With sparse traffic, the prefix will be written again more often. Logs therefore need the actual cache-read rate, while the financial model needs separate rates for cache reads and writes. Anthropic’s prompt caching documentation lists the current multipliers and retention options.
F*ive gigabytes of knowledge: storage, retrieval, and versions are separate costs
Press enter or click to view image in full size
*
A knowledge base has several layers: source files, cleaned text, retrieval chunks, vector representations, an index, request logs, and document versions. Backups and data retained for investigating disputed answers add more storage.
For a 5 GB knowledge base, the budget needs to include vector storage, retrieval operations, backups, and logs. Retrieved passages then enter Claude’s context and are billed again as input tokens.
A self-hosted vector database changes the line-item names but keeps the same cost logic: storage, indexing compute, search, redundancy, and observability. Frequent document changes add re-indexing work. If old versions remain active, retrieval can return a policy that is no longer valid.
Every source needs an owner, a version, and a retention period. After an update, the team should confirm that the new document entered the index, the old one stopped appearing in results, and access rights stayed intact. Otherwise, the company pays to store duplicates and receives answers based on retired rules.
10,000 conversations: an illustrative $811.20 Claude bill
Consider a hypothetical month. This is an illustrative formula based on public rates rather than Qtim project data. The Claude Sonnet 4.6 rates used below were verified on Aug 5, 2026:
- 10,000 conversations, with four model calls each: 40,000 calls;
- 8,000 tokens of stable instructions, 3,000 variable input tokens, and 500 output tokens per call;
- 95% of the stable prefix read from cache and 5% written again.
The model bill is $811.20: $360 for variable input, $91.20 for cache reads, $60 for cache writes, and $300 for output.
Without caching, the same token volume would cost $1,620. The difference is $808.80. The table covers model tokens only; retrieval and storage depend on the infrastructure selected for the product.
The calculation also excludes application servers, databases, queues, monitoring, backups, paid integrations, and specialist time. It does not include taxes, currency conversion, or data-residency premiums. Development costs for the product itself are covered in our separate guide to MVP development costs in 2026. Post-launch operations should be estimated from actual traffic.
Track Codex by completed engineering task
I would not allocate Codex consumption to the cost of a customer conversation. Codex reads code, task history, logs, and command output, then produces changes and explanations. After launch, this cost grows during bug fixes, test updates, incident analysis, and code review.
Codex now uses token-based credit rates for most ChatGPT plans. Input, cached input, and output consume credits at different rates, as shown in the current Codex rate card. For management purposes, I would still calculate credits per completed engineering task, then track context volume, reruns, and automations. That derived metric shows whether a large context and several attempts produced a useful result.
Claude and ChatGPT subscriptions belong in the team’s tooling budget. Keeping them separate from the product API makes changes easier to explain. A higher total can come from more customer traffic, longer Claude responses, or engineering tasks that needed more attempts. One combined number hides the cause.
A wrong answer has its own budget
Forty thousand calls for $811.20 can look inexpensive. The number is only useful while the system cites the current document, respects access rights, and hands difficult cases to a person at the right moment. A bad recommendation can create a support ticket, an invalid order, or legal exposure.
Quality control starts with a test set. It should include routine questions, rare cases, typos, conflicting documents, and attempts to push the system beyond its allowed role. Each example needs an expected source, required answer elements, and a handoff condition.
Run the set again after changes to the prompt, model, retrieval logic, or knowledge base. Add real failures from production logs, and calibrate automated scoring against expert decisions.
This budget line has its own formula:
Quality control = test runs + expert review of a sample + error analysis + fixes.
There is no universal dollar rate for it. One product checks whether an answer cites the correct policy. Another verifies product compatibility. A third confirms whether the user has permission to trigger an action in a CRM. Before launch, define the review frequency, sample size, and person responsible for disputed cases.
Five guardrails keep post-launch costs under control
I would set five limits before users arrive.
- Separate budgets for the customer-facing API and the team’s AI tools. This keeps product traffic and engineering work from obscuring each other.
- Cost per completed customer outcome and its 95th percentile. An average hides long, expensive conversations.
- Context limits. Set caps for history length, retrieved passages, output length, and the share of tokens served from cache.
- Routing rules. Send simple requests to a smaller model, complex ones to a stronger model, and high-risk actions to a person.
- A quality threshold. Define acceptable rates for answers without sources, incorrect actions, and human handoffs, then alert on cost spikes and quality degradation.
The useful number is the cost of a completed outcome together with the cost of checking it. If you already have an AI workflow and an expected traffic profile, talk to the our team. We can map the full response path across tokens, retrieval, storage, and quality control before the first monthly bill becomes a surprise.
Top comments (0)