The Developer's Guide to AI Tokenomics: Why Your Coding Agent Can Burn Through Credits in Minutes

code&cosmos — Sat, 10 Oct 2026 16:59:34 +0000

You never came close to your context limit, yet your team's AI credit balance dropped sharply. Here's the real math behind agent loops, hidden context, reasoning tokens and prompt caching.

A developer or QA automation engineer sits down with a new task. The organization has rolled out AI coding assistants (Cursor, GitHub Copilot, Claude Code, or an internal agent harness) and given everyone a monthly credit quota.

You ask the assistant to fix a few flaky tests, generate some mock data and debug an integration timeout. Then you grab a coffee.

When you get back, the usage dashboard shows a big chunk of the month's quota already gone.

Your first reaction is disbelief: "The UI says my context is only at 45,000 tokens out of 1 million. How did a few minutes of work cost this much?"

AI pricing isn't broken. Tools have moved from single-turn completions to autonomous agent loops, and most engineers were never shown how token consumption actually works under the hood.

Note on numbers: Prices, credit conversions and token counts below are illustrative. They vary widely by vendor, plan, model and tool. Check your own provider's pricing page and usage dashboard for real figures.

1. The core misconception: context window vs. cumulative usage

Keep two different measurements separate:

Context window capacity is the size of the desk: how much text the model can hold at one moment.
Cumulative token consumption is the water meter: everything sent to and generated by the model across every call.

A "1M context window" means the model can hold roughly that many tokens in a single request without failing. You aren't billed for the size of the desk. You're billed every time paper is slid onto it.

If you load 50,000 tokens of code into a conversation and go back and forth 10 times, the model re-reads that 50,000-token foundation on each turn (prompt caching softens this, covered in section 5).

The UI may show Current context: 58,000 / 1,000,000, while the billing meter has already recorded more than half a million input tokens.

2. The agent multiplier: one prompt is not one API call

Early chat assistants were simple: you sent a prompt and got an answer.

Today's coding assistants are agents. When you type something like "Fix the failing payment test," the model doesn't answer in one shot. It starts a loop: list files, read some of them, read the implementation, run the test, read the error, edit code, check the diff, run the test again.

Each step is a separate model call, and each one appends its results to the growing history.

The accumulation math (illustrative)

Step	What gets added	Input tokens sent that call
1	Base instructions and your prompt	15,000
2	File list	17,000
3	File contents	23,500
4	Client implementation	31,000
5	Error log output	35,500
6	Git diff	42,000
Total		164,000

The final context is 42,000 tokens, but 164,000 input tokens were sent across six calls.

Because each call resends everything before it, total input grows roughly with the square of the number of steps. It isn't exponential, but it climbs fast. If the agent gets stuck in trial and error (edit, fail, read stdout, retry 8 to 12 times), a single innocent prompt can run into several hundred thousand tokens before you see a green check.

3. Hidden context: what's sent besides your prompt

A 12-word prompt in your IDE is never just 12 tokens. Several things ride along with it.

A. System instructions and tool definitions

To act as an agent, the model needs instructions on how to behave and descriptions of every tool it can call: terminal, file search, edit, browser and so on. Depending on the tool, that can run to thousands or even tens of thousands of tokens at the top of every request.

B. Editor context

Many IDE assistants pull in extra context to guess what you mean: the current file, recently viewed files, selections, and sometimes snippets from open tabs or search results. How much is included depends heavily on the tool and its settings. It's rarely every open tab in full, but unrelated files can still add meaningful noise.

C. Giant terminal dumps

When a test suite crashes, it's tempting to paste the whole terminal output. A full stack trace with warnings, deprecation notices and raw JSON can easily add thousands of tokens in one paste. In an agent loop it gets resent on every later step.

4. Pricing asymmetry: input, output and reasoning

Tokens aren't all priced the same. The general pattern across major providers:

Token type	Relative cost (rough pattern)
Cached input	Cheapest, often a fraction of normal input
Fresh input	Baseline
Output (generated text)	Usually several times the input price
Reasoning ("thinking") tokens	Usually billed as output

Exact ratios differ by provider and model, so check the current pricing page rather than relying on any single table.

The hidden reasoning cost

Reasoning models, and models with extended thinking turned on, generate internal deliberation before they answer. You may never see those tokens in the chat, but they're typically billed at the output rate. A hard architectural question can generate thousands of reasoning tokens on top of the visible answer.

5. Prompt caching: the savior and the trap

Resending huge histories would be ruinous, so major providers offer prompt caching.

How it works

When a model processes text, it builds internal key-value (KV) representations of the tokens. With caching, the provider stores those for a repeated prefix. If your next request starts with exactly the same tokens, that part can be read from the cache at a large discount instead of being recomputed.

The catch: prefix matching

Caching works from the start of the prompt forward, and caches expire after a period of inactivity, often minutes.

If you change something near the top (edit the system prompt, insert a file above the conversation, flip a setting that changes early context), everything after that point misses the cache and is billed at the full input rate again.

The practical lesson is to keep stable content at the top, append new content at the bottom, and avoid needlessly reshuffling early context.

6. Do `SKILL.md` or rules files cost more or less?

Many teams package reusable instructions as SKILL.md, .cursorrules or similar rules files.

Turn cost: if a tool always attaches a rules file, it adds its size to every request. If the tool loads a skill only when it's relevant, you pay for it only in those turns. Check how your tool behaves.
System cost: modular instructions beat one giant always-on prompt covering every framework, lint rule and convention your company has.
Iteration factor: the most expensive tokens come from unnecessary retries. If a short skill file gives the agent the exact test setup it needs, it may succeed on the first pass instead of looping through five failed runs.

A thousand tokens of good instructions up front can save many times that in avoided retries.

7. Where tokens go: the heaviest activities in the dev and test lifecycle

Some tasks are naturally cheap. Others turn into long agent loops, huge inputs or heavy reasoning. Here are the usual heavy hitters, roughly grouped by lifecycle stage, with why each one burns tokens and how to rein it in.

Planning and design

Activity	Why it's expensive	How to contain it
"Understand this whole codebase" or architecture walkthroughs	The agent crawls directory trees and reads many files to build a mental model	Point it at a few key modules or an existing architecture doc first
Large refactor or migration planning (framework upgrades, monolith to services)	Reads many files and often triggers long reasoning	Plan one module or layer at a time
Requirements-to-design with a reasoning model	Lots of hidden thinking tokens billed as output	Use reasoning models only for genuinely hard tradeoffs

Development

Activity	Why it's expensive	How to contain it
Open-ended bug hunting ("why is this broken?")	Exploration plus repeated run, read, edit loops	Reproduce first, then give the failing test, file and function
Multi-file feature generation	Many reads and edits, and every step resends the growing history	Break into small, scoped steps with explicit files
Repo-wide changes (renames, API changes, dependency bumps)	Touches many files, and each diff and check adds context	Use scripted tools such as codemods, `sed` or IDE refactors for the mechanical part
Fixing build or compile errors in a loop	Long compiler output gets resent on every retry	Fix the first error, trim output, cap retries
Long-running chats that mix many tasks	Stale context from earlier work rides along	Start a fresh session per task

Testing and QA

Activity	Why it's expensive	How to contain it
"Make all the tests pass" loops	Runs the full suite, reads huge output, edits, reruns, repeats	Target one failing test at a time (`pytest path::test -x -q`)
Generating tests for an entire module or service at once	Reads lots of source and produces lots of output	Generate per function or class, using a template or example test
Debugging flaky or intermittent tests	Many reruns, timing logs and guesswork	Collect evidence first (seed, logs, failure rate), then ask
E2E and UI test debugging (Selenium, Playwright)	Verbose logs, DOM dumps, screenshots, traces	Share only the failing step, selector and trimmed error
Large mock or test data generation	Output tokens are the priciest kind	Ask for a small sample plus a generator script
Coverage-gap analysis over the whole repo	Reads coverage reports and many source files	Run the coverage tool yourself and paste only the uncovered lines
Analyzing big CI logs	Thousands of log lines per paste or read	Filter to the failing job and step, using `grep` or `tail`
LLM or RAG evaluation runs (for example LLM-as-judge scoring)	Every test case is one or more model calls, so runs multiply quickly	Start with a small dataset, cache results, and use cheaper judge models where possible

Code review, security and maintenance

Activity	Why it's expensive	How to contain it
Reviewing large PRs or full diffs	Big diffs plus surrounding context	Review file by file, or focus on risky areas
Security or dependency audits across the repo	Scans many files and lockfiles	Run SAST and dependency scanners first, then ask the AI about specific findings
Writing docs for a whole project	Reads lots of code and outputs long text	Document one module at a time from its public interface
Incident or root-cause analysis on production logs	Huge log volumes and repeated hypothesis loops	Pre-filter by time window, service and error ID

The common pattern

Almost every expensive activity combines one or more of these:

Wide input: many files, big logs, full diffs or whole reports.
Many iterations: retry loops, repeated test runs, trial and error.
Large output: bulk tests, big datasets, long docs.
Heavy reasoning: hard problems sent to thinking models.

Cut any one of the four and costs drop. Cut two and they fall dramatically.

8. The developer and tester playbook: 5 rules for AI cost control

You don't need a FinOps team to keep AI costs sane. You need some basic prompt hygiene.

Rule 1: Start fresh for each discrete task

Long threads accumulate stale file dumps, old test output and dead ends. Some tools summarize or trim long threads automatically, but carried-over noise still costs tokens and can confuse the model.

The rule: start a new session for each separate ticket, bug or subtask. Don't drag debugging baggage into feature work.

Rule 2: Control what context goes in

Closing unrelated tabs can help in editors that pull from open files, but explicit context matters more.

The rule: attach exactly the files you want (for example with @-mentions) and state the scope: "Only change billing_service.py and test_billing.py."

Rule 3: Use explicit pointers instead of open-ended crawling

Avoid prompts like "Find out why the tests are failing and fix them." The agent has to explore directory trees, configs and many files, spending tokens the whole way.

Better: run the test yourself, isolate the failure, and point to it:
"test_refund_rounding in tests/test_billing.py fails with this assertion. The logic is in billing/refunds.py, calculate_refund(). Fix the rounding."

Rule 4: Right-size the model

Don't use the heaviest reasoning model for mock JSON fixtures, regex or unit-test boilerplate.

Use fast, cheaper models for routine scaffolding, syntax and boilerplate.
Save frontier reasoning models for multi-file bugs, concurrency issues and complex refactors.

Rule 5: Trim logs and output

Don't paste 400 lines of terminal output. Keep the failing assertion, the few most relevant stack frames and the input that triggered it. When an agent runs commands itself, ask for quiet output, for example pytest -q or pytest -x --tb=short.

Bonus: cap retries. If the agent has failed the same fix three times, stop it, look yourself, and give it a sharper pointer.

Summary

AI tools don't bill by the hour or by how clever the answer is. They bill by volume and iteration: every input token, every generated token, every reasoning token, and every step of every loop.

Treat an assistant like an all-knowing oracle and let it wander your repo, and agent loops will multiply your usage fast. Scope the task, point to exact files, trim noisy output, choose the right model, and let caching work for you, and you can keep the same engineering speed at a fraction of the cost.

Originally published on Medium.

DEV Community: code&cosmos