AI-assisted development is rapidly moving beyond autocomplete and simple chat interfaces.
We are entering the era of agentic development.
Instead of asking an AI model to generate a function, developers can now give an agent a task such as:
"Find the cause of this bug, inspect the relevant files, implement the fix, run the tests, and verify that everything still works."
The agent may then read dozens of files, search the repository, call tools, execute commands, inspect the results, modify code, encounter an error, retry, and continue until the task is complete.
This is incredibly powerful.
But there is another side to it that is much easier to ignore:
all of those interactions consume tokens.
And as AI becomes a normal part of software development, understanding how those tokens are used will become increasingly important.
This article is the first part of a three-part series about token-efficient agentic development.
Before talking about optimization, monitoring, model selection, or local AI, we first need to understand what we are actually consuming.
What Is a Token?
Large Language Models do not process text exactly the way humans do.
They do not simply see words.
Instead, text is divided into smaller units called tokens.
A token can represent:
- a full word,
- part of a word,
- punctuation,
- whitespace,
- a number,
- or a fragment of source code.
For example, a simple sentence such as:
The user authentication failed.
might be split into several tokens.
Source code is tokenized in the same way.
const user = await getUserById(id);
The model does not necessarily see this as one logical programming statement. It sees a sequence of tokens representing pieces of that statement.
The exact tokenization depends on the model and tokenizer.
This is the first important concept:
Tokens are the basic units of information processed by a language model.
Input Tokens and Output Tokens
At a high level, AI usage can be divided into two categories.
Input tokens
Everything sent to the model.
This may include:
- your prompt,
- conversation history,
- system instructions,
- project instructions,
- source code,
- documentation,
- tool results,
- terminal output,
- error messages,
- retrieved files,
- previous agent steps.
Output tokens
Everything generated by the model.
For example:
- explanations,
- generated code,
- tool calls,
- commands,
- plans,
- responses.
A simple interaction might therefore look like this:
Input:
2,000 tokens
Output:
800 tokens
Total:
2,800 tokens
For a normal chatbot interaction, this is relatively easy to understand.
Agentic development makes the situation more complicated.
Why Agentic Development Changes Everything
Imagine asking an AI:
Create a TypeScript function that validates an email address.
The model receives a small prompt and produces a relatively small response.
Now compare that with:
Investigate why user registration sometimes fails,
find the relevant frontend and backend code,
fix the issue,
run the tests,
and make sure the solution follows the existing architecture.
An agent handling this task might:
- inspect the repository structure,
- read several files,
- search for registration-related code,
- inspect API calls,
- inspect backend validation,
- read existing tests,
- modify code,
- execute tests,
- receive an error,
- inspect the failure,
- modify the code again,
- rerun the tests,
- summarize the solution.
Each step creates additional context.
A simplified workflow could look something like this:
Developer
↓
Agent
↓
Read files
↓
Model
↓
Search repository
↓
Model
↓
Modify code
↓
Run tests
↓
Model
↓
Read errors
↓
Modify code again
↓
Run tests again
Every interaction between the model and its environment may involve additional tokens.
This creates what we can call an agent loop.
The Agent Loop
A traditional AI interaction is often:
Prompt → Model → Answer
An agentic workflow is closer to:
Task
↓
Reason about next action
↓
Use tool
↓
Receive result
↓
Evaluate result
↓
Use another tool
↓
Receive result
↓
Continue...
The important part is that the model often needs context from previous steps to decide what to do next.
That means a task that appears simple from the developer's point of view may involve a surprisingly large amount of model interaction.
The developer might write only:
Fix the login bug.
But the agent could process tens of thousands of tokens before completing the task.
This creates an important distinction:
Prompt length is not the same thing as total AI usage.
In agentic development, the visible prompt may represent only a small fraction of the actual workload.
The Context Window
Another important concept is the context window.
The context window represents how much information a model can consider during an interaction.
The context may contain things such as:
System instructions
Project instructions
Developer prompt
Conversation history
Source files
Documentation
Tool outputs
Terminal logs
Previous agent actions
A larger context window allows the model to work with more information.
That sounds purely beneficial.
But more context is not automatically better.
Consider an agent working on a frontend validation bug.
Ideally, it might need:
Form component
Validation schema
API client
Relevant types
Related tests
Instead, imagine the agent loads:
Entire repository structure
40 unrelated components
Large package lock file
Generated code
Old logs
Documentation
Backend files unrelated to the feature
Thousands of lines of terminal output
The agent now has much more information.
But most of it is irrelevant.
This is context pollution.
And context pollution has two major costs.
First, it consumes more tokens.
Second, it can make it harder for the model to focus on the information that actually matters.
More Context Is Not Always Better Context
One of the easiest mistakes in AI-assisted development is assuming:
"If the model knows everything about the repository, it will perform better."
Sometimes that is true.
Often it is not.
A better principle is:
Give the model enough context to solve the task, but not everything you have.
This is very similar to software design itself.
We rarely want every component to depend on the entire system.
Good software architecture tries to reduce unnecessary dependencies.
Good AI workflows should do something similar with context.
Instead of:
Entire repository
↓
Model
prefer:
Relevant files
Relevant instructions
Relevant documentation
↓
Model
Context should be treated as a resource.
Hidden Token Consumption in Development Workflows
Developers often think about token usage only when they type a prompt.
But modern AI development tools can consume tokens in many other places.
For example:
Repository exploration
An agent may read many files before finding the relevant ones.
Search results
Repository searches can return large amounts of text.
Terminal output
A build failure might produce hundreds or thousands of lines of logs.
Test output
Large test suites can generate significant amounts of context.
Documentation
Agents may automatically load documentation or project instructions.
Repeated context
Information already processed earlier may appear again in later interactions.
Agent retries
An agent can attempt one solution, fail, analyze the failure, and try again.
Subagents
Some systems allow one agent to delegate tasks to additional agents.
Each subagent may have its own context and model usage.
None of these mechanisms are inherently bad.
They are often exactly what makes an agent useful.
The problem starts when we stop thinking about their cost.
A Simple Example
Imagine two agents solving the same problem.
Agent A
Agent A receives:
Fix the validation bug in the registration form.
It reads the entire frontend repository.
Then it reads several backend files.
It runs the full test suite.
The test suite produces a large log.
The agent modifies the wrong component.
Tests fail.
It reads another set of files.
It tries again.
Eventually, the bug is fixed.
Agent B
Agent B receives:
The registration form incorrectly accepts dates in the future.
The form is located in:
src/features/registration/
Validation is handled with Zod.
Find the relevant schema, fix the validation,
and run only the related tests.
The second agent has a better starting point.
It may inspect fewer files, run fewer commands, generate less irrelevant output, and finish in fewer steps.
Both agents solve the same problem.
But their resource usage can be dramatically different.
This is the core idea behind token efficiency.
Token Usage Is Not the Same as Token Efficiency
There is an important distinction here.
The goal should not be:
Use as few tokens as possible.
That can easily become counterproductive.
Imagine a small model uses 20,000 tokens while repeatedly attempting to solve a difficult architectural problem.
A more capable model might solve the same problem using 8,000 tokens.
Even if the stronger model is more expensive per token, it may still be the more efficient choice overall.
That means we should not optimize only for:
Tokens Used
We should think about something closer to:
Useful Work Produced
────────────────────
Resource Consumption
Or, more simply:
Token Efficiency
A useful mental model is:
Token Efficiency =
Useful Output / Token Cost
This is not meant to be a precise mathematical metric.
It is a way of thinking.
A workflow that uses more tokens but reliably solves the problem may be more efficient than one that uses fewer tokens but requires constant human intervention.
Developer Time Matters Too
There is another resource that should not be forgotten:
developer time.
Imagine optimizing an AI workflow so aggressively that developers spend ten minutes preparing the perfect minimal context for a task that the agent could have solved automatically in thirty seconds.
Technically, token usage decreased.
But total productivity may have become worse.
A better optimization target is something closer to:
AI cost
+
Developer time
+
Failure rate
+
Iteration count
Token optimization should therefore support productivity rather than fight against it.
The goal is not to make AI usage artificially cheap.
The goal is to eliminate waste.
Agentic Development Is Becoming an Engineering Problem
When only a few developers occasionally use AI, inefficient token usage may not matter very much.
But imagine a larger engineering organization.
Suppose:
200 developers
×
multiple AI interactions per day
×
agents reading repositories
×
automated tool calls
×
multiple models
Small inefficiencies suddenly become large ones.
An unnecessary repository scan performed once is irrelevant.
Performed thousands of times across an organization, it becomes infrastructure cost.
This is why AI usage will increasingly require the same kind of thinking we already apply to other engineering resources.
We monitor:
CPU usage
Memory usage
Cloud infrastructure
Database queries
Network traffic
API calls
It makes sense to eventually treat:
AI model usage
Context size
Token consumption
Agent iterations
Model selection
with similar discipline.
This is where ideas such as AI FinOps start becoming relevant.
The Most Expensive Model Is Not Always the Best Model
Modern development environments increasingly provide access to multiple AI models.
That creates another important optimization problem.
Different tasks require different levels of capability.
For example:
Rename a variable
and
Redesign the authentication architecture of a distributed system
are very different tasks.
Yet developers sometimes use the same high-capability model for both.
This is similar to running every workload on the largest available cloud machine.
It works.
But it is rarely efficient.
A mature AI development workflow should eventually be able to answer:
What kind of task is this?
How complex is it?
How much context does it require?
Which model is sufficient?
Should this task even use a cloud model?
Could a smaller or local model handle it?
We will explore this in the next part of the series.
Local Models Change the Equation
Token optimization is not only about reducing usage.
Another option is changing where the computation happens.
Open-weight and locally hosted models can make certain workloads independent from traditional per-token API pricing.
For some tasks, organizations might use:
Cloud models
+
Local models
+
Specialized smaller models
instead of sending every task to the most capable external model available.
However, local AI does not make computation free.
The cost simply moves.
Instead of paying directly for tokens, organizations may need to think about:
GPU infrastructure
Electricity
Hardware
Deployment
Maintenance
Model serving
Scaling
Monitoring
This creates another engineering tradeoff rather than eliminating the problem.
The Real Goal: More Value per Token
AI coding tools will continue to become more capable.
Agents will read more code, execute more commands, use more tools, and solve increasingly complex tasks.
Trying to prevent them from consuming tokens would defeat much of the purpose.
The better question is:
How much useful engineering work are we getting from the resources we consume?
That leads to a much healthier approach.
Instead of:
Use fewer tokens.
think:
Avoid unnecessary context.
Avoid unnecessary agent loops.
Use the right model for the task.
Provide better instructions.
Monitor usage.
Measure outcomes.
Use expensive models where they create value.
Use cheaper or local models where they are sufficient.
The goal is not minimum token usage.
The goal is maximum useful work per token.
What's Next?
This article focused on the foundations:
- what tokens are,
- input and output tokens,
- context windows,
- agent loops,
- context pollution,
- hidden token consumption,
- and the difference between token usage and token efficiency.
In Part 2, we will move from theory to practice.
We will look at how developers and engineering teams can actually reduce unnecessary AI usage through:
- token monitoring,
- smarter model selection,
- context management,
- better prompts and project instructions,
- limiting unnecessary repository access,
- controlling agent loops,
- local AI models,
- and AI FinOps practices.
Then, in Part 3, we will combine everything into a practical framework for building a token-efficient agentic development workflow.
Because the future of AI-assisted software development is not simply about using more AI.
It is about using AI efficiently.
Thanks for reading — I’m Marxon, a developer and AI specialist exploring how AI reshapes the way we build, manage, and think about technology.
If you enjoyed this year-end special, follow me here on dev.to
and join me on X where I share shorter thoughts, experiments, and behind-the-scenes ideas.
Let’s keep building — thoughtfully. 🚀
Top comments (3)
The useful work / resource consumption framing is the right one, and I'm glad you landed there instead of "minimize tokens." The Agent B example nails it — the better-scoped starting prompt isn't just more efficient, it's more likely to get the right answer. The cost reduction is almost a side effect.
The developer time variable is the one most people skip when benchmarking agent workflows. A 30% token reduction that adds 20 minutes of context-wrangling isn't an optimization.
Exactly. Token count alone is a pretty weak optimization metric.
If reducing token usage increases developer effort, iteration count, or failure rate, you've probably just moved the cost somewhere else. That's why I think the more useful question is: how much reliable engineering work are we getting for the total resources spent?
And I agree on Agent B — good context isn't only cheaper context. It's usually better context. In many cases, lower token usage is simply the result of giving the agent a clearer problem to solve.
Curious how others feel about this.
Do you use AI as a copilot, a code generator, or mostly as a second opinion?
And honest answers only — where did it actually save you time this week?