Introduction
Have you ever run into this situation? You ask AI to fix a few bugs or adjust a screen, and your usage allowance is almost gone before much work has been completed.
As AI becomes more capable of handling complex development tasks, we naturally start giving it more work. But greater capability does not mean every run is efficient. Sometimes, what really consumes resources is repeatedly searching for information, guessing requirements, heading in the wrong direction, and fixing problems introduced by earlier changes.
Model choice, reasoning settings, context length, and platform billing policies all affect cost. But among the things developers can control, one question is especially worth addressing first: Are we making AI do too much work that could have been avoided?
This article covers several common situations that consume tokens, along with working habits I believe are worth developing.
1. First, Understand Where the Tokens Go
Tokens are the basic units a model uses to process content. They are not directly equivalent to words or characters. In addition to our instructions and the AI’s responses, code, documents, conversation history, and tool results sent to the model may all become part of its input.
Some reasoning models also generate reasoning tokens that are not directly shown in the final response. How these are billed or counted against usage limits varies by model and platform.
From the workflow a user can observe, a development task can roughly be broken down as follows:
Understand the requirements → Find information → Analyze and plan → Implement → Validate → Revise if necessary
This is a useful way to describe the workflow, not a fixed sequence of internal thinking steps followed by every model.
If the requirements are vague, AI has to fill in the gaps with assumptions. If it searches in the wrong place, it has to search again. If the acceptance criteria are unclear, it may keep working on things you never needed. Each extra loop can mean another round of model calls, data retrieval, and output.
Also, API token charges, subscription usage allowances in chat products, and request rate limits are different mechanisms. They cannot be directly converted into one another. Feeling that your allowance is running out quickly is not, by itself, evidence that a provider has reduced the model’s capabilities.
2. Unable to Find the Target: Spending Heavily Before Making Any Changes
Suppose you need to change the code for a particular screen. You would usually start by asking: Which page? Which component? Which API? Then you would narrow the scope using search or the project’s directory structure.
AI needs those clues too.
A request like “There’s a bug with orders in production. Please fix it” may force an agent to investigate everything from the project structure, routes, screens, and services to the database. Tools and indexes affect how it searches, but insufficient information can certainly broaden the search.
Remember: AI cannot read your mind. When clues are missing, it can only form assumptions based on the information available—and those assumptions may be wrong.
Check the Most Likely Cause First, Then Broaden the Search
Before giving instructions, try to organize the symptoms, reproduction steps, expected and actual results, error messages, and potentially relevant pages, APIs, or files.
For example, instead of saying “Orders are broken,” you could write:
After switching to the second page of the order list and clicking Cancel Order, the screen still shows the original status. It only updates after a refresh. Please first check the list’s state updates and cache invalidation logic, confirm the cause, and then make changes.
This description helps AI prioritize the most likely path.
However, you must distinguish a possible cause from a confirmed fact. If a frontend caching issue is only your guess, say so explicitly. Otherwise, AI may treat an incorrect premise as an established conclusion. If the first hypothesis does not hold up, let it broaden the search based on evidence.
Build an Index So AI Knows Where to Start
As a project grows, you can maintain a concise architecture index covering the main modules, routes, data flows, core services, and test entry points. Where supported, semantic indexes or code knowledge graphs can also help locate relevant symbols.
The purpose of an index is to provide starting points, not to rewrite the entire project as a second set of documentation. The tradeoff is maintenance: when the code changes, key index entries must change too. Outdated documentation can be more misleading than no documentation at all.
Human review is still necessary. What you want to maintain is a record of facts, not hallucinations.
Do Not Treat Conversation History as the Project’s Only Memory
After a long development session, a single conversation may contain old requirements, failed approaches, temporary decisions, and assumptions that were later rejected. Once the context has been compressed, some details may also be incomplete.
Record important decisions, the current state, and remaining tasks in documents that can be inspected and updated. When a conversation has drifted away from the current situation, start a new session with a concise handoff summary, and instruct AI to use the current files and verifiable results as the source of truth.
Starting a new conversation does not automatically restore modified code. If you need to roll back, you still have to inspect the Git diff or the tool’s checkpoints.
3. Understanding Documents: Give Each Step to the Right Tool
Another common scenario is asking a development agent to read a specification document and immediately start implementing it:
PDF/Word → Content extraction → Specification analysis → Comparison with the code → Implementation and validation
This workflow is reasonable, but that does not mean every step needs to be handled by the same expensive model.
For documents with a text layer, use a dedicated tool to extract the content first. Scanned documents may require OCR—optical character recognition. When tables, flowcharts, or layout relationships matter, preserve their structure and use a model with visual understanding when necessary. Converting everything to plain text can also lose important information.
Once the content is available, choose a suitable model to organize the specifications based on their complexity. If the developer already understands the domain, directly providing confirmed technical requirements can often eliminate an extra round of interpretation and restatement.
However, dividing the work is not free. Handoffs between models introduce costs through repeated reading, information transfer, and possible omissions. If a short document can be handled well in one pass, deliberately splitting it among multiple agents may not be worthwhile.
Also, make the priority of information sources explicit. For example: requirements confirmed for this task take precedence over older specifications; if the document conflicts with the current code, list the differences first and then confirm how to proceed.
4. Starting Too Early: The More You Do in the Wrong Direction, the More It Costs
We often send a prompt and wait for AI to deliver a result. Only halfway through—or after everything is finished—do we realize it chose the wrong technology, changed the wrong workflow, or treated test data as the production design.
Planning first is usually helpful when building a system, adding functionality across modules, or making changes involving data models and permissions.
For example, GitHub Copilot’s Plan mode can research a task and propose a plan before changing any code. Whether other tools offer the same capability, and what they call it or how it works, depends on the product. Even without a dedicated mode, you can ask AI to propose an approach and identify questions that need clarification before proceeding.
During planning, it is worth confirming:
- Which technologies will be used?
- What outcome is required, and what is outside the scope?
- Which screens, modules, and data will be affected?
- Will the implementation use real data or mocks? Which platforms must it support?
- How will the completed work be accepted?
Planning also consumes tokens, so every small change does not need to become a full requirements interview. The point is to adjust the amount of upfront clarification to the task’s uncertainty and impact.
5. Ambiguous Wording: One Missing Sentence Can Lead to Many Revisions
Sometimes the issue is simple. “Make this match the one above” may feel obvious to a person, but AI may not know exactly what “this” and “the one above” refer to.
Before sending a request, quickly check four things: the target, its location, the constraints, and the acceptance criteria.
Please update the Save button on the member settings page, at the bottom of the form on the right. Use the project’s existing primary button style and preserve the current save logic. For acceptance, confirm that it works properly on both desktop and mobile.
Clarity does not require length. A few sentences that remove key ambiguities are usually more useful than a long description without clear boundaries.
6. Cache Misses: You May Pay Repeatedly for the Same Content
Caching can reduce some costs, but first we need to distinguish two things: conversation memory and a prompt cache are not the same concept.
Prompt caching mainly reuses previously processed input prefixes to reduce repeated computation. It does not simply reuse the previous answer, nor does it mean the model will remember everything or reason more accurately.
Cache hits typically depend on factors such as matching prefixes, content length, expiration periods, and platform settings. Staying in the same session does not guarantee a hit. Cache read discounts and write charges may also differ across models.
A Concrete Cost Example
The following example uses the publicly listed standard prices for the Claude Sonnet 4.6 API, in US dollars per million input tokens. It compares only the unit prices for the same amount of input, excluding output, other tool fees, and special pricing conditions.
Writing to the cache may cost more; repeated reads are where savings can emerge. For one million fixed input tokens, with one write followed by four reads:
- If every request is billed as regular input, the total is $15.
- If a five-minute cache is successfully used, the total is $3.75 + $0.30 × 4 = $4.95.
This is a simplified calculation assuming the reused content fully qualifies for caching and continues to produce cache hits. It excludes new content added in each turn and all output.
For API developers who can control request content, it helps to place stable rules, tool definitions, and shared data near the beginning, with frequently changing requirements toward the end. Check usage data to verify actual cache hits. With ready-made development tools, what you can do depends on the controls and visibility the tool provides.
Accumulating irrelevant history just to pursue cache hits may not be worthwhile either. Cheap unnecessary content is still unnecessary content.
7. Misunderstandings: Identify the Level of the Error Before Fixing It
Even with substantial information, AI may still change the wrong screen, choose the wrong component, or design a data structure that does not fit the business. Different situations call for different corrective actions.
The Overall Direction Is Wrong: Roll Back and Redefine
If AI misunderstood the requirements from the start and all subsequent changes rest on that incorrect premise, rolling back to the state before the changes and adding clear conditions is usually a cleaner approach.
First, however, confirm what the tool’s rollback actually covers. Some tools only return to an earlier point in the conversation; others also restore some files. Changes to databases, external services, or changes caused by commands may not be undone.
Only One Part Is Wrong: Keep the Results and Narrow the Fix
If you requested three things and the first two are complete, starting everything over because the third is wrong would waste work.
Be explicit: “Keep the results of the first two items and adjust only the third.” Then provide the actual result, expected result, and relevant location. After the fix, also check whether it affected the completed parts.
Use References to Reduce Abstract Descriptions
Suppose you want a hairstylist to cut your bangs. Would you describe the function curve f(x) = x/4 + 1, or show a reference photo?
For most people, the latter is easier to communicate. Working with AI is similar: existing screens, components, code examples, and API formats can all serve as concrete references.
But specify which parts should be reused. For example: “Use only the spacing and button style from this screen as a reference. Follow the current requirements for data fields and interaction flow.” Otherwise, AI may copy unsuitable details from the example as well.
Use Screenshots to Locate the Issue, Then Verify Through Actual Behavior
When a project has many similar components and text descriptions keep missing the target, screenshots with marked areas, combined with page paths and reproduction steps, usually make it easier to locate the issue.
However, a screenshot only shows a screen at one point in time. It cannot, on its own, prove that the data is correct, interactions work, or the problem has been resolved. When necessary, have an agent with appropriate permissions interact with the application and reproduce the issue, then verify the outcome using relevant tests or logs. This reduces the chance of declaring success based on appearance alone.
8. Doing Too Much: Let AI Spend Its Effort Where Judgment Is Needed
The situations above have something in common: AI did a lot of work, but some of it did not move the task forward.
Examples include repeatedly converting the same document, looking up standard commands again, reading entire huge logs, or running unrelated project-wide workflows for a small change.
The principle is simple: turn repeatable processes into tools, and use the model for the parts that require judgment.
Turn Repeated Workflows into Tools and Skills
Repeatable work such as file conversion, formatting, data checks, and report generation is well suited to scripts. A Skill can then explain when to use those scripts, what inputs they require, and how to verify the results.
A Skill is not a zero-cost shortcut. Its value lies in reducing the need to rediscover and orchestrate a workflow every time. The more stable the underlying tools are, the easier it is to accumulate benefits.
Even before creating a Skill, you can start with project instruction files supported by your tools and an index under docs/, such as the AGENTS.md file used by some agents. File names, loading rules, and support vary by tool. Placing a document in a folder does not mean every agent will read it automatically.
Multiple Agents: Calculate Handoff Costs Before Assigning Roles
For large tasks, you can borrow from project management and separate requirements clarification, architecture design, implementation, testing, and operational checks.
But you do not need an agent for every job title. Each agent may still need to read background information, exchange information, and validate results. More roles can mean higher coordination costs. Parallel execution can shorten waiting time without reducing total token usage.
A more reasonable approach is to divide work according to task boundaries, dependencies, and acceptance criteria. Let a single agent with appropriate tools handle small tasks, and split larger tasks into pieces that can be completed independently.
The value of agentic engineering lies in clear task decomposition, accurate handoffs, and reliable validation. Whether it is worthwhile should be judged by the total cost and quality of completing the task.
Conclusion: Saving Tokens Starts with Reducing Rework
Sometimes tokens burn quickly because the task is inherently complex. Other times, it is because we send AI in circles among unclear goals, outdated context, and repetitive work.
Rather than simply shortening prompts, I care more about clarifying four things before handing over a task: what needs to change, where to start, what constraints apply, and what counts as done.
Combine that with an appropriate model, concise context, reusable tools, and necessary validation, and each run has a better chance of moving toward the goal.
What we should measure is the total cost of completing the same task—including tokens, waiting time, and the time spent on human corrections.
References
- GitHub’s official documentation on Plan mode
- Prompt caching | OpenAI API
- Anthropic’s official pricing information
Coming Next: How Do You Give AI a Requirement?
Why do some ideas still require repeated revisions from AI even when we feel we have explained them clearly?
When people communicate, shared experience often fills in the details left unsaid. Requirements given to AI need those unspoken understandings translated into conditions that can be understood, executed, and verified.
In the next article, we will explore how to turn the ideas in your head into requirements AI can reliably implement.
The original article was written in Traditional Chinese, and the Japanese and English versions were translated using ChatGPT.
Top comments (0)