Model prices matter, but the largest waste often starts before the first token is billed. Undefined jobs make agents read too much, retain stale context, retry without a new strategy and call expensive models for routine work.
The useful metric is not cost per token. It is cost per accepted task.
Price the completed job, not the individual token
Token prices are useful for procurement. They are a poor measure of whether an agent is economical.
Suppose one model costs half as much per million tokens but needs three attempts to produce an acceptable result. Another model costs more per token but completes the job once, with the right evidence and format. The cheaper model may create the more expensive workflow.
The useful unit is cost per accepted task.
For each recurring agent job, track:
- input tokens
- cached input tokens
- output and reasoning tokens
- tool calls
- retries
- elapsed time
- human review time
- whether the result passed the acceptance check
OpenAI separates input, output, cached, and reasoning tokens in its usage reporting. Anthropic exposes input, output, cache-write, and cache-read usage for Claude. Those fields become meaningful when they sit beside the business outcome.
“This run used 80,000 tokens” tells you very little. “This agent resolved 42 support cases at $0.18 per accepted case, with four escalations and two reworks” gives an operator something to manage.
Give the agent a bounded job
An agent job needs six parts:
- Trigger: What starts the work?
- Approved inputs: Which records may it read?
- Permitted actions: Which tools may it use, and with what authority?
- Output: What must it produce?
- Acceptance check: How will the system or reviewer decide that the work passes?
- Stop and escalation rules: When must it stop, ask, or hand the work over?
Consider a weekly sales-pipeline agent.
“Analyse the CRM and tell me what matters” gives the agent an unlimited reading assignment. It may scan years of records, revisit dead opportunities, search the web for every account, and produce a long report that nobody can act on.
A bounded job looks different:
Every Friday at 4pm, review open opportunities changed in the past 14 days. Flag records with no next action, a close date in the past, or a value above $25,000 with no activity for seven days. Produce a table with the opportunity, issue, supporting CRM fields, and recommended owner action. Do not change CRM records. Escalate missing or conflicting data to the sales operations lead.
Every Friday at 4pm, review open opportunities changed in the past 14 days. Flag records with no next action, a close date in the past, or a value above $25,000 with no activity for seven days. Produce a table with the opportunity, issue, supporting CRM fields, and recommended owner action. Do not change CRM records. Escalate missing or conflicting data to the sales operations lead.
The second version defines the query window, rules, evidence, output, permissions, and escalation path. The agent has less freedom, but it has a much better chance of completing the work in one pass.
Context is an operating budget
Teams often treat a large context window as permission to keep everything.
Long-running agents accumulate conversation history, tool schemas, retrieved documents, screenshots, intermediate reasoning, failed attempts, and raw tool output. Much of that material helped at one point. It does not need to travel through every later step.
Model providers have built cost controls around this problem. Anthropic’s context-management guidance recommends compaction for long-running conversations and selective removal when finer control is required. Google Cloud’s context-caching guidance describes the cost and latency created when applications repeatedly send the same documents, instructions, or media for processing.
Use three context layers:
- Stable context: policies, tool definitions, schemas, and instructions that rarely change
- Task context: the records and evidence needed for the current job
- Working context: temporary tool results and intermediate notes that can be discarded after the step
Stable context is a candidate for caching. Task context should be retrieved narrowly. Working context should expire.
Prompt caching can reduce the price of reused input, but it does not make irrelevant context useful. Anthropic’s prompt-caching documentation and OpenAI’s prompt-caching guidance both reward repeated, stable prompt prefixes. A prompt that changes on every run will miss the cache. A bloated prompt can still cost less when cached while remaining slow, distracting, and hard to debug.
Context policy belongs in the runtime design, alongside access control and retry policy.
Put a budget around retries and parallel work
Retries make agents look persistent. Unbounded retries make them expensive and unpredictable.
Each retry should answer a specific failure:
- Was a tool temporarily unavailable?
- Did the output fail a schema check?
- Was required evidence missing?
- Did the model misunderstand the task?
- Has anything changed that makes another attempt likely to succeed?
If the answer is unclear, another identical attempt is gambling with tokens.
Set a maximum attempt count for each step. Change the strategy before retrying. Use a cheaper model for extraction or classification, then route ambiguous cases to a stronger model. Stop and escalate when the same error repeats.
Apply the same discipline to parallel agents. Five agents researching the same question can provide breadth, or they can produce five overlapping summaries that a sixth agent must reconcile. Parallel work earns its cost when the tasks are independent, the expected outputs differ, and the coordinator knows how to combine them.
Give every branch a purpose, a token or time ceiling, and a return format.
Acceptance criteria reduce rework
An agent can complete every technical step and still fail the job.
A content agent may publish a grammatically clean article that sounds generic. A support agent may draft a polite answer without resolving the customer’s issue. A research agent may collect twenty sources without identifying which claim each source supports.
Acceptance criteria should be testable before the agent starts.
For a research brief, that might mean:
- every factual claim has a primary source
- sources fall within the specified date range
- conflicting evidence is identified
- the final brief stays under 800 words
- the recommendation names an owner and next action
The agent can check most of those conditions before handing the work to a person. Early checks are cheaper than full reruns.
Human review also needs a defined decision. Ask the reviewer to approve, reject with a reason code, or escalate. Free-form feedback such as “make it better” sends the agent back into another undefined job.
Run a 30-minute cost review
Choose one agent workflow that runs often enough to matter. Pull ten recent executions and record the total tokens, tool calls, retries, duration, review time, and pass rate.
Then inspect the expensive runs:
- Which context was repeatedly sent but rarely used?
- Which step used a stronger model than the decision required?
- Which retries repeated the same strategy?
- Which tool results remained in context after their useful step?
- Which missing acceptance rule caused rework?
Change one boundary at a time and measure the next ten runs. Shorten the retrieval window. Cache the stable prefix. Cap retries. Route routine steps to a smaller model. Add a schema check before human review.
Do not optimise for the lowest token count. Optimise for the lowest cost that still produces an accepted result at the required speed and risk level.
Before buying another subscription or moving the whole workflow to a cheaper model, write the agent’s job in six lines: trigger, inputs, actions, output, acceptance check, and stop rule. If those lines remain vague, the cost will remain unpredictable.
The Nexius Labs guide to Loop Engineering and token efficiency explains why AI cost should be measured per successful outcome — not merely per model call.
Top comments (0)