DEV Community

Cover image for Token prices lie: how cheap AI agents burn budgets
Andrey Altrouter
Andrey Altrouter

Posted on Originally published at altrouter.ai

Token prices lie: how cheap AI agents burn budgets

A developer launches an agent for background bug fixing on a cheap model and a standard framework. They expect costs in pennies but risk getting a bill that can grow 31 times larger due to uncontrolled auto-retries.

We will show why optimizing for cheap tokens leads to a high cost-per-outcome and why 40 to 60 percent of the budget goes to garbage in the context.

Agent tasks burn millions of tokens per ticket

An agent writing code consumes a thousand times more tokens compared to a regular chat about code. A 2026 study states this directly. A programmer sets a task to fix an error in the application logic. The program starts working. First, the script requests the directory structure. Then it reads the contents of several source code files. After that, it writes an action plan. Next, it calls text editing tools. Finally, it runs tests and checks the compiler logs. Every tool call requires passing the entire previous history of actions in a new API request.

Statistics show the real scale of consumption. A single agent task with SWE-bench complexity burns between 1M and 3.5M tokens. SWE-bench evaluates the ability of systems to solve real problems from public repositories. This is absolutely normal behavior for successfully completing a complex ticket. You pay for every iteration of searching files and reading error logs. The context grows like an avalanche. On the first step, the model reads a short prompt. On the tenth step, the model reads the prompt, nine of its past answers, the contents of three files, and nine terminal responses with test results. The process repeats dozens of times until all checks pass.

The problem arises when choosing a model. An engineer opens the catalog and sees the prices for DeepSeek V4.1 Flash at 0.15 to 0.30 dollars per million input tokens. The temptation is to give the script an infinite number of attempts. Allowing a cheap neural network to make fifteen mistakes seems logical while a flagship model gets only one try. Practice proves this approach wrong. The strategy breaks against the transformer attention mechanism.

The cheap model chokes on its own context errors

Every program step is added to the conversation history. The model sees its previous incorrect code and a long compiler error log. The self-conditioning process begins. Neural networks show a sharp drop in accuracy over long distances. They accumulate their own past errors in the context.

The presence of bad code in the prompt forces the system to make more mistakes in future steps. The attention mechanism assigns weight to all tokens in the window. A hallucination from a past response affects the generation of the next one. The program starts walking in circles. It tries to fix the syntax, breaks the logic, rolls back the changes, and breaks the syntax again. Cheap versions are especially prone to wordy apologies. They generate long paragraphs of text promising to fix the error. The bill grows for generating these apologies at the output token rate.

Research shows a sad picture. Between 40 and 60 percent of corporate API budgets goes to operational inefficiency. The budget is spent on redundant system prompts and empty reasoning. The neural network repeatedly reads a massive chunk of garbage from past failed attempts. The payload drowns in the logs of previous crashes. The project owner pays to process useless text on every new call. A longer failure history means fewer chances for a successful outcome.

Naive auto-retries break external systems

Scripts often have access to external tools. They create tickets, send emails, and make database commits. A naive retry mechanism upon failure becomes dangerous for the infrastructure.

Stylish stapler driving a huge stack of staples into the exact same spot on a blank sheet of paper

Scripts successfully complete tasks under normal conditions with an 84 percent probability. With minor injection failures, the recovery rate drops to 47 percent. An injection failure means a non-standard API response. The tool returned a server error or broken HTML instead of the expected JSON format. The neural network fails to understand the refusal reason and simply calls the tool again with the same parameters.

In 53.33 percent of cases, naive auto-retries lead to duplicated external effects. The program receives a timeout error from the issue tracker. The ticket was actually created on the server. The script repeats the request and creates a duplicate. A loop emerges, cluttering production systems with thousands of identical records. Investigating such incidents takes hours of the team working time. We mentioned this in our article about creating AI agents. Automatic action rollbacks work poorly due to the unpredictable state of external systems.

Default framework settings bankrupt you

Different frameworks have vastly different safeguards. Programmers install a library and leave the default settings.

The OpenAI Agents SDK interrupts work upon reaching ten iterations. This is a hard limit from the creators of proprietary models. LangGraph in its base configuration allows execution of up to 10007 steps. The open-source framework shifts the responsibility for budget control to the programmer. Ten thousand steps with a growing context will guaranteed drain the balance of any project.

Humans often misjudge task complexity for a script. A developer sets a limit of one hundred steps for the simplest bug fix. They expect completion in three steps. The program gets stuck reading a file and uses the entire allocated limit.

An incident with complex image processing is a good example. The script received a base64 encoding of a picture as text. The string consisted of hundreds of thousands of characters. The program got stuck in a loop while trying to read this massive array of characters as executable code. Token consumption increased 31 times before an operator spotted the error. The tokenizer splits the meaningless set of letters into a huge number of tiny fragments. The default limit allowed a single process to burn a monthly budget in a couple of hours.

Standard calculators hide the arithmetic progression

A normal request cost calculation multiplies the prompt size by the token price. This approach works for generating text or translating articles. Agent scripts break this formula. The agent context grows dynamically with each new action. An arithmetic progression effect occurs.

A developer gives the agent ten steps to complete a task. At each step, the script calls a tool and receives a response of ten thousand tokens. Multiplying ten steps by ten thousand tokens to get one hundred thousand total tokens seems logical. Real math works differently.

On the first step, the model processes ten thousand tokens. On the second step, the model processes the first ten thousand plus a new ten thousand. On the third step, the volume reaches thirty thousand. By the tenth step, the prompt size grows to one hundred thousand tokens. The total volume of processed input data over ten iterations is five hundred fifty thousand tokens. You are billed for this entire volume.

Every compiler error or long database log settles permanently in the conversation history. The script rereads this garbage on every subsequent iteration. The company pays to read the exact same error log nine times in a row. This progression turns cheap tokens into massive bills when an infinite loop occurs.

A flagship hitting it on the first try costs less than ten attempts

We can calculate the costs using specific numbers from pricing pages. Take the task of refactoring a large module. The requirement is to read 2M tokens of repository context and write a new class.

One perfect dart in the center of a dartboard and a pile of broken cheap darts on the floor

Compare three models from the catalog. We take GPT-6.1 Sol, Gemini 3.8 Flash, and DeepSeek V4.1 Flash. Imagine two scenarios. The first scenario assumes a clean solution in two steps. The first step eats 2M input and gives 50k output tokens. The second step eats 2.05M input and gives 150k output. Total input is 4.05M tokens. Total output is 0.2M tokens.

The second scenario describes a failure and a loop of fifteen steps. Total input reaches 35M tokens due to arithmetic progression. Total output reaches 2M tokens due to apology generation.

Model Input price (1M) Output price (1M) Success in 2 steps Failure in 15 steps
GPT-6.1 Sol $2.00 $10.00 $10.10 $90.00
Gemini 3.8 Flash $2.00 $10.00 $10.10 $90.00
DeepSeek V4.1 Flash $0.30 $1.20 $1.45 $12.90

Calculate the clean success for GPT-6.1 Sol. Input: 4.05M multiplied by 2.00 dollars. Output: 0.2M multiplied by 10.00 dollars. Final price: 10.10 dollars.

Calculate the failure for DeepSeek V4.1 Flash. Input: 35M multiplied by 0.30 dollars. Output: 2M multiplied by 1.20 dollars. Final price: 12.90 dollars.

The cheap model spent more money and failed the task. The difference in the base token price is completely offset by the volume of garbage in the context. Money is deducted for reading the same logs over and over. Read more about pricing mechanics in our API bill breakdown.

Switching to a reasoning model guarantees no escape from the loop

Moving to models with a chain of thought seems like an obvious solution. Practice shows the limits of this approach. Internal thinking does reduce the frequency of syntax errors in the early steps. The neural network plans actions better and makes fewer typos in variable names. The loop still happens when encountering a non-standard response from an external tool.

Large cloud providers do not guarantee bitwise reproducibility. The generated response can change with identical prompts even at zero temperature. The Mixture of Experts architecture directs requests to different cluster nodes. Different expert networks process the request. An agent runs locally for debugging. It passes the problem area without errors. The code goes to production. The script loops in the exact same spot again.

Non-determinism makes debugging infinite loops extremely hard. You cannot reliably reproduce the state where the script went crazy. Saving tokens by replacing standard models with reasoning versions in agent loops remains fragmented and depends heavily on the specific platform.

How to strictly limit the agent on Monday

You can stop the budget burn with a few modifications to the orchestrator code.

First, you need to set a hard step limit. Ten iterations are enough for most routine tasks. If a program fails to finish in ten steps, it will fail in a hundred. Getting a crash notification is better than a bill for millions of tokens.

Next, you must configure aggressive context filtering. Delete long error traces before the next request. Leaving only the last log line with the exception description is sufficient. Deleting intermediate reasoning steps from the history helps. This will slow down the avalanche growth of input tokens and reduce the self-conditioning effect. The neural network will stop relying on its past hallucinations.

Limit the budget and the number of attempts at the execution loop level. Here is an example for a script that calls an image generation API in a loop. The tool checks the iteration limit and interrupts work upon failures.

import requests

api_key = "ar-test-key-123"
url = "https://api.altrouter.ai/images/generations"
max_steps = 10

for step in range(max_steps):
    response = requests.post(
        url,
        headers={"Authorization": f"Bearer {api_key}"},
        json={
            "model": "nano-banana-pro",
            "prompt": "A developer looking at a huge API bill, cinematic lighting"
        }
    )

    if response.status_code == 200:
        print("Task completed successfully.")
        break

    print(f"Attempt {step + 1} failed. Status: {response.status_code}")
    # Log filtering logic before the next attempt goes here
Enter fullscreen mode Exit fullscreen mode

Hard limits like 10 iterations and context filtering protect against the arithmetic progression of expenses.

Top comments (0)