Why agentic architectures multiply inference spend, and how to price them before deployment
Recently, a coding agent was asked to fix a one-character typo in a README file (true story, bear with me). It listed the repository’s open issues, created a branch, committed the change, and opened a pull request, consuming more than 21,000 input tokens along the way. One keystroke of value, a small novel’s worth of compute.
Stories like this are no longer entertaining; in 2026 they are line items on an ever expanding bill. The economics of a standard LLM deployment and the economics of an agentic deployment share almost nothing beyond the vendor invoice, and most companies discover the difference after the architecture decision has already been made.
One Request Is Never One Call
Gartner’s March 2026 analysis puts agentic workloads at 5 to 30 times more tokens per task than a standard chatbot, and typical production agents land between ten and twenty model calls for a single user request.
The arithmetic worsens with ambition. RAG pipelines ship large context windows with every query. Always-on monitoring agents scan logs, inboxes, and market data around the clock, consuming compute whether or not a human is watching. These background workloads barely existed in enterprise budgets two years ago. Today they represent a growing share of inference spend that most finance teams never approved, because nobody itemized it.
At 10,000 users, a single agentic feature can run between $150,000 and $750,000 per month. The pilot that looked viable at fifty users was measuring a different system. The code did not change, and neither did the model. Volume changed, and volume turned out to be the entire story.
There is even a rough way in which this hits. Analyses of failed agent deployments place the cost cliff between 500 and 5,000 users, the range where cloud API pricing stops making sense and teams face a forced migration to self-hosted GPUs they never planned for. One documented startup watched its unit economics invert between 700 and 1,000 concurrent users and killed the product.
The system worked technically. It failed as a business, and the failure was baked in at the whiteboard stage.
The Context Tax
Token prices are falling, which makes the spending pattern look irrational until you decompose it. According to recent analysis, inference for a GPT-3.5-class model cost $20 per million tokens in 2022 and roughly $0.07 by October 2024, a 280x drop. Yet average enterprise AI budgets grew from about $1.2 million a year in 2024 to $7 million in 2026.
The bills kept climbing after the discount had already landed, which tells you the driver was never price. Consumption is outrunning it, by a wide margin.
The biggest driver hides inside the agent loop itself. Every reasoning step re-sends the system prompt, the tool definitions, the instructions, and the accumulated conversation state.
Companies are paying, repeatedly, for the model to reread material it has already processedAdd output pricing, which runs several times higher than input pricing at most providers, and any workflow that drafts long responses at each step compounds the problem further.
Vendor Pricing Follows the Cloud Playbook
There is a second reason the bills feel survivable right now. Current model pricing is subsidized. Inference providers are pricing below cost to win workloads, a strategy that closely resembles what cloud providers ran a decade ago. Discounted compute locked customers into architectures, and once the architectures were immovable, rates normalized. The finance teams that were surprised by cloud cost escalation in 2017 and 2018 remember how that conversation went.
The early signals are already visible. One industry analysis tracked a 4x increase in OpenAI pricing over twelve months for certain workloads. Whether that exact figure holds across providers matters less than the direction. Companies building agentic systems on today’s subsidized rates are underwriting their unit economics with someone else’s venture capital, and that subsidy carries an expiration date nobody will announce in advance.
The playbook worked the first time. There is little reason to expect a rewrite.
Building the Cost Model Into the Architecture
Research suggests small language models can handle 60 to 80 percent of enterprise agent subtasks at 10 to 30 times lower inference cost. Classification, formatting, and retrieval rarely need a frontier model, and a router that reserves expensive reasoning for the steps that genuinely require it changes the cost curve more than any pricing negotiation will.
The next lever is trimming and caching aggressively: pass each step only the context it needs rather than the entire history, and cache tool results and semantically similar responses so the system stops paying to generate the same answer twice. Since re-sent context drives the majority of agent spend, this is usually where the largest savings sit.
Capping the loops matters just as much. Classifying errors lets the agent know which failures deserve a retry and which should escalate to a human. An uncapped retry policy is an uncapped budget.
None of this holds without governance in place early. The emerging discipline of FinOps for AI exists for exactly this reason: tracking spend per task, per step, and per tool call matters because aggregate monthly numbers hide the one workflow consuming 40% of the budget on its own. Assigning an owner and setting thresholds now, while the numbers are small enough to discuss calmly, means that when the CFO eventually asks what an agent costs, someone has an answer more specific than the invoice total.
Where the Loop Earns Its Keep
None of this argues against agents. Where a loop replaces hours of skilled work, twenty model calls are a bargain at almost any token price. The discipline lies in knowing which workflows earn their loops and which are burning compute to move a comma.
The architecture decision and the budget decision are the same decision. Making them separately just delays the meeting where they merge and the invoice dictates the true outcome.
…
Nick Talwar is a CTO, ex-Microsoft, and a hands-on AI engineer who supports executives in navigating AI adoption. He shares insights on AI-first strategies to drive bottom-line impact.
→ Follow him on LinkedIn to catch his latest thoughts.
→ Subscribe to his free Substack for in-depth articles delivered straight to your inbox.
→ Watch the live session to see how leaders in highly regulated industries leverage AI to cut manual work and drive ROI.
Top comments (1)
The concept of "One Request Is Never One Call" really stood out to me, as it highlights the often-overlooked complexity of agentic workloads. The fact that typical production agents can land between ten and twenty model calls for a single user request is staggering, and it's no wonder that inference spend can quickly spiral out of control. I've seen similar issues in my own work with LLMs, where a small change in user volume can lead to a massive increase in compute costs. The idea that token prices are falling, but consumption is outrunning the discount, is a particularly interesting insight - it suggests that companies need to focus on optimizing their agentic architectures, rather than just relying on cheaper tokens. How do you think companies can better anticipate and manage these hidden costs, especially when it comes to scaling agentic features?