In 2026, the most common mistake I see teams make when budgeting AI agent infrastructure is measuring cost at the wrong level of granularity. You look at your dashboard, see a reasonable per-session average, and conclude the system is running efficiently. It isn't. The number is real; the picture it paints is not.
According to McKinsey's analysis of AI agent total cost of ownership (McKinsey, 2025), organizations routinely underestimate AI agent expenses by 2 to 3 times when they fail to account for hidden operational expenses including infrastructure, monitoring, and tool integration overhead. That gap isn't a rounding error. It's a structural blind spot baked into how most teams instrument their pipelines.
The blind spot has a specific cause: MCP tool calls.
The Scenario Every Agent Builder Recognizes
Picture a support automation pipeline running in production. The per-session average looks fine. Your finance team is happy. Then someone on the engineering side pulls a week of raw logs and starts breaking down spend by individual operation rather than by session.
What they find is uncomfortable. A handful of retrieval operations are consuming a disproportionate share of total token budget. Not because the reasoning model is expensive, but because each external lookup injects a large payload into the context window before the model ever sees the query. The session average looked acceptable because cheap operations outnumbered expensive ones. The expensive ones were invisible in aggregate.
I ran into exactly this pattern building the Autonomous SDR pipeline. Our initial estimate was based on prompt tokens alone: roughly half what we actually measured once the system ran in production. The Researcher component cost more than the Judge, which was counterintuitive until we traced why. Anthropic's web_search tool injects 30,000 to 40,000 tokens of web content into the context window per invocation. Our estimate was based on prompt tokens alone. Measured cost per lead came in at nearly double that estimate. That's why we now publish ITP-measured figures rather than theoretical projections. The gap between estimate and reality is consistently around 2x on any pipeline that uses web-search-enabled operations.
This isn't an edge case. It's the default behavior of any agent that calls external tools without per-operation instrumentation.
Why MCP Tool Calls Compound Silently
The Model Context Protocol standardizes how agents interact with external systems: databases, search APIs, file systems, calendar integrations. Each discrete interaction is a "tool call." The problem is that most observability setups track token consumption at the session boundary, not at the individual operation boundary.
Consider what happens inside a single agent run. The reasoning layer receives a user query. It decides to retrieve context from a knowledge base. That retrieval returns several thousand tokens of text. The model processes those tokens, generates a response, then decides it needs to verify a fact via a web lookup. The lookup returns another large payload. By the time the session closes, the token count reflects all of that, but your dashboard shows one number: total session spend.
The compounding happens because agents are non-linear. A single session might trigger two retrieval operations, one web search, and a database write. Or it might trigger twelve, depending on how the reasoning layer interprets the query. Session-level averaging smooths over that variance entirely. You see the mean; you miss the distribution.
Teams building on n8n or similar orchestration platforms often have the raw data available in their execution logs. The issue isn't data availability. It's that nobody has wired up the per-operation accounting layer. Each node execution records its own metadata, but aggregating that into a cost breakdown by operation type requires deliberate instrumentation that most teams skip during initial deployment.
This connects directly to a broader pattern we've written about in why enterprise AI deployments fail at the operational layer: the build phase optimizes for functionality, and the measurement infrastructure gets deferred until costs become a crisis.
A Debugging Methodology That Actually Works
The fix is not complicated, but it requires changing what you measure before you change anything else. Here's the sequence we use.
Step 1: Instrument at the operation boundary, not the session boundary. Every external interaction, whether it's a database query, a search API hit, or a file read, should emit its own telemetry event. That event should include: operation type, input token count, output token count, latency, and a session identifier for joining back to the parent run. If you're running on n8n, this means adding a logging node after each tool-type node in your pipeline, not just at the workflow's terminal step.
Step 2: Build a cost breakdown by operation type. Once you have per-operation data, group it. You're looking for the operations that appear frequently and carry high token payloads. Web search and document retrieval are the usual suspects. Classification operations are typically cheap. Reasoning steps sit somewhere in the middle depending on context length.
Step 3: Set per-operation budget thresholds. After you know what each operation type actually costs in production, you can set circuit breakers. If a retrieval operation returns more than a defined token count, truncate before passing to the reasoning layer. If a session has already consumed more than a defined budget, route to a cheaper fallback path rather than continuing to invoke expensive operations.
Step 4: Re-measure after each change. This sounds obvious, but teams frequently make multiple changes simultaneously and then can't attribute the improvement. Change one thing, measure for a week, then change the next thing. The variance in agent behavior means you need enough runs to get a stable estimate.
One honest limitation of this approach: it adds instrumentation overhead. Every additional logging node is a small latency cost and a small maintenance burden. For pipelines running at low volume, the engineering time to implement granular tracking may exceed the savings for months. This methodology pays off fastest on high-frequency pipelines where even a modest reduction in per-operation spend compounds quickly across thousands of daily runs.
There's also a subtler tradeoff. Truncating retrieval payloads to control spend can degrade answer quality. You're trading accuracy for economy. The right threshold depends on your use case, and finding it requires running quality evaluations alongside cost evaluations, not just watching the spend number drop. We use a quality standard we call BQS (our Benchmark Quality Standard) to make sure cost optimizations don't silently erode output quality in the pipelines we build.
What the Numbers Actually Tell You
Once you have per-operation data, the picture changes. You stop asking "is our agent cheap?" and start asking "which operations are worth their cost?"
Some expensive operations are worth every token. A web search that retrieves current pricing data for a sales pipeline is doing real work that a cached response couldn't replicate. A document retrieval that pulls the exact contract clause a support agent needs is justified. The goal isn't to eliminate expensive operations. It's to eliminate expensive operations that don't improve output quality.
The operations that consistently fail that test are redundant retrievals (the agent fetches the same context twice because it doesn't carry state between steps), over-broad searches (the query is too general and returns a large payload when a narrow query would have returned a small one), and unfiltered API responses (the external system returns everything and the agent processes everything, when a filter parameter would have returned only what's needed).
Each of these is fixable at the pipeline level without touching the reasoning model at all. Redundant retrievals disappear when you add a simple cache keyed on the query. Over-broad searches improve when you add a query-refinement step before the lookup. Unfiltered responses shrink when you pass the right parameters to the external API.
The McKinsey finding that organizations underestimate AI agent expenses by 2 to 3 times (McKinsey, 2025) maps directly to this pattern. The underestimation isn't happening at the model pricing level. It's happening because teams aren't counting the full token footprint of what the model actually processes, which includes every byte injected by every external operation in the chain.
If you're building multi-agent systems, what ForgeWorkflows calls agentic logic, the problem compounds further. Each component in the chain may be individually instrumented, but the handoffs between components often carry context payloads that nobody is accounting for. The output of one reasoning step becomes the input of the next, and if that output is verbose, the downstream components pay for it in token spend.
You can browse the full range of automation pipeline architectures we've built and documented at the ForgeWorkflows blueprint catalog to see how we structure cost-aware pipelines across different use cases.
What We'd Do Differently
We'd instrument before we deploy, not after costs become visible. Every pipeline we've built has taught us that retrofitting observability is harder than building it in from the start. The temptation is to ship the functionality first and add monitoring later. In practice, "later" arrives when a cost spike forces the issue, and by then you're debugging a production system under pressure instead of a staging environment at your own pace. The right time to add per-operation logging is during the initial build, not after the first billing cycle surprises you.
We'd treat retrieval payload size as a first-class design constraint, not an afterthought. When we scoped the Autonomous SDR, we didn't initially model the token footprint of web search results as a variable. We modeled the reasoning steps. That was the wrong starting point for a web-search-heavy pipeline. For any system where external data retrieval is a core operation, the payload size of that retrieval should be estimated and bounded before the architecture is finalized, not discovered in production.
We'd run quality evaluations in parallel with cost optimizations from day one. The risk of aggressive per-operation budget caps is that you optimize your way into a cheaper but less accurate system without noticing. Separating cost measurement from quality measurement means you can catch that regression early. Running both in parallel from the start, rather than treating quality as a separate phase, would have saved us at least one rebuild cycle on a pipeline where we cut spend and quietly cut accuracy at the same time.
Top comments (0)