There is a version of the enterprise AI story told in board meetings, and a version told in weekly standups, and the uncomfortable truth is that both are accurate.
In the board meeting, the chart goes up and to the right. Adoption is up, usage is up, time-to-first-output is down. Agents are running, employees are experimenting, and the company appears to have crossed from AI aspiration into Agentic AI execution. Nobody is lying. The chart is real.
In the standup, sales says the AI-generated account briefs are useful, after someone verifies the facts. Support says the agent drafts good responses, except the policy-sensitive ones, which is to say the ones that matter. Engineering says coding agents accelerate scaffolding, and that senior engineers just lost most of a sprint untangling an AI-generated migration that passed review and failed in staging. Operations says the workflow agent handles the happy path, and that when it doesn’t, someone spends two days reconstructing what the agent actually did, which systems it touched, what data it relied on, why it made the call it made, because nothing was built to replay it.
The temptation is to decide one group is wrong: the executives are high on their own supply, or the operators are foot-dragging. Neither. They are looking at different layers of the same system. Executives see the application layer. Operators live in the integration layer. And only one of those layers makes it onto the dashboard.
That is the abstraction error at the center of most enterprise AI programs, and it is worth being precise about, because the companies that fix it first are going to be very hard to catch.
Production Is Not Absorption
Dashboards measure what is easy to instrument: users, prompts, drafts, summaries, agent runs completed, time saved to first output. None of these numbers is fake. All of them measure the same thing, AI production, and production was never in doubt.
Producing more output faster is the entire point of the technology. Celebrating it is like celebrating that the printing press produces pages.
The enterprise question is absorption: can the organization convert that output into trusted work at lower total cost and risk? Because a draft is not a closed deal, a generated pull request is not shipped software, and an answer is not trust. Every one of those gaps gets closed by humans, and the dashboard is silent about all of it.
The model is fast. The company may not be. A dashboard that measures only the model will systematically overstate the company.
The AI and Agent Cleanup Tax
The missing line item deserves a name: the AI cleanup tax, the human effort required to turn AI output into something the business can actually stand behind.
It would be convenient if this tax were small. It is not. The mistake is imagining it as light editing, a fact-check here, a rewrite there. In practice, the expensive version looks like this:
A coding agent opens a pull request that compiles, passes tests, yet misunderstands a core architectural assumption. A senior engineer, your scarcest resource, spends a day and a half finding that out, because the failure isn’t in any single line; it’s in the intent.
An agent updates records across the CRM and a downstream billing system. Three weeks later the numbers don’t reconcile. Now someone is doing forensic archaeology across two systems of record, with no execution trace, trying to determine which of four hundred automated writes was wrong and whether the error propagated.
A support agent gives a customer an answer that touches a regulatory boundary. Compliance asks the only questions compliance ever asks: where did this come from, what policy did it rely on, and can you show me? If the honest answer is “we can’t reconstruct it,” the cleanup tax on that single output is measured in days of legal and engineering time, and in the awkward decision to pull the agent back from anything that matters.
This is the crucial point: the tax doesn’t hide because it’s small. It hides because it’s misattributed. It shows up as debugging, as review, as reconciliation, as “just being careful”, as ordinary work done by your most senior people. It never shows up as a cost of the AI program, so the AI program books the time savings and someone else’s budget absorbs the verification. The board sees a 40% reduction in time-to-draft; the workflow sees a 5% improvement in cycle time. The gap between those two numbers is the tax, and today almost nobody is measuring it.
Trust Is the Expensive Part
Why does this pattern repeat everywhere? Because AI is unusually good at making the visible part of knowledge work cheap. Drafting, summarizing, classifying, generating, the demo-friendly layer. But that was never the expensive part of enterprise work. The expensive part is trust: can I send this to a customer, ship this code, make this decision, and defend it in front of compliance, legal, procurement, or the board?
Trust is not a prompt problem. It is a system problem. It requires context, governed data access, policy gates, evaluation against real cases, observability, replayable execution traces, human review at the right points, and clear ownership. The value of enterprise AI does not come from the model; it comes from the operating system around the model, and that operating system is precisely what the first-generation dashboard leaves out.
Agents raise the stakes on all of this, because an agent is not an interface, it is a workflow participant. It doesn’t just answer; it acts, retrieves, decides, routes, updates, escalates, commits changes into systems of record. Which means the cleanup tax escalates with capability: when an agent drafts, the tax is editorial; when it writes to systems, the tax is operational; when it touches clinical, financial, or legal workflows, the tax is risk.
“Agent runs completed” is therefore a dangerously shallow metric. Completion is not correctness, correctness is not trust, and the right question is not did the agent run but did it complete the workflow correctly, with the right context, under the right controls, at lower total cost than the process it replaced?
Cost Per Trusted Output
The board does not need less AI measurement. It needs a dashboard built around four questions: What did the system produce? What did humans have to do before it became usable? What happened downstream? And can we prove it?
The second question is the one nearly everyone skips, which is why model output and business outcome refuse to correlate on so many dashboards. The bridge between them is rework, and rework is measurable: edit distance between raw and approved output, regeneration rates, rejection rates, review latency, exception and escalation rates, policy-block events, manual handoffs, downstream acceptance. If 600 of 1,000 generated support responses need material edits, that is not an anecdote. If reps trust 40 of 200 AI account plans enough to use, that is not a vibe. If a coding agent opens 50 pull requests and senior review time doesn’t fall, that is a system telling you exactly where it is broken.
All of it rolls up into one economic unit. Not cost per token. Not cost per prompt. Not cost per draft.
Cost per trusted output.
That is the number that connects the AI program to the P&L, the number that survives contact with a skeptical CFO, and the number that, once you start driving it down, turns AI from a line item into a compounding advantage.
The Roadmap Writes Itself
Here is the payoff for doing the harder measurement: once the cleanup tax is visible, it stops being demoralizing and becomes a diagnostic. Every failure mode points at a specific fix.
Output needs too much correction, the task boundary is too broad; narrow it. The system lacks context, fix retrieval, data access, and memory. Reviewers don’t trust it, build evals against real production examples, not synthetic demos. Exceptions take days to trace, you’re missing observability and replay, so build the trace before you scale the agent. People are shuttling data between systems by hand, finish the integration. Everything requires senior sign-off, either the use case exceeds your current maturity or your controls are underbuilt, and now you know which.
This is also why “AI strategy” fails when it arrives as a use-case inventory. A list of applications is not a strategy; it is a backlog. The strategic work is sequencing: which workflows are valuable enough, bounded enough, instrumented enough, and safe enough to absorb AI output without drowning the organization in verification? That is the difference between a demo roadmap and a production roadmap, and the market will eventually price the difference.
Down the Stack
The first wave of enterprise AI abstraction happened at the interface, type into a box, get useful output, and everyone got it roughly for free. The next wave happens lower in the stack, and it will not be evenly distributed: shared context, governed data access, workflow-specific agents, eval harnesses, policy gates, approval records, cost telemetry, execution traces. The unglamorous machinery that lets a company know not just what the AI produced, but how that output moved through the business and what it cost to trust it.
This work photographs badly. It will never demo like the chatbot did. It is also where the durable value is, precisely because it is hard to copy. Prompt volume is a commodity; anyone can buy tokens. An organization that has instrumented its cleanup tax, driven down its cost per trusted output, and learned to convert every unit of rework into a system improvement has built a capability, and capabilities compound.
The board should still get a number. It should just be the right one: how much trusted work did the AI or Agent system help complete, at what total cost, with what risk, with what evidence?
That number is harder to produce than an adoption chart. It is also the only one worth funding against, and unlike the next model release, it is entirely within your control. The tax is real, it is large, and right now it is invisible. The first company in your market to see it clearly wins.
So, go make it visible.
…
Nick Talwar is a CTO, ex-Microsoft, and a hands-on AI engineer who supports executives in navigating AI adoption. He shares insights on AI-first strategies to drive bottom-line impact.
→ Follow him on LinkedIn to catch his latest thoughts.
→ Subscribe to his free Substack for in-depth articles delivered straight to your inbox.
→ Watch the live session to see how leaders in highly regulated industries leverage AI to cut manual work and drive ROI.
Top comments (0)