The boardroom goes quiet in a particular way when the AI number comes up. I have sat in that quiet. A leadership team has spent a year and a serious budget on generative AI, the demos were dazzling, the pilot won an internal award, and now someone asks the only question that matters: what did it do to the numbers? And the answer, stated plainly, is nothing you can point to. That silence is the real story of enterprise AI right now, and it is worth understanding before you spend another dollar.
MIT's NANDA report, "The State of AI in Business 2025," released in August, is the document everyone has been quoting for exactly this reason. Its headline finding, as widely repeated, is that roughly 95 percent of enterprise generative-AI pilots produced no measurable profit-and-loss impact. I want to be careful here, because that number has been challenged on methodology and it deserves a hedge rather than a slogan: treat it as a widely-repeated claim about a widely-shared experience, not a settled law of nature. But you do not need the decimal point to be exact. Anyone who has run these programs recognizes the shape of it. The pilots work. The businesses do not move.
Here is what I have learned from putting agents and models near real production systems: when a program fails, the model is almost never the reason. The reason is organizational. It is the wrong build-versus-buy call, work that never got integrated into an actual workflow, and success measured in vibes instead of outcomes. Those are leadership failures, which is the good news, because leadership failures are the kind you can fix.
Why the programs actually fail
Consider the same NANDA report's more useful, buried finding, the one that did not make the headlines. Buying or partnering for AI tools succeeded far more often than building them internally, by roughly two to one. Sit with that. The failures cluster around the instinct that says a serious company builds its own. That instinct made sense when software was the differentiator. With foundation models, the base capability is a commodity you rent, and the differentiator is what you wrap around it.
The second failure is integration, or the absence of it. An AI feature that lives in a separate tab, that a person has to remember to go and use, that does not sit inside the flow of the work, will be abandoned by March. I have watched genuinely good models die this way. The model was fine. It was never in the workflow, so it was never in the habit, so it was never in the results.
The third failure is measurement, and it is the subtlest. Two 2025 studies should be taped to the wall of every AI program. A METR randomized study from mid-year found that experienced developers felt about 20 percent faster when using AI, while actually measuring about 19 percent slower. Read that twice. Felt productivity and measured productivity pointed in opposite directions. Then there is "workslop," a term from Harvard Business Review with BetterUp and Stanford, describing AI output that looks finished but quietly pushes the real work downstream onto whoever receives it. Around 40 percent of workers reported getting it, reportedly costing the receiver close to two hours per instance. If you measure output volume, workslop looks like a triumph. If you measure outcomes, it is a tax you are paying yourself.
The playbook
So here is the decision framework I would defend in front of any board.
Buy the commodity, build only the genuine differentiator. The base model, the vector store, the orchestration plumbing: these are commodities, and the NANDA two-to-one finding says the market agrees. Reserve your build budget for the thin layer that is actually yours, the proprietary data, the domain judgment, the workflow that no vendor understands the way you do. A good rule I hold to: if a competitor could buy the same thing next week, you should have bought it too, and spent the saved effort on the part they cannot.
Decide own-versus-rent per workload, not by ideology. In January 2025, a low-cost, open-weight model from the Chinese lab DeepSeek helped trigger a single-day fall in Nvidia's stock of roughly 589 billion dollars. Whatever you think of the specifics, the signal was unmistakable: the assumption that frontier capability only comes from a handful of expensive, closed providers is no longer safe. That reframes own-versus-rent and open-versus-closed for everyone. The mature answer is not a doctrine, it is a portfolio. Some workloads want a hosted frontier model. Some want an open-weight model you run yourself for control or cost. The decision is made workload by workload, and it is revisited, because the ground moves.
Make unit economics a first-class product decision. The 2025 developer pricing revolts, the anger at usage caps and re-priced plans on popular AI coding tools, were not just a community tantrum. They were the market discovering that flat-rate "unlimited" AI collides with real per-token cost. Every token has a price, and any product priced as if it did not is quietly insolvent. In the systems I have led, cost per successful outcome is a design constraint from day one, not a finance problem discovered in the quarterly review.
Integrate into the workflow or expect nothing. This is not a nice-to-have, it is the whole game. The winning move is to put the AI inside the tool the person already opens, at the exact moment of friction, so using it is the path of least resistance rather than an act of discipline.
Instrument outcomes, and hunt for the illusions. Given METR, never trust felt speed; measure the cycle time and the quality of the finished result. Given workslop, measure whether work actually got completed downstream, not how much output got generated upstream. The organizations that win are the ones willing to look for their own workslop and count it as a cost.
What a good leader does differently
The pattern underneath all of this is a stance. The losing leader optimizes for the flashiest pilot, the demo that gets applause, the announcement. The winning leader optimizes for landing. They ask the boring questions early: whose workflow does this enter, what outcome moves, what does a successful call cost, and how will we know it is real rather than felt. They buy without ego and build with discipline. They treat measurement as a source of truth rather than a source of good news. None of that is glamorous, and that is rather the point. The 5 percent who win, if the number is even roughly right, are not the ones who ran the most impressive experiment. They are the ones who made an ordinary thing actually work inside a real business.
I have come to think that the AI leadership skill of this decade is not model selection or prompt craft. It is judgment about where value actually accrues and the discipline to instrument it faithfully. The technology will keep getting cheaper and stranger. The DeepSeek moment will not be the last time the ground shifts under a decision you thought was settled. What holds steady is the operating question: did this land, in the numbers, for a real person doing real work.
Where this goes next
The comfortable era of "we are running some pilots" is closing. The next phase belongs to leaders who can say precisely what they bought, what they built, what each call costs, and what changed in the business as a result. That is a higher bar, and it is the right one.
I would genuinely like to compare notes with peers running these programs, because the reported failure rate suggests most of us are still learning this in public. If you have found a build-versus-buy call that surprised you, or a way to catch workslop before it reaches the customer, I would rather hear it than guess at it. The winners in this next chapter will be the ones who kept comparing notes instead of comparing demos.
I write a field-note series on shipping AI that actually works in production. The original of this piece, and the rest, live on my LinkedIn. Originally published on LinkedIn.
Top comments (0)