I build AI agents and automations for companies in Italy, so I read every "agents are eating the world" thread with one number in the back of my head: most agent projects do not pay off. MIT's 2025 study found that around 95% of generative AI pilots produced no measurable return to the P&L. A separate analysis of 847 agent deployments found 76% in crisis within 90 days.
The interesting part is not the failure rate. It is the cause. After shipping a few of these, my honest read is that the model is almost never the reason.
The real failure is upstream of the AI
When an agent project dies, the post mortem usually blames the wrong layer: the prompt, the framework, the tool calls, the context window. Those are real problems, and Berkeley's MAST study even catalogs 14 distinct failure modes in multi-agent systems. But they are symptoms.
The root cause is almost always that the company pointed an agent at a process that was never standardized. The work lived in people's heads, the rules had three exceptions nobody wrote down, the data sat in four places that disagreed. An agent does not fix that. It amplifies it. You automate the mess and now the mess runs at scale.
An agent does not reduce disorder. It scales whatever process you point it at, including the disorder.
There is data under this too. One analysis found projects with a clearly defined problem succeeded about 58% of the time, versus 22% for projects that started from a vague mandate like "add AI." And the reason the standard never gets written is structural: research out of Berkeley estimates roughly 80% of operational know-how is tacit, undocumented, carried by the people who do the work. That is the part an agent cannot infer.
The method: AI is the last third, not the first
The mental shift that changed my hit rate was to stop treating the agent as the project. The agent is the last third of the work. Here is the shape I now follow, roughly six months end to end for a real workflow.
Months 1-2: find the workflow and measure the baseline. Not the process that annoys you most, the one with high business impact that is still repeatable enough to standardize. Often it is the bottleneck, the step that throttles the whole flow, not the loudest complaint. Measure where it is today before you touch it, or you will never be able to prove the agent did anything.
Months 3-4: standardize. Write the procedure, make the decision criteria explicit, collapse the four conflicting data sources into one source of truth. This is the unglamorous part everyone skips, and it is where the 5% is won or lost. Standardizing does not mean writing a manual. It means making the process repeatable and measurable.
Months 5-6: build the agent and put it in test. Now the AI work begins: planning, tools, memory, evals, guardrails. By this point the agent has something solid to stand on, so the build is faster and the failures are boring instead of mysterious.
If that timeline feels slow, that is the point. The teams in the 95% compressed it to "weeks 1-2: install agent." The teams in the 5% spent most of the calendar before the model ever ran.
Where memory fits
There is a quieter reason these projects rot: the system has no memory. Every new session starts from zero, the decisions you made last month evaporate, and a human becomes the institution's memory by re-explaining the same context forever.
So before the production agent, I give the project itself a memory: the standard, the decisions, the open questions live in plain files the system reads and rewrites, not in a chat you hope retained something you cannot inspect. It is less "teach the model to remember" and more "give the project a memory you can open, correct, and trust." I open-sourced the small kit I use for this for Claude Cowork, called cowork-os, if you want to see the structure.
When the honest answer is "not yet"
The most useful thing I tell a prospect is sometimes "do not automate this yet." Low volume, an unstable process, a goal that is really about looking modern, a process that is broken upstream: in all of those, an agent is the wrong first move. Fix or drop the process first. Saying that out loud has won me more trust than any demo.
None of this is anti-AI. It is the opposite. The agent is genuinely the easy part once the process under it is real. The 5% is not a modeling secret, it is a sequencing discipline.
If you have shipped an agent that stuck, I am curious: how much of your timeline went to the process versus the model, and would you do that split differently next time?
Sources
- MIT, "The GenAI Divide: State of AI in Business 2025" (about 95% of pilots with no P&L return), via Fortune, 2025.
- Analysis of 847 AI agent deployments: 76% in crisis within 90 days, Medium, 2026.
- "Why Agentic AI Projects Fail" (defined problem 58% vs vague mandate 22%; about 14% reach production), Ampcome, 2026.
- "Tacit Knowledge Is Your Next Competitive Moat" (about 80% of operational know-how undocumented), California Management Review (Berkeley), 2026.
- MAST, "Why Do Multi-Agent LLM Systems Fail?" (14 failure modes), UC Berkeley, arXiv:2503.13657.
- "Agentic workflow architecture: planning, tools, memory, evals, guardrails," Vellum, 2026.
Top comments (0)