Most production agents don't fail because the model is dumb. They fail because a chain of mostly-correct steps multiplies into a mostly-wrong outcome, and nobody notices until a customer does. If you want reliable agents, the first thing to fix isn't the prompt. It's the arithmetic of how errors compound.
I run a team that builds AI agents for B2B clients. We've shipped enough of them to have a clear opinion: the single biggest predictor of whether an agent survives contact with real traffic is whether you treated reliability as a system property instead of a model property.
The number that should scare you
Here's the math, and it's illustrative but the shape is real. Say each step in your agent's workflow is 95% reliable. That sounds great. Now chain ten steps:
0.95 ^ 10 = 0.599
A workflow where every individual step works 19 times out of 20 succeeds end-to-end only about 60% of the time. Drop step reliability to 90% and a ten-step task lands at 35%. This is why a demo that nails three happy-path steps falls apart the moment you give it a real, twelve-step job. Industry write-ups on agent failure keep landing on the same conclusion: error propagation, not the variety of failure modes, is what actually kills reliability. One widely cited figure is that fewer than one in eight agent initiatives reach stable production.
The fix isn't a smarter model. A model that's 97% reliable per step still gets eaten alive over a long enough chain. The fix is to (a) make each step verifiable, (b) shorten the chain, and (c) catch errors at the step where they happen instead of three steps downstream.
Silent failure is the real enemy
A crash is a gift. You get a stack trace, an alert, a place to look. The failures that hurt are the quiet ones:
- The agent calls a tool with a slightly wrong parameter, the tool returns something, and the agent treats it as success.
- A retrieval step pulls the wrong document and the model confidently reasons over it.
- The model tells you what you wanted to hear instead of what's true.
- An early misread propagates through every subsequent decision, each one locally reasonable.
None of these throw. The agent finishes, returns a clean-looking answer, and is wrong. Microsoft's red-team taxonomy and most production post-mortems put silent, propagating errors near the top of the list precisely because they're invisible until you go looking.
The defense is verification at the boundaries. After a tool call, check the shape and plausibility of what came back before you let the model build on it. After a retrieval, confirm the chunk actually contains the entity you asked about. These checks are boring. They're also the difference between a 60% workflow and a 95% one.
Three things we do on every agent build
1. Cut the chain. The cheapest reliability win is fewer steps. We constantly ask: can two tool calls become one? Can a deterministic function replace a reasoning step? Every step you delete is a multiplier you remove from the failure equation. We wrote up the broader version of this lesson after running internal workflows as agent "departments" and watching the handoffs break: the agent is the easy part, the orchestration is the hard part.
2. Make every step assert its own success. A tool that creates a record should return the created record's ID and we should verify it exists. A step that claims it parsed an invoice should hand back structured fields we can range-check. If a step can't prove it worked, it didn't.
3. Put a human at the consequential edge. Not on every step. On the irreversible ones. Sending money, emailing a customer, deleting data, committing to prod. An agent can do all the prep and reasoning; a person approves the action that's expensive to undo. This isn't a lack of ambition. It's the same reason we don't auto-merge to main.
The verification gap applies to the code too
This whole argument has a sibling on the software side. AI tools let teams write far more code, but they did nothing to make verifying that code cheaper. We dug into that asymmetry separately, since the QA layer that catches AI-written bugs is the same kind of investment as the verification layer that catches agent failures: AI writes 4x the code, here's the QA layer that stops 4x the bugs. Same principle, different surface. Generation got cheap. Verification didn't.
Key takeaways
- Step reliability multiplies. 95% per step over 10 steps is ~60% end-to-end, not 95%.
- Silent, propagating failures hurt more than crashes because nothing alerts you.
- Shorten the chain, make each step verify its own success, and gate irreversible actions behind a human.
- A better model helps at the margin; reliability is a system property you have to engineer.
FAQ
Does a frontier model make my agent reliable?
It raises per-step reliability, which helps, but the compounding math still applies. A long enough chain of 97%-reliable steps is still unreliable end-to-end. Architecture beats model choice here.
Where do I add verification first?
After tool calls and after retrievals. Those two boundaries are where the most silent errors enter, and they're the cheapest to check.
Is human-in-the-loop a sign the agent isn't good enough?
No. It's a sign you understand which actions are expensive to reverse. Gate those, automate the rest.
If you're building agents and fighting the same compounding-failure problem, I'd genuinely like to compare notes. We've got scar tissue and we're happy to trade it. You can find us at Shanti Infosoft.
Top comments (0)