Why production agents fail on information that never made it into any system
An agent can query the right table and return the right number for every account except one, because the exception on that account lives in an analyst’s memory, and no table records it.
Teams debugging this kind of failure usually start with the model. They tune prompts or switch providers, and the error rate barely moves, because the model was never missing intelligence. It was missing the fact.
In my work as a fractional CTO, this is the most common root cause I find when enterprise agents produce wrong outputs in production. The information required to do the job correctly exists. It exists in undocumented form, distributed across the memories and judgment calls of the people who have been doing the work.
Where the Knowledge Actually Lives
Every company runs on a layer of knowledge no system captures. The business rule each new hire learns from their manager in week three. The understanding that two product names in two different databases refer to the same SKU, reconciled by whoever pulls the report. The approval that officially requires three signatures but in practice needs one phone call.
Humans handle this layer so fluidly that most organizations forget it exists. The analyst who knows the pricing exception applies it without thinking. An agent querying the same systems has no idea the exception exists and no way to discover it, because the reconciliation happens in someone’s head.
Four Context Failure Modes
Across the agent deployments I’ve audited, four failure modes account for most wrong outputs in production.
Definitions stored as institutional memory
Finance counts a customer as active if they’ve paid in the last 90 days. Sales counts anyone with an open opportunity. Both definitions are correct inside their own department, and the difference lives in a shared understanding that never made it into a schema. An agent asked for the churn rate picks one of them and produces a number half the company will dispute.
Entity identity that resolves differently across systems
The ERP calls it PRO-ENT-2. The billing platform calls it Enterprise Pro v2. Everyone who works with both systems knows they’re the same product, and no table anywhere records that fact. Agents join on what the data says, so revenue for one product splits into two.
Process knowledge that was never written down
The documented workflow says contracts route through legal. The actual workflow routes anything under $50K through a template the paralegal maintains, a shortcut established years ago and passed along verbally. An agent following the documented process produces work that is technically correct and operationally useless.
Outputs that cannot be traced to a source
When a senior analyst approves an exception, the reasoning lives in her judgment. Nothing records why. When an agent later makes a similar call, nobody can audit the chain, because the precedent it needed was never a record. Provenance breaks the moment a decision depends on knowledge with no source of truth.
The Gap Between Having Data and Using It
The industry numbers describe the same distance. DataHub’s State of Context Management Report 2026, which surveyed 250 IT and data leaders, found that 88% claim to have fully operational context platforms, while 61% frequently delay AI initiatives due to a lack of trusted data.
Read those two numbers together and the picture sharpens. Nearly nine in ten organizations believe they have the infrastructure. Six in ten keep delaying launches on top of it. The distance between “we have the data” and “AI can use the data” is where most enterprise AI budgets currently leak, and it rarely shows up as a line item because nobody owns it.
A Different Kind of Investment
Investors have started pricing this gap. Tribal, founded by Salesforce, Wix, and Spot.io veterans, raised a $10 million seed round in May to build agents on a metadata fabric that maps a system of record’s business rules and dependencies before any agent acts. The founding team’s read of the market matches what I see inside companies. Building AI prototypes is easy. Shipping trusted change inside a live enterprise system is brutally hard, and the difficulty concentrates in the context layer underneath the model.
Most AI roadmaps are built to buy tools and integrate APIs. Closing the context gap looks different. It means interviewing the analyst who carries the pricing exceptions and encoding what she knows. It means building the entity resolution table that finally reconciles PRO-ENT-2 with Enterprise Pro v2. Someone has to write down the actual approval workflow, shortcuts included, and decide which ones the agent is allowed to follow. This is slow, unglamorous work that involves calendars more than compute.
Start With What Would Break
A practical first step I give clients requires no vendor. For each workflow an agent will touch, ask what would break if the most tenured person on that team left tomorrow. The answers form a map of your undocumented knowledge, and that map predicts where your agents will fail before you deploy them.
Then fund the extraction work explicitly. Give it an owner and a deadline, the same way you would any infrastructure project, because that is what it is. Encode the definitions, resolve the entities, and record the reasoning behind exceptions so outputs can be traced.
Your org chart is part of your data architecture. Every retirement and every reorg deletes records no backup will recover. Agents make that loss visible because they fail where humans compensated. The undocumented layer was always a liability. Agents just turned it into an error rate you can measure.
…
Nick Talwar is a CTO, ex-Microsoft, and a hands-on AI engineer who supports executives in navigating AI adoption. He shares insights on AI-first strategies to drive bottom-line impact.
→ Follow him on LinkedIn to catch his latest thoughts.
→ Subscribe to his free Substack for in-depth articles delivered straight to your inbox.
→ Watch the live session to see how leaders in highly regulated industries leverage AI to cut manual work and drive ROI.
Top comments (0)