Your finance team built an agent that helps close the books. It connects to the ERP, reads journal entries, and drafts reconciliations. In the demo, everything worked beautifully.
Then came the real month-end close. The agent misread invoice statuses. It recommended wrong accounts. It escalated exceptions that had already been resolved. Your team spent the weekend rechecking everything from scratch.
What went wrong? Not the model. Not the agent framework. The problem was the data.
Most companies obsess over which model to use, which agent platform to adopt, or how to orchestrate workflows. But in an enterprise context, models are increasingly interchangeable. What cannot be bought or copied is your company's context: how you define a "customer," how your approval chains work, what counts as a policy exception, and how your business entities relate to each other.
Without a strong data foundation, your agent will sound confident and be wrong. It will make recommendations that look reasonable but violate your actual business rules. This isn't model hallucination — it's something far more dangerous for operations.
The three layers that separate a demo agent from a production agent: data foundation, execution layer, and governance runtime.
The Real Cost of Operational Hallucination
We talk a lot about AI hallucination — models making things up. In enterprise settings, a more insidious problem emerges: operational hallucination. The agent's output sounds credible, but it's wrong against your actual business reality.
Your finance agent says an invoice is unpaid — but the ERP status already changed. Your HR agent quotes a leave policy from a document that was superseded six months ago. Your supply chain agent reroutes a shipment without understanding actual inventory constraints.
The problem isn't just accuracy. The problem is that agents start influencing actions, priorities, and decisions. Every wrong answer creates rework, delays, or compliance risk.
This is why the gap between a successful pilot and a failed production rollout is almost never about conversation quality. It's about data readiness.
Structured Data: The Operational Backbone
If your agent needs to act in enterprise systems — check status, validate conditions, trigger workflows — it depends on structured data. Customer records, orders, invoices, supplier master data, employee profiles, contracts, tickets.
But having an ERP or CRM doesn't mean your data is ready for agents. Structured data needs six characteristics to be useful:
Consistent business definitions. What does "active customer" mean? When is an order "fulfilled"? If definitions vary across functions or countries, your agent will make inconsistent decisions.
Clear ownership. Every data domain needs a business owner, not just a technical administrator. Without ownership, data quality problems get labeled as "system issues" while your agent keeps failing.
Traceable lineage. Your agent needs to know where data came from. If a dashboard field comes from layered transformations, can you be sure the agent is reading current business state?
Monitored quality. Completeness, uniqueness, consistency, timeliness — these can't be assumed. Duplicate vendor masters or outdated org charts will break agent workflows.
Strong semantics. Data needs meaning that travels across systems. This is where enterprise data models and master data management become critical.
Secure access. Agents shouldn't read core tables directly. They need interfaces that enforce permissions, maintain audit trails, and provide stable schemas.
Unstructured Data: Where Context Actually Lives
Many organizations discover the value of unstructured data only when they start building agents. Policies, contracts, emails, call transcripts, SOPs, knowledge articles — these were passive archives. In agentic AI, they become active context layers.
Your customer ticket status lives in CRM, but the real context — what the customer was promised, the emotional tone, the root cause — lives in transcripts and chat history. Your supplier master data is clean, but commercial terms and contract exceptions live in PDFs. Your employee data is in HRIS, but local policies and FAQ exceptions live in portals and emails.
Unstructured data requires a disciplined pipeline, not just "upload documents to a vector store." You need controlled ingestion from authoritative sources, classification to separate policies from drafts, intelligent chunking with metadata, retrieval that respects permissions and context, and lifecycle management so expired documents don't stay active.
The temptation is to dump everything in. Resist it. Start with high-value, authoritative corpora: official SOPs, active contracts, verified knowledge articles, curated policy documents. Not every file your company has ever created.
Governance Must Move from Policy Documents to Runtime
Traditional data governance stops at documents, committees, and manual controls. For agentic AI, governance must execute at runtime.
The question shifts from "who can access this data?" to "who can access this data through an agent, for what purpose, in which workflow, with what level of autonomy, and does this access result in insight or action?"
Permissions must be checked at retrieval time, not after the answer is generated. Your HR agent shouldn't pull compensation data for unauthorized users. Your procurement agent shouldn't expose strategic contracts to casual requesters. Your finance agent shouldn't display entity data outside a user's scope.
Audit trails must explain not just that access occurred, but what data was retrieved, from which source, under what permission, in which workflow, and how it influenced the agent's decision. When an agent gives a bad recommendation, you need to trace whether the problem was data quality, wrong retrieval, missing metadata, or unenforced policy.
Before You Scale, Ask These Questions
The difference between a pilot and production is data readiness. Before expanding your agentic AI footprint, check whether:
- Your priority structured data domains have consistent business definitions
- Customer, supplier, employee, and invoice data have clear owners
- Data quality is monitored for completeness, consistency, and timeliness
- Agents access structured data through interfaces that enforce permissions
- Your unstructured data corpus is curated and distinguished from drafts
- Metadata like version, effective date, region, and classification exists
- Retrieval respects permissions consistent with source systems
- Retention policies exist for documents, transcripts, and interaction history
- You can trace what data an agent used to make a recommendation
Watch for warning signs: "We'll clean data later." Core master data still debated between functions. Agents pulling answers from documents with unclear authority. Service accounts with overly broad access. No version metadata on policies. Retrieval that ignores user permissions.
These aren't technical debt. They are scaling blockers.
What This Means in Practice
For engineering leaders and platform teams, this translates to concrete architectural decisions. Your agent framework should not directly query production databases. Instead, build a data access layer that exposes curated views with enforced permissions. Use metadata registries to tag documents with version, effective date, and region. Implement retrieval-time access control that checks user scopes before returning context. And design observability that logs every data touchpoint — not just model calls.
Your data engineers should treat agent readiness as a first-class requirement, alongside reporting and analytics. Your governance team should define runtime policies, not just static documents. And your product owners should validate agent behavior against real business state, not demo data.
Closing Thoughts
The most honest question you can ask before building more agents is not "which model?" It's "which data is our source of truth, who owns it, and how do we ensure our agent only acts on what's real?"
For a deeper dive into the architecture and governance patterns discussed here, see the original article on data foundations for agentic AI.
Top comments (0)