Your AI copilot demo worked. The model answered every question in the sandbox, latency was fine, and the stakeholders nodded. Then you connected it to production and the answers turned vague, wrong, or quietly incomplete. The reflex is to blame the model — swap one vendor for another, try a fine-tune, wait for the next release. That rarely fixes anything, because the model was probably never the problem.
Enterprise AI rollouts stall on data, not intelligence. Customer information is spread across a CRM, a billing platform, two or three support tools, a warehouse, and a legacy system nobody wants to touch. The model reasons perfectly well over whatever you hand it. It just cannot see a coherent picture of your business, so it answers from fragments.
The model was never the bottleneck
Picture a support copilot fielding a simple question: what is the status of the Acme account? To answer, it needs the subscription tier from billing, open tickets from the help desk, the renewal date from the CRM, and maybe usage data from a product database. Four systems, four different identifiers for the same company. Salesforce calls it account 0014x, Stripe calls it customer cus_J4k2, Zendesk calls it organization 360A. None of them match, and none of them know the others exist.
Hand a frontier model all four records and it will reconcile them fine. The hard part is getting all four to the model in the first place — correctly joined, fresh, and filtered to what this specific user is allowed to see. That is an integration problem and a governance problem. It is not a model-capability problem, and no amount of model shopping makes it go away.
This is why AI agents in the enterprise underperform the demo. The demo ran on one clean dataset. Production runs on your real data estate, and that estate is fragmented. A CIO AI strategy that budgets for model licenses and GPU time but not for the data plumbing is funding the wrong half of the project.
What AI data integration actually involves
"Connect the data" sounds like a weekend of API work. It is not. Here is what the phrase actually covers.
Entity resolution. You need one canonical identity per customer, per product, per contract — deciding that cus_J4k2 in Stripe and account 0014x in Salesforce are the same company, and handling the cases where "Acme Inc", "Acme, Inc.", and "ACME" need to collapse too. Without this layer, every cross-system question returns a partial answer.
Schema and semantic alignment. The field status means an invoice state in billing, a ticket state in support, and a deal stage in the CRM. If your retrieval layer hands all three to the model under the same label, it will conflate them. Someone has to map fields to a shared vocabulary.
Freshness. A copilot answering from a vector index rebuilt nightly will confidently quote yesterday's data. For account questions, stale is the same as wrong. You have to decide which facts need real-time lookups and which can be cached.
Governance and permissions. This is the one teams skip, and the one that causes incidents. The agent must inherit the access rules of the person asking. A support rep's copilot should not surface another customer's revenue — and an agent crawling every system with a single service account will happily cross that line.
The most expensive enterprise AI failure is not a wrong answer — it is a correct answer shown to the wrong person. An agent querying your systems through a broad service account bypasses every row-level permission your applications enforce. Propagate the requesting user's identity and access scope all the way down to the retrieval layer before you let an agent touch production data.
The work to do before you wire up a copilot
You do not need to unify your entire data estate before shipping anything. You need to unify enough of it for one workflow.
Pick a single workflow. "AI across the company" has no definition of done. "A support copilot that answers account-status questions" does. Scope to one job with measurable success.
Build a thin canonical layer. For the entities that workflow touches — customers, subscriptions, tickets — create one resolved view with a stable ID, even if it starts as a single materialized table. You are not building a data warehouse; you are building the smallest join that makes the workflow correct.
Name a system of record per field. Decide that billing owns subscription tier, the CRM owns renewal date, the help desk owns ticket state. When systems disagree, the agent needs a rule, not a guess.
Instrument retrieval. Log every record the agent pulled for every answer. When it is wrong — and early on it will be — you want to see whether it retrieved bad data or reasoned badly over good data. Those are different bugs with different fixes.
A fast diagnostic: take five answers your copilot got wrong and trace each one. If most failed because the right record was missing, unreachable, or stale, your problem is data integration. If most failed because the model had the right records and still reasoned poorly, then — and only then — is model choice worth revisiting.
Structured systems are only half the fragmentation. The other half is institutional knowledge — runbooks, policies, past decisions, onboarding docs — scattered across old wikis, shared drives, and chat threads. An internal copilot retrieving from five half-maintained wikis produces five half-right answers. Consolidating that knowledge into one searchable, permissioned workspace is unglamorous, and it is among the highest-leverage things you can do for retrieval quality.
Enterprise AI adoption is gated by data fragmentation, not model quality. The teams shipping working copilots did not win on model choice — they won by doing the entity resolution, schema alignment, and permission propagation that the demo let them skip. Do that work on one workflow, prove it, then widen. The model will be ready when you are.
Originally published at pickuma.com. Subscribe to the RSS or follow @pickuma.bsky.social for new reviews.
Top comments (0)