Enterprise data teams were trained on a single story: centralize first, analyze second. We pull CRM rows into a warehouse. Snapshot finance into a lake. Export documents into a search index. Chunk everything for embeddings. Paste the top fifty chunks into every agent prompt.
That story made sense when the hard problem was getting data into one place so humans could run reports. It made less sense the moment agents started asking questions every few seconds — across CRM, payroll, tickets, contracts, and APIs that were never meant to be duplicated.
Moving data was never free. In the AI age, it became the most expensive mistake on the roadmap.
What the AI age changed
Freshness became a product requirement
A nightly ETL job was fine for Monday’s dashboard. It is not fine when a user asks an agent at 4:47 p.m. whether a customer’s entitlement changed five minutes ago. Every copy introduces lag — and agents amplify stale answers with confident language.Token economics punish “dump everything”
Re-embedding millions of rows, re-sending warehouse samples, and stuffing document corpora into context does not scale. Teams report orders-of-magnitude token burn compared with scoped queries over live systems — same question, fraction of the cost.Compliance follows the copy
GDPR, HIPAA, SOC 2, and customer DPAs care about where data lives, not where your slide deck says it “should” live. Every lake, index, and vector store is a new surface to audit, breach, and explain. The agent did not create the compliance scope — your pipeline did.Copies diverge from truth
Source systems change column names, retire tables, and fix bad rows continuously. A shadow copy in S3 or Pinecone drifts. The agent answers from the shadow; finance closes books from the source. That is not a hallucination — it is an architecture mismatch.
Move data vs. query in place
Move everything
ETL → lake → warehouse → embeddings → RAG → prompt. High latency to build, high cost to run, hard to prove which row authorized an answer, and painful when access rules differ by role.
Query in place
Agent asks in business terms → reasoning layer maps to live sources → governed fetch → proof bundle. Records stay in Postgres, Salesforce, MongoDB, CSV, and REST APIs where they already belong.
RAG is not a data strategy
Retrieval-augmented generation solved a narrow problem: give the model relevant text. It did not solve relationships (“who owns this account?”), row-level access (“may this AE see payroll?”), or evidence (“show me the record that justified this sentence”).
When teams treat the vector store as the new system of record, they have simply moved the data again — into chunks with fuzzy boundaries instead of tables with keys.
You do not need another copy of your business. You need a governed way for agents to ask your existing systems — in vocabulary humans already use.
The thin layer that replaces the copy
The alternative is a reasoning layer between agents and siloed systems — not a new database, not another lake. Three ideas:
Playbook — what the business meansEntities, relationships, and access rules in version-controlled JSON — “customer,” “owns account,” “may see payroll for records they manage.”
Bindings — where data livesOne playbook, many sources: Postgres for CRM, CSV for payroll, SOQL for Salesforce — without merging them into one warehouse.
Proof — why the answer is trueEvery reply can carry source records and paths, so auditors and engineers replay the inference — not debate the model’s mood.
Anything Graph CLI implements this pattern for developers today: open-source adapters for SQL, CSV, Salesforce, MySQL, SQL Server, MongoDB, and REST — connect MCP in Cursor or Claude, ask in playbook terms, get answers from live data with nothing copied into a prompt dump.
When copying still makes sense
We are not arguing against analytics warehouses or archival storage. Aggregates, historical trends, and regulated retention still belong in purpose-built systems.
The shift is narrower and more urgent: do not copy operational data into an agent context path by default. Train, batch-report, and archive where copies help. Answer agents where truth already lives.
The bottom line
The AI age did not remove data gravity. It made duplication visible — in token bills, in compliance reviews, in wrong answers that sounded right.
The teams shipping production agents are converging on the same architecture: federated query, governed vocabulary, proof on every reply. Less pipeline. Less lag. Less risk.
Stop moving data for AI. Start reasoning over it where it already is.
Top comments (2)
Query in place is a good layer on top of the data - but it's a layer, not a replacement for what's underneath. You can't delegate the whole analysis to an agent. Deterministic ETL, batch jobs, the warehouse: that all stays, especially once the data is large. Someone still maintains it either way.
"Copies diverge from truth" is real, and proof on every reply is the part most people skip. But "stop moving data" overstates it - live query fixes the agent's read path, it doesn't remove the reasons the warehouse exists.
Proof on every reply — that's the line that changes the conversation around AI reliability. We chase hallucination metrics when the real problem is data architecture drift. Same thing in test automation: everyone obsesses over pass rates, nobody asks if the thing you tested matches the thing in production.