Data usually moves through a few stages before anything gets to reason over it. It starts in transaction systems, servicing platforms, SaaS tools, and document repositories. From there it lands in a lake, typically in three stages: bronze, silver, and gold. Bronze is raw data, barely touched. Silver is cleaned up and standardized. Gold is the finished version, ready to use.
At every one of those stages, someone is making governance decisions. Who’s allowed to see this. How long it gets retained. Whether it needs to be masked before it moves forward. Whether it’s even trustworthy enough to promote to the next layer. That work is supposed to happen quietly in the background, so that by the time data reaches gold, an analyst building a report doesn’t have to think about any of it.
Agents are starting to sit right where that analyst used to sit, sometimes at the gold layer, sometimes reaching further upstream than that. And almost none of the governance built into that pipeline was designed with an autonomous actor on the other end. It assumed a person, a query, and a dashboard. Not something that reads the data and then goes and does something with it.
Most of what’s being written about agentic AI right now is orchestration frameworks, memory, tool-calling, multi-agent patterns. Almost none of it asks the more basic question: can the agent actually trust the data it’s reaching into? If the answer is no, none of the orchestration sophistication matters. You’ve just built something that fails faster and with more confidence.
Why this is a different problem than “AI-ready data”
Enterprises have been chasing AI-ready data for years now. Clean it, catalog it, govern it, make it available for models to train on or reason over. That work matters, but it was built around a specific assumption: a human or a model is going to read this data and produce an answer.
Agents break that assumption. An agent doesn’t just read data and summarize it. It reads data and then acts on it, approves something, routes something, updates a record, kicks off a downstream process, often without a human checking its work first. That changes what “ready” actually means.
A stale customer record is a minor annoyance in a dashboard. It’s a real problem when an agent acts on it. A dataset with unclear lineage is fine for a BI report nobody’s betting money on. It’s not fine when an agent’s decision needs to be explained to a regulator six months later, and nobody can say for certain where the underlying number came from.
So the bar for “AI-ready” quietly moves. It’s not just about quality and structure anymore. It’s about whether the data can support an autonomous action, and whether you can reconstruct what the agent saw and why afterward.
A walkthrough: the hardship review agent
Let’s take an example. Picture an agent built to triage loan hardship requests, something a servicer might use to decide whether a request gets fast-tracked for a forbearance plan or routed to a person for review.
To do this well, the agent needs to reason across three different kinds of data, and of the three, only the structured data actually made the bronze to silver to gold journey I described above in any meaningful way.
The structured layer is the loan servicing system: payment history, current balance, delinquency status. Clean rows, clear schema, row-level security and audit logs behind it in most shops. This is the layer governance programs are genuinely built for.
The semi-structured layer is the hardship application itself, usually a form submission or an income verification feed. JSON or something close to it, with fields that aren’t always filled in the same way twice (income entered as “$4,500" in one submission and “4500.00" in another). Manageable if the schema is documented and versioned, though in practice it usually stops between bronze and silver, since reaching gold takes modeling investment that operational data like this rarely gets ahead of an actual consumer asking for it. Nothing about JSON prevents it from being modeled all the way to gold. It’s a backlog problem, not a technical ceiling, and an agent that needs to reason over this data is exactly the kind of consumer that finally justifies doing that modeling work.
The unstructured layer is everything else. The hardship letter the borrower actually wrote. A pay stub uploaded as a PDF. Maybe a transcript from a call with a servicing rep last month. This layer typically bypasses the medallion pipeline entirely. It goes from a document repository straight into a vector index, because that’s the path that makes it retrievable, and retrievability is what the agent project needed. That bypass is where the trouble starts, and it’s worth being precise about why.
What actually happens to access control when you embed a document
When a document gets chunked and embedded, you’re creating a derived artifact that lives in a different system, under a different storage model, with a different access path than the source ever had. Whatever ACL sat on that pay stub in the document repository doesn’t follow it into the index unless someone deliberately carried it across.
There are roughly four ways teams handle this, and they fail differently.
None of these four are wrong exactly. The point is that “we handle access control on the vector store” is not a complete answer, and the follow-up question is which of these you’re doing and where the enforcement actually sits.
Then there’s re-embedding. Documents get updated, chunking strategies get tuned, embedding models get swapped for better ones, and each event regenerates the derived artifacts. Access metadata has to survive that regeneration. I’ve seen setups where the original ingestion carefully propagated ACLs, and the re-indexing job, written six months later by someone else, didn’t.
Scoping access when the consumer has a task instead of a job
Most data access models were built around people and their job functions. An analyst in finance sees finance data, and that grant is durable, tied to identity, reviewed on some quarterly cadence.
An agent doesn’t have a job function in that sense. It has a task. Back to the hardship agent: on one particular run, it needs this borrower’s payment history, this borrower’s application, and this borrower’s uploaded documents. It does not need every borrower’s records, which is what it almost certainly has, because the service account it runs under was provisioned against the servicing domain rather than the invocation.
That gap between what the task requires and what the identity permits is where most of the real exposure sits, and it’s not obvious how to close with the tools most enterprises have.
The direction I think this has to go is attribute-based access control evaluated per invocation rather than per session: the agent doesn’t hold a standing grant to the servicing domain, it presents a token scoped to loan number X for the duration of this task, and the data layer enforces that scope on every read regardless of which system the read lands in.
That’s clean to describe and hard to build, for three reasons.
I don’t think anyone has fully solved this. But I’d rather see a team say “we know our agent’s identity is over-provisioned relative to its task, here’s our compensating control” than assume the service account model covers it.
Lineage that has to survive a question six months later
Lineage has always been a governance artifact. Someone produces a diagram, it goes in a catalog, it’s reviewed when an auditor asks. With agents it becomes something closer to a runtime obligation, because the question changes from “where does this table come from” to “what did this specific agent run see, and why did it act the way it did.” For the hardship agent, that’s the difference between a diagram of the servicing pipeline in general and being able to say exactly what it read before recommending one borrower’s forbearance decision.
Answering that means capturing, per invocation, the task context, the queries issued and against what, the specific chunks retrieved with source identifiers and versions, the tool calls made, and the decision produced. Not a summary. Durable references you can resolve later.
The structured side of this is tractable. Queries are loggable, tables are versioned, a warehouse can usually tell you what a query returned at a point in time.
The unstructured side is where it falls apart, and the reason is the re-embedding problem again. If you log chunk identifiers for that pay stub or hardship letter, and the corpus gets re-chunked eight months later, those identifiers point at nothing, or worse, at different text. Reconstructing what the agent actually read requires either immutable chunk storage with real versioning, or logging the retrieved content itself rather than pointers to it. The first is operationally heavier than most teams expect. The second creates a second copy of your sensitive data sitting in your audit store, which is its own governance problem.
That tension is real and I don’t think there’s a free answer. But it’s worth deciding deliberately rather than discovering it during an examination.
Where This Fits in Build, Scale, Govern
This is the same Build, Scale, Govern pattern, just one layer down from the agent. In Build, teams give agents whatever data access is easiest to set up. Getting it working matters more than getting the access right. In Scale, that shortcut starts to hurt. New agents copy the same broad access, and the gaps add up. In Govern, this is what gets checked. Not “is your data clean” but “did this system know only what it should have.”
The data layer doesn’t move through Build, Scale, Govern on its own schedule. It follows the agent layer, usually a step behind.
What I’d actually watch for
I don’t think the answer here is a new platform category, and I’d be skeptical of anyone selling one. Most of what’s needed is existing governance machinery applied at a granularity it wasn’t designed for.
The question worth asking isn’t lakehouse versus warehouse, or which vendor has the best vector search. Plenty of systems can log what an agent retrieved. Plenty can check permissions at the moment of retrieval. Far fewer can put those two things together into evidence that still holds up later, showing exactly what unstructured content was touched and under what access rules, months after the fact. That’s the real test, and for unstructured data, most platforms today aren’t built for it.
That gap gets more expensive as agents move from pilots into things that run unattended. It’s the layer I think deserves more attention than it’s getting. Not the agent. What’s underneath it.
References
Board of Governors of the Federal Reserve System, OCC, FDIC. SR 26–2, “Revised Guidance on Model Risk Management,” April 17, 2026. https://www.federalreserve.gov/supervisionreg/srletters/SR2602.htm (full text: https://www.federalreserve.gov/supervisionreg/srletters/SR2602a1.pdf)
NIST Special Publication 800–162, “Guide to Attribute Based Access Control (ABAC) Definition and Considerations.” https://csrc.nist.gov/pubs/sp/800/162/upd2/final
OWASP Cheat Sheet Series, “RAG Security Cheat Sheet.” https://cheatsheetseries.owasp.org/cheatsheets/RAG\_Security\_Cheat\_Sheet.html
Databricks, “What is the medallion lakehouse architecture?” https://docs.databricks.com/aws/en/lakehouse/medallion
Top comments (0)