This is part 2 of a series that started with the full reference diagram in part 0 and covered identity and the gateway in part 1. Stage 2 covers the column marked 2 on that diagram, DATA & RETRIEVAL, with three boxes: Enterprise data sources, Ingestion & indexing, and Retrieval index. The Retrieval index box is labeled "RAG, scoped by role," and roles only exist once stage 1's Roles & scopes box is in place, which is why this stage comes second. Nothing in this stage writes anything or sends anything, so the worst outcome available here is a wrong answer.
I use the same company as in part 1, Qingchuan, about 40 people, 8 engineers and 2 in marketing. Qingchuan is made up and only serves to walk the stage. Coming out of stage 1, everyone logs in through the identity provider, every call goes through the gateway, and each person carries a role. What they can ask about is still whatever the model already knew, because none of Qingchuan's own material has been indexed yet.
Enterprise data sources: docs, tickets, DB
The first box names the internal systems that are in scope. The label is "docs, tickets, DB". At Qingchuan that means the engineering wiki, the support ticket system, and a customer table in the product database. Without a written list, "the AI has our data" is a sentence nobody can check, and the scope of an incident cannot be stated when one happens.
The property that matters here is that each of those systems already has its own access rules, and those rules do not travel with the content. A wiki page restricted to engineering is restricted by the wiki. Copy its text into an index and the restriction stays behind in the wiki. Done looks like this: a written list of the systems in scope, each line carrying the access rule that system enforces today and who administers it, and anything not on the list stays out of the index until it is added to the list.
Ingestion & indexing: chunk, embed
Content is pulled from those systems, split into chunks, and turned into vectors that can be searched by meaning. The label is "chunk, embed". Without this box there is nothing to search and every answer comes from what the model was trained on.
Two things go wrong here, and both are quiet. The first is that chunking drops the context that made a passage safe to repeat. A paragraph from a page marked deprecated, a line from a policy that was never approved, and a customer's own words quoted inside a ticket all read as plain statements once they are a chunk on their own. The second is permissions. At query time there is a vector and a similarity score, so whatever access rule applies has to be attached to the chunk at ingestion time, when the source document is still identifiable.
The other half of this box is what happens after the first load. Documents get edited, deleted, and reclassified, and an index that was built once starts answering from material that no longer exists. Done looks like this: every chunk carries the identity of its source document and the access rule that document had at ingestion, and deleting or reclassifying a source document removes or reclassifies its chunks within a window the team has written down.
Retrieval index: RAG, scoped by role
This is where the role from stage 1 gets used. A question arrives with the asker's role attached, the index is filtered to the chunks that role may see, and the search ranks only what survives the filter. The label is "RAG, scoped by role". Without the filter, Qingchuan's wiki, tickets, and customer records sit in one index, and a question from marketing can pull back a customer record that only support should see. The index was never split by role.
The order of operations is the whole box. Filtering before the search means the model is handed material the asker was already allowed to read. An instruction in the prompt saying to answer only from documents this person may see is a request to a system that has already been given the material. Done looks like this: the filter runs before the search, the same question from two roles returns different chunks, and the scoping is enforced by the index, and the model has no part in it.
Checking an answer is slower than getting one
The read-only property of this stage is easy to read as low stakes. What it actually means is that the cost of this stage is paid in people's time. An answer that arrives with three internal documents cited next to it takes longer to check than an answer with nothing cited, because the citation makes it look already checked, and the only way to know is to open all three and read the passages.
I have run three separate lines of work where output got faster after AI was added, and on all three the bottleneck moved to the person doing the review. The volume went up and the number of people able to check it stayed the same. At Qingchuan the same thing shows up as a queue: producing a draft answer takes seconds, confirming it against the sources takes minutes, and the number of people willing to do the confirming becomes the ceiling on how much the system gets used.
The work that lifts that ceiling is making a single check cheap. Done looks like this: every answer carries the chunks it used, each with a link that opens the original document at the passage, so checking one answer costs a click.
The exit check before stage 3
Run this with real documents and two real accounts. Pick a document only support may see, and pick a question whose answer appears nowhere else. Ask it as a marketing person and the answer does not contain it. Ask the same question as a support person and it does. Then delete every instruction about permissions from the prompt and run both again. If either result changes, the scoping was in the prompt and the box is not built yet.
The second half covers the index staying true to its sources. Delete a document in the source system, wait out the refresh window the team wrote down, and ask a question that document used to answer. Its content no longer comes back. Until both halves pass with the system doing the enforcing, stage 3 is not the next thing to work on, because an agent reading through this index inherits exactly whatever scoping this stage failed to apply.
The DATA & RETRIEVAL column is drawn identically on the self-hosted variant of the diagram, and the choice between the two starts to matter here for one reason: indexing sends the text of internal documents to whatever computes the embeddings, so on the cloud variant that content crosses the enterprise boundary during ingestion, on the same path stage 1's PII / DLP filter was put in front of. Part 3 covers agents and read-only connectors, where the scoping built in this stage becomes the bound on what an agent can read on its own.
Top comments (0)