A retrieval-augmented assistant answers from documents it fetched first. Most access reviews ask what the assistant is allowed to open. Fewer ask what it is allowed to say about what it found.
In regulated systems, that second question is where the gaps are.
The citation title is a disclosure
Picture a compliance assistant at a bank. A branch employee asks how wire escalations are handled. The answer is accurate and harmless. The first citation reads "Case file: Hartwell Holdings, sanctions review."
Nobody showed the employee the file. They were shown its name. Filenames, matter titles and folder paths often hold client names, patient names and case types. Treat them as data, because they are.
The summary is written from whatever was retrieved
If the search pulled in passages the user could not see, the model will use them. Filtering the final answer afterwards rarely works, because the passage has already been blended into the prose.
Take a clinic's billing clerk asking about denial patterns for a procedure. The summary mentions a diagnosis. It came from a clinical note the clerk has no right to read, and the clerk never opened that note.
The cache remembers the first user
Some systems store generated summaries so the next question is cheaper. Others build the search index using one service account's view of the documents. In both cases, a later user can get an answer shaped by the first user's permissions.
An insurer's claims assistant can do this quietly. An adjuster with broad access asks about a policy line, and the summary is saved. A contractor with narrow access asks the same question and gets the saved version.
What to check
- Where is the permission filter applied? It should sit inside the search itself, before anything is retrieved, not after the answer is written.
- Do titles, filenames and snippets go through the same check as the body text?
- Are stored summaries tied to the permissions of the person who asked, or shared across users?
A test worth running
Sign in as the lowest-privilege user the tool serves. Ask a question whose best answer lives in a record that user should never see.
Then read three things. Read the answer. Read every citation, including titles. Read the refusal, if there is one.
That last one matters more than it looks. A message like "I can't show you the Hartwell file" tells the user the file exists. A safe response behaves as if the record were not there.
Permissions on the document store are only the start. The assistant has to respect them in the search, in the summary, in the citation and in the cache.
I'm Jonathan Melton. I run FusionAL, which reviews how AI agents and MCP tools are permissioned when they reach regulated systems.
Top comments (0)