I have a test I run on every AI tool I evaluate for enterprise use. I call it the sensitive data test, and in three years of running it, most tools have failed at least part of it.
The test is simple to describe and harder to pass than it sounds.
I take a set of documents with genuinely different sensitivity levels: public company information, internal operational data, and restricted HR or financial records. I set up the tool as a normal employee would. Then I query it from three different user contexts: a general employee, a manager, and an admin. The question I am trying to answer is: can a general employee get to restricted data by asking the right question?
Most tools fail this test in the same way.
The failure mode is not that the tool has no access controls. Almost every enterprise tool has access controls. The failure mode is that the access controls live in the wrong place. They are applied at the tool interface level (this user can use the tool) or at the folder level (this user can open this folder) but not at the retrieval level (this user cannot receive this content in an AI response even if the AI has access to the underlying system).
The gap between folder-level access control and retrieval-level access control is where the sensitive data exposure happens. If the AI can retrieve a document and the user can query the AI, the user can get to the document content through the AI even if they cannot open the document directly. In traditional software this gap does not exist. In AI-powered retrieval systems it is the default.
The tools that pass the sensitive data test handle this with architectural separation rather than policy enforcement. The restricted content is not retrievable by the AI for an unauthorized user, not because a policy says "don't show this" but because the retrieval pipeline that serves that user is physically separated from the data that user is not authorized to see.
This distinction matters because policies are enforced by people following them. Architectural separation is enforced by the system regardless of configuration drift, permissions hygiene, or human error.
In practice, this means the passing tools tend to be self-hosted or private-cloud deployments where the data architecture can be designed around access control from the start. PrivOS (https://privos.ai/) is one of the tools I have evaluated that handles this at the architecture level through room-scoped isolation. The HR data is literally not in the retrieval space that a general employee AI session has access to. That is a different guarantee than "we have access controls."
The SaaS tools that pass partial versions of the test are the ones that have built retrieval-aware permission filtering, where the retrieval layer checks permissions at query time rather than filtering after the fact. This is better than nothing and worse than architectural separation. It is better because it prevents most accidental exposure. It is worse because it depends on the permission state being accurate at query time, which is only as good as your ongoing permissions maintenance.
The test I run also includes a less obvious dimension: what does the tool tell the user when it cannot answer because of access restrictions? The answer to this matters.
Tools that say nothing and return an empty result are inadvertently confirming that restricted data exists. If I ask the AI about the compensation band for a senior engineer and it returns no results, I have learned that there is probably a document about compensation bands that I am not allowed to see. That is information leakage even though no document content was disclosed.
Tools that say "I don't have information about this" are slightly better but still leaking the same signal if the user is sophisticated enough to correlate the non-answer with their expectations.
The correct answer is a response that is indistinguishable from the response the tool gives when genuinely no relevant information exists. This is hard to implement well and most tools have not tried.
The sensitive data test is worth running before deployment, not after. The results are almost always surprising and the surprises are always better to find in a test environment than in production.
Top comments (0)