DEV Community: Vaultiscan

How Can I Make an AI Chatbot Answer Questions Using My Company's Own Documents?

Vaultiscan — Mon, 12 Oct 2026 05:53:40 +0000

Short answer: don't retrain the model. Connect it to your documents. The standard approach is retrieval-augmented generation (RAG): Your files are indexed, and the relevant sections are retrieved for each question, which the language model then generates an answer from, citing the sources of every statement. This is the way nearly all tools that enable staff to chat with your documents operate today, both a single-PDF program and an enterprise program.
The model is rarely the hard part. The difficult part is ensuring that the chatbot respects who can see what, knows which version of the document is the current one, and doesn't guess when it doesn't find it.
This is no longer a side project. The McKinsey's The State of AI (August 2026, 1,719 participants across 97 countries) found that chatbots were the most widely scaled AI application, with 47% reporting that their organisation is using them across the enterprise. The same survey reveals the top role for AI agents is in IT, knowledge management, and software engineering.
How does RAG work?
1. Connect and ingest
Retrieve documents from where they reside: SharePoint, OneDrive, Google Drive, Confluence, and shared drives. Scanned documents must be OCRed (optical character recognised) prior to their use.
2. Chunk and embed
Divide each document into a few paragraphs, and then use a vector embedding, a numeric representation of the meaning of each passage, to represent each paragraph. These are added to an index, along with other metadata like source, date, owner and access rights.
3. Retrieve
When a question is asked, the system searches for matching text segments in the text (typically using keywords as well) and restricts which parts of the text are displayed based on permissions.
4. Generate with citations
The model is provided with the question and the retrieved passages and is asked to answer based solely on these passages without citing any of them or saying that the question cannot be answered.
This approach was presented in a 2020 Facebook AI Research paper and has since become the standard one for grounded question answering.
Why not fine-tune a model on your documents instead?
It's the common instinct, and usually the wrong one. On factual knowledge, the RAG vs fine tuning evidence leans clearly one way. Microsoft researchers published their study at EMNLP 2024 and concluded that the RAG "consistently outperforms" unsupervised fine-tuning for knowledge it had read during training and entirely new knowledge.
In practice, a finely-tuned model is unable to indicate which document an answer is from.
Practically, a fine-tuned model can't show which document an answer came from. It goes stale the moment a policy changes. It also cannot enforce access rights, since all it has learned is accessible to anyone who uses it. Fine tuning is used for tone, format and specialist vocabulary. It's not a file system.
Why not just upload the files to a public chatbot?
For a handful of non-sensitive files, that works fine. For company knowledge, it's where data leaks start. Harmonic Security analysed 20,000 files that employees uploaded to more than 300 generative AI and AI-enabled SaaS apps between April and June 2025, and found nearly 22% contained sensitive content.
Three realistic routes
1. Off-the-shelf "chat with PDF" tools
Quick and inexpensive to use on public or low-risk documents. Not intended for permissions, audit trails or thousands of files.
2. Build your own
Getting a prototype up and running will be fast if you use a framework like LangChain or LlamaIndex, a vector database, and a model API. Production is the long part: connectors, permission sync, version control, evaluation, and monitoring all become your team's job.
3. An enterprise platform
Glean, Coveo, Moveworks and VaultiScan address slightly different aspects of the problem, starting with searching in your workplace, searching for your customers, searching for your employees, and starting from your governed documents and data answers. You trade some control for connectors, permissions and governance that are already in place.
What actually decides whether it works
RAG reduces errors. It won't delete them. According to a 2025 study, published in the Journal of Empirical Legal Studies, two top legal research tools based on RAGs—Lexis+ AI and Westlaw AI-Assisted Research—hallucinated between 17% and 33% of the time on average. Four things separate a trusted chatbot from an abandoned one:
Permissions that follow the user
Only the person who is asking should open what is retrieved. Otherwise it is a data leak through a good looking interface.
One source of truth per document
If three versions of the travel policy sit in three folders, the chatbot will find all three. Decide which is current before indexing.
Citations on every answer
It should be possible for people to click through and check, and make blind trust a quick verification.
Room to say, "I don't know." An assistant who doesn't answer when it is not necessary to answer is more valuable than one who will do so at any time. Then compare the results to a fixed list of real questions that your staff ask - before launching, and after every major content update.
Where VaultiScan fits
Those four failure points are what VaultiScan, our enterprise knowledge agent platform, is designed around. Vaulti GPT pulls data from SharePoint, One Drive, Azure Storage, and other sources; provides only what each user can see; and provides the source for each answer. Data stored and processed within the customer's own Azure environment, and it is never used to train the underlying model. Teams wishing to control retrieval within their own intranet or app can embed it within the Vaulti SDK and create the interface as they see fit. The trade-off: It requires the right type of implementation – not the fast implementation – and it's best for organisations that are already familiar with Microsoft Azure.
Quick answers to common follow-ups
Can this be done without a developer?
Yes, for a few files, using an off the shelf tool. If you're looking for company-wide deployment with permissions, factor in the involvement of IT.
Which documents should go in first?
Begin with the most immediate business use case for AI for employees' questions: HR policies, how-to documents for IT, onboarding, and product documentation. They are asked about frequently, change is infrequent and easily checked.
Final Thoughts: Retrieval Over Retraining
The question seems to imply that it's about making the AI smarter. It's simply about making your documents "discoverable, up-to-date and correctly permissioned" and then setting the model to their location. Retrieval is better than retraining for company knowledge, citations ensure that every answer can be checked, and permissions ensure that each answer is within limits.
Once these three are fixed, an AI assistant for documents is not just a “launch day” thing, it's a “day 100” thing. Begin with one team, one neatly produced set of documents, and a list of genuine questions.

Enterprise AI Security: Best Practices for Protecting Business Knowledge

Vaultiscan — Mon, 24 Aug 2026 07:52:29 +0000

Key Takeaways
• The basic AI security control is “Permission-aware retrieval” and not “broader indexing.” An assistant should only surface what the person asking is already entitled to see.
• Shadow AI is a measurable exposure, not a theoretical one. IBM X-Force Threat Intelligence Index 2026 revealed that an estimated 300,000 AI chatbot credentials are available on the dark web.
• The focus of AI privacy risks is now from Exposure to Inference. Gartner forecasts that by 2029, the majority of privacy incidents will be caused by inferences made about individuals as a result of AI, not actual data exposure.
• Regulatory deadlines are already on the calendar. The EU AI Act's high-risk obligations will kick in on 2 December 2027 and 2 August 2028, leaving a shrinking timeline for the development of governance.
The model remains the focus of most enterprise AI security budgets, especially with regard to red team prompts, and testing for jailbreaks, observing for 'hallucinated' output. That is the visible risk. The more expensive one is underneath. If an AI assistant is able to query contracts, claims files, financial models and HR records upon request, it's not the AI assistant that's actually the attack surface. The knowledge base it can reach is. AI security best practices for 2026 have to start from that premise: govern what the model can see before spending more effort on what it says.
Seven Practices That Actually Reduce AI Risk
1. Inventory what the AI can reach before you police what it says
Most AI governance programmes start with model behaviour: acceptable-use policies, content filters, prompt logging. Some enterprises have already licensed an AI governance platform for exactly that layer. Few extend the same discipline to a knowledge map.
Security and data groups should have a list of all the sources an assistant is plugged into—usually a combination of contracts, claims files, engineering specs, HR data, and financial models—before it is activated. There would be no way to determine the policy for any new permissions that are requested later if that inventory wasn't there.
2. Make retrieval permission-aware, not just login-gated
Permission-aware retrieval is when the AI system validates a user's current permissions when answering a question, and not just when they log in.
A single sign-on prompt isn't enough: once someone is inside the perimeter, an un-permissioned retrieval layer treats every indexed document as fair game. The rule that already governs the file system needs to carry through to the assistant: if a contractor cannot open a compensation file manually, an AI system should not be able to summarise it for them either. Platforms built around this principle, such as Vaultiscan's Vaulti Lake, its permission-aware data layer, enforce the boundary at the point of retrieval instead of relying on the model to self-censor.
3. Close the shadow AI gap with a sanctioned alternative
Employees route around a policy that never gave them a fast, approved option to follow. In fact, there is evidence that unsanctioned use of AI has already become an exploitable market, as IBM's X-Force Threat Intelligence Index 2026 reported that 300,000 AI chatbot credentials were listed for sale on the dark web. Blocking unauthorised tools without offering a comparable, governed one simply pushes the behaviour further out of view. A named alternative, with similar convenience but inside the security perimeter, is the more durable fix than a policy memo alone.
4. Treat AI outputs as inferences, not just answers
An AI assistant does not only retrieve what a document already says. It can also infer what a document does not say, drawing conclusions about a person, a deal, or a risk from patterns across many sources at once. Gartner predicts that by 2029, most privacy incidents will result not from direct exposure of personal data but from AI-generated inferences about individuals and expects spending on data-integrity protections to reach parity with data-confidentiality spending by 2028. This is the sharp edge of data privacy AI risk: a security review that only checks what an assistant discloses verbatim is checking the wrong half of it.
5. Keep a verifiable audit trail back to the source
If an AI-generated answer cannot be traced back to the document, clause, and version from which it was generated, a compliance team will be unable to check it and will have no way to defend it in an audit. Log retrieval calls, reference the underlying document in the response, and version-control the knowledge base, enabling an answer to be reproduced or challenged after the fact, and not just believed. Vaultiscan's Vaulti GPT, its governed AI assistant layer, is built to cite the source document and version behind every retrieved answer for exactly this reason.
6. Build for the regulation that is already scheduled
AI data protection is no longer only a security team's internal standard; it is increasingly governed by scheduled compliance dates. The EU AI Act became applicable from 2 August 2026, with specific obligations for high-risk uses of AI in sensitive sectors (like biometrics, critical infrastructure, education and employment) taking effect on 2 December 2027, and the obligations for AI embedded in regulated products on 2 August 2028. Implementing EU AI Act compliance as a design input, rather than a design deadline, allows for more runway than waiting for those dates and applying a governance framework at the last minute.
7. Plan for the AI-specific incident, not just the generic one
A phishing playbook won't provide an answer to a prompt injection that exfils a list of clients out of a chat response, and a generic breach playbook won't answer what to do if the compromised asset isn't a database, but rather a fine-tuning dataset. Companies must have an incident-response plan designed for AI failures, such as a broken connector, a poisoned source document, a jailbreak that pumps back content that it should have blocked, and so on — and that plan must be practiced, not written.
Frequently Asked Questions
What is the biggest AI security risk for an enterprise deploying generative AI tools?
The most under-addressed risk is not the model misbehaving; it is the model reaching business knowledge it was never scoped to see. Restricting what an AI assistant can retrieve, not only what it can say, closes the larger gap.
What is shadow AI, and why does it matter for data protection?
Shadow AI refers to AI tools that have not been reviewed, approved, or integrated into an organisation's governance framework — typically a public AI bot used to summarise a confidential document. There are 300,000 AI chatbot credentials already available for sale in the dark web, IBM's X-Force Threat Intelligence Index 2026 says, meaning this exposure isn't theoretical.
Does the EU AI Act apply to how enterprises secure AI systems, or only to high-risk use cases?
Both. The Act enters into force on 2 August 2026, while the obligations in sensitive areas will come into effect on 2 December 2027, and the rules for product embedded high-risk systems on 2 August 2028. General purpose enterprise AI systems have transparency and risk-management responsibilities long before any use case can be deemed high-risk.
What does “permission-aware” AI retrieval actually mean in practice?
It means the AI system checks a user's existing entitlements before returning an answer, the same way a file system already does, rather than indexing every document equally and trusting the model to withhold what it should not share.
The Perimeter Moved to the Prompt
The right security question about enterprise AI is not “can the model be tricked.” It is “what can this assistant already reach.” A jailbroken model with no access to sensitive knowledge is an inconvenience. It is the well-behaved model with unrestricted access to it that is the real exposure.
All of this — the inventory, the compliance calendar, permission-aware retrieval, treating inferences as outputs, and a rehearsed incident-response plan — converges on the same point: the knowledge layer, not the model. That's where AI data protection really counts, and it's not just an add-on feature that's tacked on at the end, after an assistant is already in production.