RAG for Company Knowledge and Support: Architecture, Controls and Rollout

Originally published on the Edilec blog: https://edilec.com/blog/ai-1080/rag-for-company-knowledge-and-support/

Retrieval-augmented generation, or RAG, can help employees and support teams answer questions from approved policies, product documentation, procedures and case history. The system searches a controlled corpus at request time, supplies selected passages to a model and asks it to answer from that evidence. This makes knowledge easier to update, but it does not make every answer correct. A production design must treat content quality, retrieval, authorization, generation and human escalation as separate controls.

What RAG does and where it fits

Ingest approved sources, divide them into retrievable units, attach metadata and permissions, create lexical and vector representations, retrieve candidates, rank them, assemble a bounded context and generate an answer with evidence references. RAG is an information system with a generative interface, not merely a prompt attached to a document folder.

Retrieval is not authorization

A relevant document may still be restricted. Enforce access before passages enter model context, and test authorization at query time. Hiding a citation after generation does not undo disclosure to the model or user.

Prepare the knowledge base before tuning the model

Duplicate procedures, expired policies and contradictory product notes will produce inconsistent answers regardless of model quality. Inventory sources and classify each as authoritative, supplementary or excluded. Chunking should follow meaning, not an arbitrary character count.

Design retrieval and answering as observable stages

Lexical search is strong for error codes, names and distinctive phrases; vector search helps when user wording differs from the documents. Hybrid retrieval runs both and combines their candidates. Do not assume more retrieved chunks always improves answers.

Evaluate the pipeline, not only the final prose

Build a representative evaluation set from real, de-identified questions, including common requests, rare terms, typos, ambiguous wording, restricted documents and questions with no answer. Measure recall, ranking quality, groundedness, citation correctness, abstention quality and authorization.

A staged RAG rollout plan

1. Inventory and govern one bounded domain with named source owners.
2. Build retrieval first, and tune it against labeled questions before adding generation.
3. Add answer generation offline and test citations and no-answer behavior.
4. Run a limited internal pilot with required review.
5. Expand with gates only when authorization, freshness and relevance meet criteria.
6. Operate continuously and retain a tested rollback path.

Key takeaways

• Treat RAG as a governed information system, not a prompt attached to a document folder.
• Clean, classify and assign owners to source content before tuning retrieval.
• Enforce user and document permissions before evidence enters model context.
• Evaluate retrieval, groundedness, citations, abstention and operations separately.
• Start with a bounded, reviewable use case and expand only through measured gates.

More from Edilec: How AI agents work in business workflows (edilec.com/blog/ai-1024) and Edilec's AI automation services (edilec.com/services/ai-automation).