Language models are fluent, but they only know what they saw during training. Ask one about your own report and it will either admit it has no access or, worse, guess. During my internship at Valentius Kryptix, I built a small Retrieval-Augmented Generation (RAG) system to see how much a model improves when it can read your documents first.
The idea in one minute
RAG has two steps. Retrieval finds the passages of your documents that are most relevant to a question. Generation hands those passages to the model together with the question and tells it to answer only from that context. The model is never retrained. It simply gets the right text at the right moment.
What I built
Using Python, Chroma and Gemini, I ingested two real documents, about 9,000 words in total: my prompt engineering report and a long set of notes on agribusiness forecasting. The text became 40 overlapping chunks of roughly 2,000 characters. Each chunk was embedded and stored in a local Chroma database. For every question, the system retrieves the top four chunks and places them in the prompt.
A quick test
I asked: "In my prompt engineering report, what was the schema match rate for V1?"
The plain model said it had no access to my files. The RAG version retrieved three chunks from my report, one containing the exact result, and answered 1/6, which matches the report. It is one test, so it shows the pipeline works rather than proving it is perfect.
Three lessons
1. Chunking is a design decision. Chunks that are too large carry noise, and chunks that are too small lose context. Overlap and cutting at paragraph or sentence boundaries kept each piece readable.
2. Look at retrieval before blaming the model. I printed the retrieved chunks before connecting generation. In one test, the fourth chunk came from an unrelated part of the notes. The answer was still correct, but it showed that top-k search is not flawless.
3. Instructions matter. Telling the model to answer only from the given context, and to say so when the answer is missing, makes its behaviour easier to trust and to test.
Limits
My system is small: two documents, one run per question and a fixed top-4 retrieval. Next I would try re-ranking, compare different chunk sizes, and measure retrieval quality on a larger set of questions.
Read the full article
This is a condensed version. For the complete walkthrough, with screenshots and the with-vs-without-RAG comparison, read the original post on the Valentius Kryptix blog:
Read the full article
https://valentiuskryptix.com/powerful-rag-chatbot-reliable-answers-from-your-own-documents/
Code: https://github.com/samruddhishirgave-droid/rag-chatbot
Top comments (0)