Building a Simple RAG System with Python, ChromaDB and Gemini

Sanjay Kj — Tue, 22 Sep 2026 08:28:58 +0000

Large Language Models are impressively fluent, but they have a blind spot: they can only answer from what they learned during training.

Ask one a question about a specific PDF, a company's internal handbook, or a niche textbook, and it may guess, generalize, or simply admit it doesn't know.

Retrieval-Augmented Generation (RAG) closes this gap by giving an LLM a way to look things up before it answers — turning a closed-book exam into an open-book one.

During my AI internship at Valentius Kryptix, I built a lightweight RAG system to explore exactly this idea, using a Data Structures and Algorithms PDF as the test document.

Rather than treating RAG as a black box, the goal was to understand why each stage of the pipeline exists — not just how to wire it together.

🔍 The core insight: retrieval only works if "relevance" is defined semantically, not literally.

A keyword search for "queue operations" could fail if the document phrases the same concepts as "Enqueue and Dequeue functions."

This is where embeddings come in. Using all-MiniLM-L6-v2 via Sentence Transformers, the system converts both the document and the user's question into vectors that capture meaning rather than exact wording.

Storing these vectors in ChromaDB makes it possible to find conceptually related content even when the exact words don't match.

The most instructive part of the project wasn't the pipeline itself, but comparing the model's behavior with and without retrieval.

Asked:

"What are the operations performed on a queue?"

Without RAG, Gemini answered from general training knowledge — correct in spirit, but disconnected from the actual source material.

With RAG, the same question returned an answer grounded in the specific document, correctly surfacing:

• Enqueue
• Dequeue
• Peek/Front
• Rear
• isFull
• isEmpty

That side-by-side comparison made the value of retrieval tangible in a way that reading about RAG never quite does.

💡 The bigger takeaway: a RAG system is really a bridge between two very different technologies — a vector database good at finding information, and a language model good at explaining it.

Neither one alone solves document-based Q&A well; together, they compensate for each other's weaknesses.

This piece is a shorter reflection on that design insight. The full build — including the complete five-step workflow, code, and implementation details — is covered in the original article.

📖 Read the full article:
https://valentiuskryptix.com/how-to-build-a-retrieval-augmented-generation-rag-system-in-just-5-steps/

RAG #RetrievalAugmentedGeneration #GenerativeAI #ArtificialIntelligence #MachineLearning #LLM #ChromaDB #Python #AIInternship #ValentiusKryptix

DEV Community: Sanjay Kj

Building a Simple RAG System with Python, ChromaDB and Gemini

RAG #RetrievalAugmentedGeneration #GenerativeAI #ArtificialIntelligence #MachineLearning #LLM #ChromaDB #Python #AIInternship #ValentiusKryptix