DEV Community: Sanyam Kansal

AI-RAG-Bot — chat with your documents. 100% local, 100% private, $0 to run

Sanyam Kansal — Sun, 18 Oct 2026 19:09:30 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

What I Built

AI-RAG-Bot is a local "chat with your documents" assistant: drop files into a folder, ask questions in your terminal, and get answers grounded strictly in those files. It reads .txt, .pdf, and images — and for scanned PDFs and photos of documents, it runs OCR (Tesseract + OpenCV), so even a photographed receipt becomes searchable knowledge.

Who would pay for this? Anyone who handles documents they're not allowed to paste into a cloud chatbot:

A freelancer whose client contracts are under NDA
An accountant handling clients' tax documents
A law student working through case files
A small clinic with scanned intake forms

For all of them, the "easy" option today — pasting a client PDF into a hosted AI service — means shipping sensitive data to a third-party server. AI-RAG-Bot's entire value proposition is that nothing ever leaves the laptop: embeddings live in a local ChromaDB, the model runs through Ollama, and the only network call is the initial model download.

And because there's no per-query API bill, the unit economics work: set it up once for a client and it costs $0/month to run, forever. Try building that margin on top of a closed API.

Code

sanyamkansal / AI-RAG-Bot

Local RAG system for intelligent document question answering using OCR, vector search, and a local LLM

AI RAG Bot

AI RAG Bot is a simple, locally-hosted Retrieval-Augmented Generation (RAG) system built with Python. It allows you to query a knowledge base of your own documents, including text files, PDFs, and images (using OCR). The bot processes these documents, creates vector embeddings, stores them in a local ChromaDB instance, and leverages an LLM via Ollama to generate answers grounded in the retrieved document context.

Key Features

Multi-format Document Support: Process .txt, .pdf, .png, .jpg, and .jpeg files.
Optical Character Recognition (OCR): Extracts text from images and PDFs using Tesseract OCR, PyMuPDF, and OpenCV.
Smart Chunking: Cleans and chunks document text to fit context windows efficiently.
Local Vector Database: Uses chromadb for persistent, local storage of vector embeddings.
Sentence Embeddings: Utilizes the lightweight and fast all-MiniLM-L6-v2 model from sentence-transformers to generate embeddings.
State Tracking: Keeps track of document modification times and sizes (document_state.json…

View on GitHub

How I Built It

Every layer of the stack is open source, and the open pieces are exactly what make the project work:

Local inference: Ollama serves Google's open-weight gemma3:1b — the smallest Gemma variant, about 1 GB, so it runs on modest laptops with no GPU. RAG keeps prompts short and grounded in retrieved chunks, which is exactly why a 1B model is enough here: it answers from the provided context and says "information not found" when the answer isn't in the documents.
Embeddings: all-MiniLM-L6-v2 from sentence-transformers — small, fast, CPU-friendly.
Vector store: ChromaDB in persistent local mode. The entire "database" is a folder on disk.
Extraction: PyMuPDF for text PDFs; OpenCV + pytesseract (Tesseract OCR) for scans and images.
State tracking: document_state.json remembers each file's size and modification time, so unchanged documents aren't re-processed on startup.

The pipeline: scan documents/ → extract text (OCR where needed) → clean → chunk (~500 characters) → embed → store. At query time, the question is embedded too, ChromaDB returns the most similar chunks, and gemma3:1b is prompted to answer only from those chunks. That "I couldn't find that" behavior matters more than raw cleverness when someone is asking about a contract.

Why Does Open Innovation Matter?

Privacy a closed API structurally can't offer. This isn't a policy promise ("we don't train on your data, trust us") — it's architecture. There is no server to send anything to.
Cost. Embeddings and generation run on hardware the user already owns. For the freelancer use case, that's the difference between a tool free to run and one with a monthly bill per seat — the reason this side hustle works at all.
Freedom to swap. Want a different model? ollama pull anything and change one string. Different embedder? One line. No vendor decides what this tool can become.
Offline by default. It works on a plane, in a basement archive, anywhere with no internet. Only a local stack can promise that.

Prize Categories

Best Use of Gemma — AI-RAG-Bot runs Google's open-weight gemma3:1b as its generation model, served entirely locally through Ollama. No API key, no cloud calls: the model lives on the user's own machine.

Building Dukaan Sathi: An AI Voice Assistant for India's Kirana Stores

Sanyam Kansal — Sat, 15 Aug 2026 17:13:55 +0000

Over the past 9 days, I took on the Voice for Bharat Challenge to build an AI Voice Agent tailored for India. The result? Dukaan Sathi, a fully functional voice assistant designed specifically for local Kirana store owners (dukaandars).

Here is the story of how Dukaan Sathi evolved from a simple starter template into a multi-agent powerhouse capable of managing inventory, handling escalations, and speaking in authentic, pure Hindi.

🚀 The Problem & The Solution

India runs on Kirana stores. Dukaandars are incredibly busy people, often juggling inventory, customer queries, and billing all at once. They don’t have the time to sit at a laptop and type data into complicated software.

Why voice? Voice is the most natural, frictionless interface. I wanted a Dukaandar to be able to just speak to their phone: "Mere paas kitna chawal bacha hai?" (How much rice do I have left?) and get an instant, helpful response.

Dukaan Sathi is that solution. It is a helpful, polite, and intelligent voice agent that acts as a digital assistant for the store.

🛠️ Key Features Built Over 9 Days

Dukaan Sathi isn't just a basic chatbot with text-to-speech bolted on. It's a complete, intelligent system. Here are the highlights of what I built:

Authentic Indian Voices: Powered by Murf Falcon, Dukaan Sathi uses extremely natural-sounding Indian voices. The main agent uses a polite female voice (Anisha), ensuring a welcoming tone.
Strict Native Script Guardrails: One of the most important features I enforced was ensuring the LLM always outputs Hindi in the native Devanagari script (नमस्ते) and never in Romanized Hinglish (namaste). This ensures the text-to-speech engine pronounces words perfectly without awkward accents.
Persistent Memory: The agent remembers returning customers! Using SQLite and LiveKit's Job Context, the agent seamlessly pulls up past interactions so it doesn't have to re-ask for your name.
Tool Use & Analytics: The agent can actively fetch stock balances and log call outcomes. I built a real-time Call Analytics Dashboard in Next.js that updates the moment a call ends!
Human Escalation: When a customer gets angry or the agent doesn't know the answer, it triggers a create_escalation tool that saves a transcript and logs a ticket for the human owner to review.
Multi-Agent Handoffs: The crowning achievement! If a user asks for a refund, Dukaan Sathi transfers the call to a dedicated Returns Specialist. The caller actually hears the voice change to a completely new male voice (Samar), and he instantly knows the context of the conversation.

🧗 The Difficult Parts & Lessons Learned

Pretending everything worked perfectly on the first try wouldn't be helpful! Here are some hurdles I faced and how I solved them:

1. The Language Barrier (Romanized vs. Devanagari)

The Problem: LLMs naturally default to writing Hindi in the Latin alphabet (Hinglish) because that's how a lot of their training data looks. However, when you feed Romanized Hindi into an Indian Text-to-Speech engine, the pronunciation often breaks down or sounds unnatural.
The Solution: Heavy prompt engineering. I had to explicitly instruct the agent: "Always write every language in its own native script. Hindi -> Devanagari (नमस्ते), never romanized." This instantly fixed the pronunciation issues.

🛠️ Build Your Own Voice Agent

Want to build your own version of Dukaan Sathi? It's easier than you think. You need four main components:

Speech-to-Text (STT): To transcribe the user's voice.
LLM: The "brain" (e.g., OpenAI or Gemini).
Text-to-Speech (TTS): To generate the agent's voice (Murf Falcon is highly recommended for Indian voices).
Real-time Transport: LiveKit handles the ultra-low latency WebRTC streaming.

Getting Started

Clone the Starter: Start with a LiveKit + Next.js + Python template.
Environment Variables: You'll need API keys for LiveKit, your LLM, and your TTS provider. Never commit these to GitHub! Always store them in a .env.local (frontend) and .env (backend) file.
Run the Backend (Python): Type uv run src/agent.py dev in your terminal.
Run the Frontend (Next.js): Type npm run dev in your terminal.
Test it: Open localhost:3000, click connect, and start talking! You can write a custom SYSTEM_PROMPT in agent.py to give your agent any personality you want.

This 10-day challenge proved that voice AI is no longer a futuristic concept—it is accessible, buildable, and ready to solve real-world problems in India today.

Thanks to LiveKit and Murf AI for providing the incredible tools that made this possible!

Want to see the code behind Dukaan Sathi? Check out the full source code for my 10-day journey on GitHub here: https://github.com/sanyamkansal/AI-Voice-Agent