OmniRAG — A Knowledge Engine That Handles Anything You Throw at It

I got tired of the standard RAG demo that only works on one PDF and falls apart the moment you throw anything more complex at it. OmniRAG is what I built to fix that .

It handles PDFs, images, codebases, audio, and video — all in the same pipeline. You upload your files, ask questions, and get answers with citations that tell you exactly where each piece of information came from.

The part I spent the most time on is the search layer. Most RAG systems use pure vector search, which works okay but misses things that don't embed well — exact identifiers, code symbols, specific numbers. OmniRAG uses hybrid search: vector embeddings for semantic meaning and BM25 keyword search for precision.

🛠️ What it handles:
PDFs — Full text extraction with chunking, table parsing, and metadata. Works on research papers, contracts, and reports.

Images — OCR for text in images plus vision AI for understanding diagrams, charts, and screenshots.

Code Repositories — AST-aware indexing so it understands structure, not just text.

Audio and Video — Transcribes audio using Whisper, then indexes the transcript.

Web Pages — Point it at a URL and it scrapes the content into your knowledge base.

Multi-Turn Conversation — Maintains conversation memory, so follow-up questions work.

Citations — Every answer includes source references with document name and page or timestamp .

⚙️ How it works
When you upload a file, it goes through a routing step that figures out what type it is and sends it to the right parser. The output gets chunked, each chunk gets an embedding, and everything lands in ChromaDB. The BM25 index is built in parallel. When you ask a question, both indexes get queried simultaneously, results are merged and reranked, and the top chunks are passed to the LLM.

🧰 Tech Stack
Python, FastAPI, ChromaDB, PostgreSQL, Redis, Whisper, and optional Ollama for local AI.

🔗 Get Started
Check out the repo: github.com/isidhartha/omni-rag