DEV Community

Zusu12
Zusu12

Posted on

🚀 Building My First Hybrid RAG Agent: A CS Student's Journey into AI

#ai #rag #learning #beginners

🚀 Built a Hybrid RAG Agent — Here's What I Learned

Hey DEV's! 👋

I'm a Computer Science student who's been diving deep into AI lately. Instead of just watching tutorials, I decided to build something real — a Hybrid RAG Agent that retrieves information from documents and generates accurate answers.

🤔 The Problem I Was Solving

Most beginner RAG projects use only vector search for retrieval. But here's what I discovered:

  • Vector search is great for semantic meaning but misses exact keyword matches
  • Keyword search (BM25) catches exact terms but doesn't understand context
  • Single-method retrieval = answers that are either too vague or miss the point entirely

The solution? Hybrid RAG — combining both approaches with intelligent reranking.

🏗️ Architecture 

User Query
    │
    ▼
┌─────────────────────────────────┐
│ Hybrid Search Retrieval Engine  │
│ ├─ Semantic (ChromaDB + Cosine) │
│ └─ Lexical (BM25 Keyword Match) │
└────────────────┬────────────────┘
                 │ Top-K Candidates
                 ▼
┌─────────────────────────────────┐
│     Cross-Encoder Reranking     │ ← Evaluates full passage-query relevance
└────────────────┬────────────────┘
                 │ Top-3 Highest Scoring Chunks
                 ▼
┌─────────────────────────────────┐
│      LLM Generation Context     │ ← Appends conversation history memory
└────────────────┬────────────────┘
                 │
                 ▼
          Answer + Sources
Enter fullscreen mode Exit fullscreen mode

🛠️ Tech Stack

  • Backend: Python + Flask
  • Vector DB: ChromaDB (persistent)
  • Embeddings: sentence-transformers (all-mpnet-base-v2)
  • Keyword Search: BM25Okapi
  • LLM: Hugging Face API (Qwen2.5-7B / Mistral / Llama-3.1)
  • Reranker: ms-marco-MiniLM-L-6-v2 (cross-encoder)
  • Frontend: Custom HTML/CSS/JS with glassmorphic dark UI

Features:

  • Multi-format docs (PDF, TXT, DOCX, Markdown)
  • Stateful conversation memory
  • Persistent storage (survives restarts)
  • Source citations in responses

🎯 How It Works

1. Document Ingestion

  • Split into chunks (500 chars, 50-char overlap)
  • Embed with sentence-transformers
  • Store in ChromaDB + build BM25 index

2. Query Processing

  • Vector search finds semantically similar chunks
  • BM25 finds exact keyword matches
  • Both run in parallel

3. Cross-Encoder Reranking

  • Combines results from both searches
  • Cross-encoder scores each query-chunk pair for relevance
  • Sends only top-3 most relevant chunks to LLM
  • This is the highest-ROI improvement for RAG systems

4. Response Generation

  • LLM generates answer with context + conversation history
  • Returns grounded response with source citations

🎯 Challenges I Faced

  • Vector-only missed exact terms → Added BM25 for keyword matching
  • Too many irrelevant chunks → Cross-encoder reranking fixed this
  • Lost conversation context → Added stateful memory for multi-turn queries
  • Not production-ready → Added persistent storage, health checks, clean API

🚀 What's Next

  • Add query expansion and HyDE for better recall
  • Implement evaluation metrics (precision@k, recall@k)
  • Add metadata filtering (date ranges, doc types)
  • Deploy to Railway/Render with CI/CD

🙏 Let's Connect!

  • What AI projects are you working on?
  • Tried hybrid search or reranking in your RAG?
  • Want to collaborate on open-source AI?

Drop a comment below! Let's build together. 🚀

Tags: #ai #rag #machinelearning #python #artificialintelligence #webdev #beginners #students

Top comments (0)