The worst failure mode of a retrieval-augmented generation system is not an error. It's a fluent, confident answer the documents never supported. Nothing crashes, nothing logs, and the reader has no way to know. That's the problem I built RAG.NextUpgrad to solve, and it's still the feature I'm proudest of in it.
The constraint that shaped everything
This had to run on a free-tier host. That instance gives you 512 MB of memory. Loading torch plus sentence-transformers for local embeddings alone needs roughly 500 MB, so I switched to hosted embeddings and the whole app now runs around 220 MB. No GPU, no paid tier — anyone can actually try it.
Retrieval: dense and sparse, fused
Pure vector search is good at meaning and bad at exact strings — product names, error codes, a specific figure. So retrieval runs two searches at once: dense embedding search and BM25 keyword search, merged with Reciprocal Rank Fusion. No score normalisation needed between the two systems, which is exactly why RRF is the right first choice.
The fused list is a candidate set, not an answer. A cross-encoder reranker reads the query and each candidate together and re-scores them — far more precise than the first-pass bi-encoder, far too slow to run over the whole corpus, so it only sees the top of the fused list.
The gate: when to say nothing
After reranking, the top result has a similarity score. If it doesn't clear a configured threshold, the language model is never called. The API returns a fixed string: "I don't know based on the provided documentation."
- The gate sits before generation, not after. Asking a model "are you sure?" after it already answered is asking the system that hallucinated to grade itself.
- The refusal is deterministic. Same string every time, so the frontend can detect it and the logs can count it.
- The threshold is a product decision, not a constant — set it too high and paraphrased questions fail, too low and you're back to confident nonsense.
When the gate passes, generation defaults to Groq-hosted Llama 3.3 70B, with Anthropic and OpenAI behind the same LLM port as a config switch — every provider also has an automatic fallback, so a rate-limited API doesn't kill an ingestion run mid-job.
Everything else that had to be true
Scanned PDFs get OCR'd automatically via Tesseract, page order preserved, with live per-page progress. Retrieved text is wrapped in tags that tell the model it's untrusted reference data, never instructions — that matters the moment you allow URL ingestion, which this does. Structured JSON logs, Prometheus metrics, and rate limits are in from day one, not bolted on.
If you're building your own
Build the evaluation set before the pipeline. I tuned retrieval by eye for longer than I should have — a few dozen question-answer pairs with known source chunks, scored on every change, would have found the right threshold in an afternoon instead of a week of guessing.
Code is open at github.com/Pranjulrathour/RAG.NEXTUPGRAD. If something in the pipeline looks wrong to you, open an issue.
Pranjul Rathour · GenAI engineer, 3x hackathon winner, campus mentor. Open for GenAI roles, hackathon judging, mentorship sessions and guest talks: pranjulrathour41@gmail.com · Invite me to your campus
Portfolio & blog · LinkedIn · Bluesky · GitHub · Dev.to
Top comments (0)