The Problem
When building BotForge, our AI no-code chatbot platform, we needed a retrieval system that could handle messy, real-world user queries — typos, partial phrases, semantically similar-but-differently-worded questions.
A naive vector search alone wasn't good enough. It's powerful but brittle to out-of-vocabulary terms and exact keyword lookups.
The Solution: Four-Tier Parallel Retrieval
We ran four retrieval strategies simultaneously using Promise.all\, then merged results with a weighted scoring function.
\javascript
const [semanticResults, textResults, regexResults, fuzzyResults] = await Promise.all([
semanticSearch(query, embeddings), // weight 1.8x
mongoFullTextSearch(query), // weight 1.5x
regexKeywordSearch(query), // weight 1.0x
fuzzyPerWordMatch(query), // weight 0.6x
])
\\
Tier 1: Semantic Search (1.8× weight)
Using Gemini gemini-embedding-2\ to produce 3072-dimensional vectors, we compute cosine similarity against stored document embeddings. This catches meaning — "how do I reset my login?" matches "account recovery options" even with no shared words.
Tier 2: MongoDB Full-Text Search (1.5× weight)
A native MongoDB Atlas text index for fast, exact keyword hits. Great for technical terms, product names, and precise phrases.
Tier 3: Regex Keyword Matching (1.0× weight)
Each significant word in the query is compiled to a case-insensitive regex. Catches partial matches and hyphenated variants.
Tier 4: Fuzzy Per-Word Matching (0.6× weight)
Levenshtein distance matching per query word — handles typos and misspellings like "configuraton" → "configuration".
Weighted Score Merging
Each result carries a base score from its retrieval strategy. We deduplicate by chunk ID, sum scores across strategies, and sort descending:
\javascript
function mergeResults(tiers, weights) {
const scoreMap = new Map()
tiers.forEach((results, i) => {
results.forEach(({ id, score, chunk }) => {
const weighted = score * weights[i]
scoreMap.set(id, {
chunk,
total: (scoreMap.get(id)?.total || 0) + weighted,
})
})
})
return [...scoreMap.values()].sort((a, b) => b.total - a.total).slice(0, 5)
}
\\
Results
- ~90% retrieval accuracy on held-out test queries
- ~780ms median latency end-to-end (including embedding generation)
- Robust to typos, paraphrasing, and domain-specific terminology
The parallel architecture was the key insight — running all four strategies concurrently keeps latency close to the slowest individual strategy (semantic search) rather than multiplying it.
What I'd Do Differently
- Cache embeddings for frequently asked questions
- Add a re-ranking step (cross-encoder) for the top 10 candidates
- Explore ColBERT-style late interaction for finer-grained scoring
Top comments (1)
Parallel RAG only helps if the merge layer is honest about disagreement. The most useful systems expose which branch found what, where sources conflict, and when the final answer is stitched from partial evidence.