DEV Community

Devanshu Biswas
Devanshu Biswas

Posted on

RAG-Fusion: Ask the Question Many Ways, Then Fuse the Ranks (RRF)

#ai #rag #llm #beginners

One way of phrasing a question is one roll of the dice — and vector search can systematically miss the best document because of it. RAG-Fusion fixes that: have the LLM ask the question several ways, retrieve for each, then fuse the rankings. Here's the whole technique, including the elegant little formula at its heart.

🔀 Interactive demo: https://dev48v.infy.uk/prompt/day10-rag-fusion.html

The problem with a single query

Embeddings retrieve whatever's nearest to your exact phrasing. But the doc that answers you might be worded like a different phrasing. So a single query has blind spots it can't see past.

Step 1: Let the LLM rewrite the query 

const queries = await llm(`Generate 4 different search queries for:
"${userQuery}". Vary the wording and angle. One per line.`);
Enter fullscreen mode Exit fullscreen mode

"Why am I tired?" becomes "causes of fatigue," "low energy reasons," "poor sleep effects" — each probing a different region of your vector space.

Step 2: Retrieve a ranked list for each 

const lists = await Promise.all(
  queries.map(q => vectorSearch(q, { k: 10 }))
);
Enter fullscreen mode Exit fullscreen mode

Now you have several ranked lists. The genuinely good doc often appears in many of them — that repetition is the signal.

Step 3: Reciprocal Rank Fusion (the magic)

RRF is beautifully simple. Each doc earns 1/(k + rank) from every list it appears in (k is usually 60, rank is 0-based position). Sum across lists:

const scores = {};
for (const list of lists)
  list.forEach((doc, rank) => {
    scores[doc.id] = (scores[doc.id] || 0) + 1 / (60 + rank);
  });
Enter fullscreen mode Exit fullscreen mode

A doc that ranks decently across several lists beats a doc that ranks #1 in just one. It rewards consensus.

Step 4: Sort, keep the top few, answer 

const fused = Object.entries(scores)
  .sort((a, b) => b[1] - a[1])
  .slice(0, 4)
  .map(([id]) => byId[id]);

return llm(`Answer using ONLY these:\n${fused}\nQ: ${userQuery}`);
Enter fullscreen mode Exit fullscreen mode

Why RRF instead of averaging scores?

Different queries return similarity scores on different scales — averaging them is apples-to-oranges. RRF ignores the messy scores and uses only rank, which is comparable across any retriever. That's also why it can fuse totally different systems (vector + keyword + reranker) into one list.

The takeaway

Ask many ways → fuse the ranks → the consensus answer wins. For the cost of a few extra cheap searches, retrieval gets markedly more robust. Play with the demo — watch the true answer climb as the rewrites vote it up.

Top comments (0)