Optimizing MongoDB Hybrid Search with Reciprocal Rank Fusion

Over the last couple of weeks, I've been exploring ways to improve search relevancy using MongoDB's hybrid search capabilities. In my previous post about understanding search scores (https://dev.to/shannonlal/understanding-search-scores-in-mongodb-hybrid-search-4lnb), I discussed how MongoDB handles scoring in both vector and text searches. Today, I want to share my experiments with Reciprocal Rank Fusion (RRF), a technique that offers a more systematic approach to combining and ranking results from different search methods.

The Challenge of Relevance in Hybrid Search

Hybrid search, which combines traditional text-based search with vector similarity search, offers a powerful way to find relevant results. However, balancing the results from these two different search methods can be tricky. How do we ensure that a highly relevant vector search result isn't overshadowed by a less relevant but keyword-matched text search result, or vice versa?

Enter Reciprocal Rank Fusion

Reciprocal Rank Fusion is a method originally developed for combining results from different search engines. It works by assigning scores to results based on their rank in each list, rather than their raw relevance scores. The basic principle is simple:

For each result in each list, calculate a score: 1 / (rank + k), where k is a constant (often set to 60).
Sum these scores for each unique result across all lists.
Rank the final list based on these summed scores.

The promise of RRF in hybrid search is that it can potentially provide a more balanced set of results, giving appropriate weight to both text and vector search outputs.

RRF in Action
To test RRF in MongoDB, I implemented a hybrid search query using the aggregation pipeline. Here's a simplified version of the key parts:

const textWeight = 0.9;
const vectorWeight = 1 - textWeight;

[
  // Vector Search
  {
    $vectorSearch: { /* ... */ },
  },
  {
    $addFields: {
      vs_score: {
        $multiply: [
          vectorWeight,
          { $divide: [1.0, { $add: ['$rank', 60] }] },
        ],
      },
    },
  },
  // Text Search
  {
    $unionWith: {
      coll: 'ai_generated_image',
      pipeline: [
        { $search: { /* ... */ } },
        {
          $addFields: {
            text_score: {
              $multiply: [
                textWeight,
                { $divide: [1.0, { $add: ['$rank', 60] }] },
              ],
            },
          },
        },
      ],
    },
  },
  // Combine and rank
  {
    $addFields: {
      combined_score: { $add: ['$vs_score', '$text_score'] },
    },
  },
  { $sort: { combined_score: -1 } },
]

This query performs both vector and text searches, applies RRF scoring to each, and then combines the results based on their RRF scores.

Uncovering the Limitations of RRF

While RRF showed promise in balancing our hybrid search results, it also revealed some limitations:

Fixed Weights: The constant weights for text and vector searches (textWeight and vectorWeight) don't adapt to the quality of results in each specific query. This can lead to suboptimal rankings when one search method significantly outperforms the other.

Rank Over Relevance: RRF focuses on the rank of results rather than their actual relevance scores. This can sometimes lead to less relevant results being ranked higher simply because they appeared in both search methods.

For example, consider a collection with these entries:

"A parked red Ferrari in a parking lot"
"A red Ferrari driving by the ocean"
"A red rose in a garden"

A search for "red sports car" might include the third entry in the results due to the text match on "red", even though it's clearly less relevant.

Conclusion and Next Steps

Implementing RRF in MongoDB has provided valuable insights into optimizing hybrid search. While it offers a straightforward way to combine different search methods, its limitations suggest that a more adaptive approach might be necessary for truly optimized results.
Moving forward, I am going to be exploring how these potential improvements:

Dynamic Weighting: Adjust the weights of text and vector searches based on the quality of their results for each query.
Score-Aware Fusion: Incorporate the actual relevance scores from each search method, not just their ranks.
Semantic Filtering: Use vector similarity to filter out semantically irrelevant results from the text search before fusion.

As we continue to push the boundaries of search relevance in MongoDB, experiments like this RRF implementation provide crucial stepping stones. The journey to perfect hybrid search continues, and I'm excited to see where these insights will lead us next.