DEV Community

Cover image for BM25 vs Dense Retrieval vs SPLADE: Which One to Start With
Basavaraj SH
Basavaraj SH

Posted on

BM25 vs Dense Retrieval vs SPLADE: Which One to Start With

#rag #llm #python #embeddings

Choosing the wrong retrieval method at the start of a RAG project costs weeks. Here's how the three main approaches actually compare on a constrained machine - before you commit to infrastructure.

The Core Difference Between These Three Methods

BM25 is a keyword-scoring algorithm - it ranks documents by term frequency and how rare those terms are across your corpus. No model weights, no GPU needed, runs fast on CPU. Dense retrieval uses an embedding model (like all-MiniLM or bge-base) to encode queries and documents into vectors, then finds the closest match by cosine similarity. It catches semantic matches BM25 misses ("affordable" matching "cheap") but needs more memory and compute.

SPLADE (Sparse Lexical and Expansion model) falls between the two: it produces sparse vectors - mostly zeros, like BM25 - but uses a transformer to expand query terms, so it handles synonyms without the memory overhead of storing millions of dense floats. It's the least commonly benchmarked of the three, which is exactly why it gets skipped - and often shouldn't be.

Real Example

Here's a minimal setup to run all three against the same query using rank_bm25, sentence-transformers, and splade (via a HuggingFace checkpoint):

from rank_bm25 import BM25Okapi
from sentence_transformers import SentenceTransformer, util

corpus = ["cheap flights to Paris", "budget travel Europe", "Paris airfare deals"]
query = "affordable Paris flights"

# BM25
tokenized = [doc.split() for doc in corpus]
bm25 = BM25Okapi(tokenized)
bm25_scores = bm25.get_scores(query.split())

# Dense retrieval
model = SentenceTransformer("all-MiniLM-L6-v2")
doc_embeds = model.encode(corpus, convert_to_tensor=True)
query_embed = model.encode(query, convert_to_tensor=True)
dense_scores = util.cos_sim(query_embed, doc_embeds)[0]

print("BM25:", bm25_scores)
print("Dense:", dense_scores)
Enter fullscreen mode Exit fullscreen mode

BM25 scores "cheap flights to Paris" highest because "flights" and "Paris" are exact matches. Dense retrieval surfaces "budget travel Europe" higher because "affordable" semantically aligns with "budget." SPLADE would expand "affordable" to include "cheap" and "budget" at query time - splitting the difference without a full embedding index.

On a 16GB machine, BM25 runs instantly on a 100K document corpus. Dense retrieval with all-MiniLM stays under 4GB RAM for similar corpus sizes. SPLADE checkpoints (naver/splade-cocondenser-selfdistil) load around 500MB but require careful batch sizing to avoid OOM errors.

Key Takeaways

  • Start with BM25 as your retrieval baseline - it's fast, interpretable, and often harder to beat than expected
  • Add dense retrieval when your queries and documents use different vocabulary (semantic gap is the real test)
  • Try SPLADE before assuming you need a full hybrid pipeline - it often closes that gap with less infrastructure
  • OOM crashes with SPLADE are almost always a batch size problem; drop batch_size to 8 or 16 first

What retrieval method are you using in production right now, and have you actually benchmarked it against BM25 on your own dataset?

Sources referenced: Towards Data Science - "How I Reproduced BM25, Dense Retrieval, and SPLADE on a 16GB MacBook"

Top comments (0)