Hybrid search with pgvector got worse until I added BM25

Disclosure

I wrote PostgreSQL for AI, a book about building this kind of system in Postgres, and hybrid search is in it. Every number below I measured for this post, on public data, and the method is at the end.

SciFact, a public retrieval benchmark, has a test claim that reads "CCL19 is absent within dLNs." CCL19 is a chemokine and dLNs are draining lymph nodes, and one abstract in the corpus answers it. I embedded the claim with nomic-embed-text and asked pgvector for the 100 nearest abstracts. The right one wasn't among them. A BM25 index put it second, because CCL19 is a rare word and that abstract contains it.

Queries like that are why hybrid search exists. You run the vector search and the keyword search and merge the two lists with Reciprocal Rank Fusion, all in one SQL statement. pgvector has no keyword ranking of its own, so the keyword half has to come from somewhere else in Postgres, and that choice decided most of the result.

Most tutorials rank the keyword half with Postgres' built-in ts_rank. On both datasets I tried, fusing that with vector search scored worse than vector search alone. Put a BM25 index in its place and the same fusion scored best of everything.

How the two halves rank

Vector search compares meanings. The question and each document become points in one space, and you get back the nearest ones. That forgives typos and paraphrases, and it has no special idea what a gene name is. To the embedding model a rare identifier is just a few tokens, and an abstract full of related biology can land closer than the one that actually names it.

Keyword search compares words. BM25, the ranking function search engines have used for about thirty years, scores a document higher when it contains the query's words, then adjusts for three things. A rare word counts more than a common one (that is inverse document frequency), and the tenth occurrence of a word adds less than the first, so repetition stops paying. A long document is pulled down a little, since it contains more words by chance.

Postgres' own full-text search is good at matching and bad at ranking. ts_rank and ts_rank_cd only look at how often and how close together the words appear inside one document. Nothing is measured across the corpus, so "the" and "CCL19" weigh the same once stop words are removed. Length only counts if you pass a normalization flag, and that flag divides by this document's length without knowing what's typical for the rest.¹

You can't merge the lists by adding scores, because a cosine distance and a BM25 score are on unrelated scales, and the scale shifts with every query. Reciprocal Rank Fusion only uses positions. A document gets 1 / (k + rank) from each list it appears in, and the sums give the order. The formula comes from a 2009 SIGIR paper by Cormack, Clarke and Büttcher, where k = 60 was fixed in a pilot run and never changed.²

The usual recipe joins the words with AND

The version in most guides has a generated tsvector column with a GIN index next to the embedding, one CTE per half, and a FULL OUTER JOIN that adds up the reciprocal ranks. The keyword half parses the question with websearch_to_tsquery or plainto_tsquery and ranks with ts_rank or ts_rank_cd.³

Both parsers join the words with AND. In a search box, where people type a few words, that's what they expect. With a natural language question, though, a document has to contain every word left after stop words before it matches at all. SciFact's test claims average 12.5 words, and with AND the keyword half returned nothing for 274 of the 300. NFCorpus questions average 3.3 words, and 131 of its 323 still came back empty.

So I measured the recipe twice, once as written and once with the words joined by OR. It's a one line change:

-- OR the lexemes plainto_tsquery produces, so a document needs any of them
SELECT replace(plainto_tsquery('english', $1)::text, ' & ', ' | ')::tsquery;

Fusion with ts_rank scored below vector search alone

The data is two public BEIR datasets with relevance labels: SciFact, with 5,183 scientific abstracts and 300 test claims, and NFCorpus, with 3,633 medical documents and 323 test questions.⁴ nDCG@10 scores how well the top ten is ordered against the labels, where 1 is perfect, and recall@100 is the share of relevant documents found anywhere in the top 100.

Method	SciFact nDCG@10	SciFact R@100	NFCorpus nDCG@10	NFCorpus R@100
ts_rank_cd, AND (websearch_to_tsquery)	0.072	0.073	0.206	0.102
ts_rank_cd, OR	0.330	0.759	0.235	0.225
BM25, pg_textsearch	0.688	0.918	0.325	0.246
BM25, pg_search	0.685	0.921	0.323	0.247
Vector, pgvector HNSW	0.703	0.928	0.347	0.297
RRF, ts_rank_cd (OR) + vector	0.594	0.942	0.324	0.298
RRF, pg_textsearch + vector	0.727	0.958	0.356	0.307
RRF, pg_search + vector	0.735	0.958	0.358	0.307

Fusing ts_rank_cd with vector search scored 0.594 on SciFact, against 0.703 for the vector half alone, and 0.324 against 0.347 on NFCorpus. Recall went up on both, so the fused list does find more relevant documents somewhere in its top 100. But it orders the top ten worse, and the top ten is what a RAG pipeline puts in the prompt.

On its own (the second row), ts_rank_cd scores 0.330 on SciFact, less than half of BM25's 0.688 on the same matches, so its order is a weak signal next to the vector list. RRF weighs both lists equally, and the weak one drags good vector hits down.

With BM25 as the keyword half, fusion helps on both. It scored 0.727 and 0.735 on SciFact, 3 to 5 percent above vector search, and 0.356 and 0.358 on NFCorpus, about 3 percent above. The two BM25 extensions came within a few thousandths of each other, as two implementations of one formula should.⁵

Here's what each method cost per query on SciFact, at the median and the 95th percentile:

Method	p50	p95	Index size	Build
BM25, pg_textsearch	0.21 ms	0.39 ms	3.7 MiB	0.9 s
BM25, pg_search	0.68 ms	1.56 ms	5.3 MiB	0.5 s
Vector, HNSW	1.51 ms	3.11 ms	20.3 MiB	2.3 s
RRF, pg_textsearch + vector	1.63 ms	2.15 ms
RRF, pg_search + vector	2.30 ms	3.67 ms
RRF, ts_rank_cd (OR) + vector	19.88 ms	57.29 ms	4.4 MiB (GIN)	0.2 s

These are server execution times on a warm cache, leaving out the network and the embedding call. The fused BM25 query costs about the same as vector search alone, since its keyword half takes a fraction of a millisecond. The ts_rank version is the slow one. With OR most documents match at least one word, and ts_rank_cd has to score every match before sorting, because the GIN index can find matches but can't rank them.

The query with a BM25 index

The table holds the text and the embedding, plus a generated tsvector for Postgres' own matching in filters. Each BM25 extension adds its own index type on top.

CREATE TABLE docs (
  id        text PRIMARY KEY,
  content   text NOT NULL,
  tsv       tsvector GENERATED ALWAYS AS (to_tsvector('english', content)) STORED,
  embedding vector(768) NOT NULL
);

CREATE INDEX docs_hnsw ON docs USING hnsw (embedding vector_cosine_ops);

pg_textsearch

-- needs shared_preload_libraries = 'pg_textsearch'
CREATE EXTENSION pg_textsearch;
CREATE INDEX docs_bm25 ON docs USING bm25 (content)
  WITH (text_config = 'english');

pg_search

-- needs shared_preload_libraries = 'pg_search'
-- CASCADE also installs pgvector, which pg_search 0.25 depends on
CREATE EXTENSION pg_search CASCADE;
CREATE INDEX docs_bm25 ON docs USING bm25
  (id, (content::pdb.simple('stemmer=english')))
  WITH (key_field = 'id');

The English stemmer on the pg_search index keeps the comparison fair. Without it "runs" won't match "running", and both text_config = 'english' and to_tsvector('english', ...) do stem.

The fused query takes the question's text as $1 and its embedding as $2, both computed once in the application.

pg_textsearch

SELECT coalesce(v.id, k.id) AS id,
       coalesce(1.0 / (60 + v.r), 0) + coalesce(1.0 / (60 + k.r), 0) AS score
FROM (SELECT id, row_number() OVER (ORDER BY d) AS r
      FROM (SELECT id, embedding <=> $2::vector AS d FROM docs
            ORDER BY embedding <=> $2::vector LIMIT 100) s) v
FULL JOIN
     (SELECT id, row_number() OVER (ORDER BY d) AS r
      FROM (SELECT id, bm25_get_current_score() AS d FROM docs
            ORDER BY content <@> to_bm25query($1, 'docs_bm25') LIMIT 100) s) k
  ON v.id = k.id
ORDER BY score DESC, id
LIMIT 10;

pg_search

SELECT coalesce(v.id, k.id) AS id,
       coalesce(1.0 / (60 + v.r), 0) + coalesce(1.0 / (60 + k.r), 0) AS score
FROM (SELECT id, row_number() OVER (ORDER BY d) AS r
      FROM (SELECT id, embedding <=> $2::vector AS d FROM docs
            ORDER BY embedding <=> $2::vector LIMIT 100) s) v
FULL JOIN
     (SELECT id, row_number() OVER (ORDER BY d DESC) AS r
      FROM (SELECT id, pdb.score(id) AS d FROM docs
            WHERE content ||| $1 ORDER BY pdb.score(id) DESC LIMIT 100) s) k
  ON v.id = k.id
ORDER BY score DESC, id
LIMIT 10;

Each half sorts and limits inside its own subquery, so it hands over its best 100, and the row_number() window has its own ORDER BY because SQL promises nothing about the order rows arrive in. The outer SELECT takes the id through coalesce, so a document only one half found still has one, and the final ORDER BY breaks ties on id so every run returns the same list.⁶ The halves are subqueries in FROM and not CTEs, and the next section explains why.

Three things made the same query 15 times slower

My first version of the fused BM25 query took 25.7 ms at the median. The one above takes 1.6 ms and returns the same rows. Three separate problems were in the way, and none of them threw an error.

First, the planner skipped HNSW. On a 5,000 row table Postgres reckoned that reading every vector and sorting was cheaper than walking the graph, and went with that. It took 16.6 ms at the median, against 1.5 ms for the forced HNSW scan. The estimate depends on table size, so a bigger table may well get the index without help. On a small one, in a test or a new product, check EXPLAIN before you trust a latency number, because here it picked a plan eleven times slower.

Second, pg_textsearch scored every row twice. Its README puts the score in the select list as content <@> to_bm25query(...). In 1.5.1 that ran the BM25 index scan and then scored each of the 100 returned rows again from its text, which took 15.5 ms against 0.5 ms for the same query returning only ids. The extension has a planner hook that's supposed to swap that expression for bm25_get_current_score(), a function that reads the score the index scan already computed. In my runs the swap never happened. Calling the function myself got the query down to 0.3 ms. It isn't in the README, so I'd check it still exists after an upgrade.

Third, the CTE was materialized. Postgres folds a CTE into the main query when it's used once and calls no volatile function. bm25_get_current_score() is declared VOLATILE, so a WITH k AS (...) around it stayed a separate step, the planner hook couldn't reach inside, and the 15 ms came back. With the same half written as a subquery in FROM, the whole fused query went from 20.8 ms to 2.2 ms on the first test claim.

All three were visible in EXPLAIN (ANALYZE, BUFFERS) on the fused query. Read that plan before you trust any latency number for hybrid search.

Which BM25 you get depends on where Postgres runs

When I wrote the book's chapter on what's coming next, in February, two teams were building BM25 for Postgres. By October there are at least five options, and where your database runs makes most of the choice for you.

Extension	From	License	Notes
pg_search	ParadeDB	AGPL-3.0	Built on Tantivy. 0.25.11 on 29 Sep 2026. Neon dropped it for new projects on 19 March 2026 and from existing ones on 21 September.
pg_textsearch	Tiger Data	PostgreSQL	1.0 in March 2026, 1.5.1 on 2 Oct 2026. On Tiger Cloud. PG 17 and 18.
VectorChord-bm25	TensorChord	AGPL-3.0 or ELv2	Needs the separate pg_tokenizer extension.
lakebase_text	Neon		Neon's replacement for pg_search on its own platform.
Tin	PlanetScale	not stated	Announced 16 Sep 2026 for PlanetScale Postgres.

pg_search and pg_textsearch both call their access method bm25, so whichever goes into a database second fails with "access method bm25 already exists". They can share a server if each gets its own database, which is how I measured them. Both also need an entry in shared_preload_libraries, so on a managed service you get whichever extension your provider ships, or none, and its extension list is the first thing to look at.⁷

Tuning k moved the score by a hundredth

In RRF, k sets how quickly credit falls off down a list. A small k gives most of the credit to the first few ranks. A large one flattens the curve, so agreement between the two lists counts for more than position. For the pg_textsearch fusion, nDCG@10 by k came out like this:

k	1	10	30	60	100	300
SciFact	0.737	0.738	0.730	0.727	0.727	0.726
NFCorpus	0.356	0.360	0.359	0.356	0.357	0.355

k = 10 beat 60 on both, by 0.011 on SciFact and 0.004 on NFCorpus, small enough that sticking with the default costs little. The choice of keyword half moved the result far more.⁸

Bruch, Gai and Ingber argue that a weighted sum of normalized scores beats RRF once the weight is tuned on a few labeled queries, so I tried it. I min-max normalized each list, tuned the weight on SciFact's 809 training claims and NFCorpus' dev questions, then scored the test sets. It reached 0.737 and 0.360, the same as RRF at k = 10. With ts_rank_cd it did lift the fusion to 0.709 on SciFact, by putting 0.8 of the weight on the vector list. If you have labeled queries, tuning either one gets you to the same place. If you don't, RRF with a BM25 half needs nothing tuned.

Fusion doesn't win on every query either. Against whichever half happened to be better for a given query, which you can't know in advance, the pg_textsearch fusion came out ahead on 25 SciFact claims and behind on 71. On NFCorpus it was ahead on 58 and behind on 125, and below both halves on 14. The fair comparison is against vector search alone, and there it was ahead on 67 SciFact claims and behind on 38, and ahead on 112 NFCorpus questions and behind on 83. That's where the average gain comes from. On SciFact, 11 claims had their abstract missing from the vector half's 100 and found by fusion, at ranks from 7th to 65th. CCL19 was the 7th.

Take one of the losses, "DMRT1 is a sex-determining gene that is epigenetically regulated by the MHM region". pg_textsearch puts the right abstract second, the vector half doesn't have it in its 100, and after fusion it sits 36th. It goes the other way too. One claim misspells metastases as "matasteses", and the vector half puts the right abstract first, but BM25 can't match the misspelled word at all, so fusion drops the abstract to 23rd. A document found by one half gets at most half the credit of one both halves found somewhere, and RRF has no way to tell which half was right for this query.

Where it stops

Both datasets are small, 5,183 and 3,633 documents. I ran everything on a laptop with Docker limited to 1 GB of memory, too little for the 57,000-document FiQA set I'd planned. The quality comparison holds at this size. The latency numbers are for tables that fit in memory, and with millions of rows the time probably goes somewhere else, starting with HNSW cache misses.

I used one embedding model, nomic-embed-text, because it runs locally. A stronger model raises the vector baseline and leaves BM25 less to add, so run the same evaluation with your own model before deciding a second index is worth it.

Both corpora are English, and so is the stemming. For Turkish, Postgres ships a turkish text search configuration and the BM25 extensions take whatever tokenizer you give them. I haven't measured how either copes with Turkish suffixes.

Hybrid search also inherits a filtering problem. If the query also has WHERE tenant_id = $3, an HNSW scan can return fewer than 100 rows after the filter. That's pgvector's most discussed issue, and it needs a post of its own.

How I measured, so you can argue with it

One container ran ParadeDB's paradedb/paradedb:0.25.11-pg18 image (PostgreSQL 18.6, pgvector 0.8.4, pg_search 0.25.11), with pg_textsearch 1.5.1 installed from Tiger's release package. pg_textsearch and pg_search lived in two databases on that server because of the bm25 name clash. The machine is an Apple M2, with 8 CPUs and 1 GB of memory given to Docker. The settings were shared_buffers = 256MB, maintenance_work_mem = 256MB, jit = off and hnsw.ef_search = 200, with HNSW at pgvector's defaults. Every vector number in the tables uses the HNSW index, forced with enable_seqscan = off for those queries, because on tables this small the planner preferred an exact scan.⁹

The data is BEIR's SciFact and NFCorpus as published. Each document is its title and text joined by a newline, embedded once through Ollama 0.34.4 with nomic-embed-text and the search_document: prefix. Questions used search_query:, and I raised num_ctx to 8192 so nothing was truncated. Every method returned its top 100 from one SQL statement per question.

Scores come from pytrec_eval, as nDCG@10 and recall@100 on each dataset's test split, with a query that returned nothing counted as zero. The SQL fusion matched the same RRF computed in Python on 599 of the 600 test questions, and the one difference is a tie broken the other way. Latency is the Execution Time from EXPLAIN (ANALYZE, TIMING OFF), the median of three runs per question after one run to warm the cache, with percentiles taken over the test questions. I measured all of it on 2 and 3 October 2026.

evaluate.py

import pytrec_eval

def score(runs, qrels):
    """runs: {query: [(doc, score), ...]} best first; qrels: {query: {doc: grade}}"""
    as_run = {q: {d: float(len(r) - i) for i, (d, _) in enumerate(r)} for q, r in runs.items()}
    ev = pytrec_eval.RelevanceEvaluator(qrels, {"ndcg_cut.10", "recall.100"})
    res = ev.evaluate({q: r for q, r in as_run.items() if q in qrels})
    n = len(qrels)
    ndcg = sum(res.get(q, {}).get("ndcg_cut_10", 0.0) for q in qrels) / n
    recall = sum(res.get(q, {}).get("recall_100", 0.0) for q in qrels) / n
    return ndcg, recall

def rrf(a, b, k=60):
    out = {}
    for q in set(a) | set(b):
        s = {}
        for lst in (a.get(q, []), b.get(q, [])):
            for i, (d, _) in enumerate(lst):
                s[d] = s.get(d, 0.0) + 1.0 / (k + i + 1)
        out[q] = sorted(s.items(), key=lambda x: (-x[1], x[0]))[:100]
    return out

embed.py

import requests

def embed(texts, prefix):
    r = requests.post("http://localhost:11434/api/embed", json={
        "model": "nomic-embed-text",
        "input": [prefix + t for t in texts],
        "options": {"num_ctx": 8192},
    })
    r.raise_for_status()
    return r.json()["embeddings"]

# documents: embed([title + "\n" + text, ...], "search_document: ")
# questions: embed([text, ...], "search_query: ")

docker

FROM paradedb/paradedb:0.25.11-pg18
USER root
COPY pg-textsearch-postgresql-18_1.5.1-1_arm64.deb /tmp/pgts.deb
RUN dpkg -i /tmp/pgts.deb && rm /tmp/pgts.deb
USER postgres
# run with: -c shared_preload_libraries=pg_search,pg_cron,pg_textsearch

The book has the longer version of this, search and RAG on one Postgres, and there's a free sample chapter if you want to see how it reads first.

---

Originally published at zeybek.dev.

---

1. Tiger's write-up lists the same three gaps. ↩

2. Tiger's hybrid search tutorial and the dev.to example below both still use 60. ↩

3. This dev.to post is a tidy example, and this one does the fusion in Python. ↩

4. Everything ran in one PostgreSQL 18.6 instance with pgvector 0.8.4, ParadeDB's pg_search 0.25.11 and Tiger's pg_textsearch 1.5.1. ↩

5. BM25 alone also lands where the literature puts it, at 0.688 and 0.325 here, against 0.665 to 0.686 for SciFact and 0.325 for NFCorpus in the published BEIR reference runs. ↩

6. In pg_search, ||| matches any of the words, the OR behavior websearch_to_tsquery didn't give. ↩

7. Chapter 12 of the book keeps the same kind of table for pgvector, pgvectorscale, pgai and the other AI extensions, provider by provider. ↩

8. With ts_rank_cd as that half, the best k on SciFact was 1, which is RRF all but ignoring the second list. ↩

9. The exact scan scored the same 0.703 on SciFact and 0.347 on NFCorpus. ↩