Filtered vector search in pgvector can silently lose recall

Varun Sharma — Mon, 20 Jul 2026 22:19:23 +0000

Why WHERE tenant_id = 42 ORDER BY embedding <-> $query LIMIT 10 can return fewer correct neighbors than you think - and how to know when it's happening to you.

Almost every RAG app eventually writes a query like this:

SELECT id, chunk
FROM documents
WHERE tenant_id = 42
ORDER BY embedding <-> $query
LIMIT 10;

Give me the ten closest chunks to this query vector - but only for tenant 42. It looks harmless. Postgres returns rows, the app moves on.

Here's the problem: those rows might not be the ten nearest filtered neighbors. Sometimes you may not even get ten rows back. And nothing - not an error, not a warning, not the query plan - tells you that recall collapsed.

I spent the last few weeks measuring exactly when this happens and built a small tool to catch it. This is what I found.

What actually runs

When Postgres chooses an HNSW index scan for ORDER BY embedding <-> $query LIMIT k, pgvector searches the vector index first and applies your WHERE clause afterward. That ordering matters more than it sounds.

HNSW is an approximate index. It walks a graph and keeps an unfiltered search frontier - by default, hnsw.ef_search = 40. That frontier is built before the SQL filter is applied. Then Postgres throws away the candidates that don't match tenant_id = 42.

So the real question isn't "how rare is tenant 42 in the whole table?" It's "how many candidates near this query vector belong to tenant 42?" If too few matching rows survive that frontier, you simply can't get 10 good filtered neighbors from the fixed-frontier scan. pgvector returns what survived the filter, and recall quietly drops.

Why doesn't the planner warn you? Because PostgreSQL's planner is costing execution work, not semantic recall. It can prefer a plan that looks cheap while returning too few correct filtered neighbors, because recall is not part of the cost model.

I measured it at a million rows

To make this concrete, I ran a benchmark on one million real SIFT vectors in Postgres 17 / pgvector 0.8.4. The filter matches 5% of the table globally, but is arranged so that for each query, none of its exact top-40 nearest neighbors pass the filter. That's the adversarial case - a filter that's common overall but absent right where you're searching. (If you've ever had a tenant whose documents live in a different region of embedding space than the query, you've met it.)

Same query, four ways of running it:

Strategy	recall@10	returned all 10 rows?
post-filter (the default)	0.34	only 25% of queries
exact scan	1.00	100%
iterative scan	1.00	100%
partial index	0.86	100%

The default plan returned about a third of the correct results, and for three out of four queries it couldn't even find ten matching rows. Every alternative improved quality: an exact filtered scan and pgvector's iterative scan (added in 0.8.0) both recovered full recall, and a partial HNSW index built for this filter was actually the fastest of the four while still hitting 0.86.

The correct plans weren't some heavy tax you pay for safety - the default was simply the wrong pick here, and the information to know that was right there. The planner just doesn't use it.

To be clear, this is a deliberately hard case, chosen to make the failure visible. With a friendly filter - one that's dense near your queries - post-filtering is fast and perfectly fine. And that's the whole point: whether the default is great or terrible depends on something Postgres never looks at.

Global selectivity is the wrong number

The thing Postgres never looks at is what I've come to call local (or frontier) selectivity: the fraction of your query's actual near-neighbors that pass the filter - not the fraction of the whole table.

Those two numbers can be wildly different. A tenant might be 0.1% of your rows but 100% of the neighbors for their own queries, because their documents cluster together. Or 5% of your rows and 0% of the neighbors for a query that lands elsewhere. Global selectivity - the number Postgres estimates - tells you almost nothing about which case you're in. What decides whether post-filtering holds up is the correlation between your filter and your vector space, and that's exactly the signal the cost model is missing.

Once you see it that way, the fix isn't "post-filtering is bad." It's "the right strategy depends on local selectivity, so someone should actually measure it."

VecAdvisor

So I built VecAdvisor - a small, read-only CLI that does the measuring.

You point it at your database and a query, and it:

reads your table statistics and reproduces Postgres's own global selectivity estimate,
runs one cheap probe - a single count(*) FILTER (...) over your query's nearest-neighbor frontier - to measure local selectivity where the query actually lands,
costs every strategy (exact, post-filter, iterative, partial index, partitioning) with a calibrated model, and
tells you which one is recall-safe, and why.

It never touches your planner or your data. It's an advisor, not a patch. A run reads like this:

EXPLAIN VECTOR  documents (1M rows, HNSW)
  filter: tenant_id = 42
  s(global) = 0.05    s(local) = 0.00   <- filter is sparse near this query

  > EXACT / iterative     recall ~1.00   returns full k? yes   RECOMMENDED
    post-filter (ef=40)   recall ~0.34   returns full k? no    <- the planner's default

  WHY: only ~0 of the nearest-neighbor frontier matches the filter, so post-filter
       cannot return 10 good rows. Use iterative scan, a filtered index, or tune ef_search.

I want to be honest about where it stands. In my measured sweep, VecAdvisor recommended a plan that met the recall target in every case where the default post-filter failed - 9 out of 9. It picked the exact fastest safe plan in 4 of those 9; in the rest it chose a safe plan that was a little slower than the absolute optimum.

"Never quietly lose recall" is the promise for this first version; "always pick the latency-optimal plan" is calibration work I'm still doing. I'd rather ship something that's honestly always safe, usually fast than pretend it's perfect.

Try it

pip install vecadvisor

It's alpha, Apache-2.0, and not affiliated with the pgvector project - it just works alongside it. The repo has the full benchmark (reproducible from a fresh checkout), the cost model, and the design notes.