DEV Community

Mads Hansen
Mads Hansen

Posted on

Your pgvector benchmark is incomplete until you add real access filters

A pgvector query can be fast and accurate on the full corpus, then lose relevant results when you add a tenant or business filter.

That does not mean the authorization filter is wrong.

It means retrieval quality must be measured inside the authorized subset.

Build an exact filtered baseline and compare ANN results using:

  • recall@k
  • empty and underfilled result rate
  • p95 latency
  • rank movement
  • exact-fallback rate

Report by filter selectivity. A global average hides tiny tenants and rare categories—the cases most likely to lose candidates.

Pin the embedding model, dataset snapshot, distance metric, PostgreSQL/pgvector versions, index parameters, and query settings. Test rebuilds, revocations, concurrent writes, and model upgrades.

For small eligible subsets, an exact distance scan may be both simpler and more accurate. For larger subsets, use ANN with a measured, bounded candidate budget.

Access controls are not a retrieval tuning knob. Keep them fixed and make the search strategy prove it works behind them.

Full guide: pgvector filtered recall tests for AI database search

Top comments (0)