Same query.
Same documents.
Same model.
And the RAG eval can still hand back a different Recall@8.
Not because the model is flaky. Because of an ORDER BY clause.
I didn't find this by watching a metric wobble. I found it reading the retrieval code, and realized the score would drift run to run even if the model never changed.
This came out of a habit I've adopted recently: I write the eval before the feature. Reviewing the retrieval pipeline behind my "Ask this Book" feature, I saw it: the retrieval layer wasn't deterministic.
Order isn't presentation. It's part of the input.
My RAG implementation is intentionally simple: plain PostgreSQL and .NET. Two retrieval strategies over the same table:
- semantic search using pgvector
- lexical search using PostgreSQL full-text search
The results are merged with Reciprocal Rank Fusion (RRF).
Here's the important part: RRF doesn't care about the retrieval scores. It only cares about rank.
If one retriever returns
A
B
C
instead of
B
A
C
RRF produces different fused scores. Different fused scores mean a different Top-K. Different Top-K means different Recall@K.
In RRF, order isn't a display detail. Order is data.
The bug
My lexical query ended like this:
ORDER BY score DESC
Looks perfectly reasonable. Except ts_rank_cd produces ties surprisingly often. Multiple chunks can have exactly the same score.
And SQL only guarantees the ordering you explicitly request. If multiple rows compare equal, PostgreSQL is free to return them in any order.
Nothing changed. Same database. Same query. Same model. Only the order of equally-ranked rows. Yet that's enough for RRF to assign different ranks, producing different fused scores and a different evaluation result.
The semantic retrieval had the same issue. Distance ties are much rarer than lexical ties, but "rare" isn't good enough for an evaluation pipeline.
The fix
The fix was almost embarrassingly small.
Before:
ORDER BY score DESC
After:
ORDER BY score DESC, id
A deterministic tie-breaker on both retrieval queries. Now equal-scoring rows always appear in the same order, RRF receives the same input every run, and the Top-K stays identical.
Notice what didn't happen. The retrieval didn't become better. It became reproducible.
Why this matters
We spend a lot of effort making the model deterministic during evaluation: temperature 0, fixed datasets, golden answers, reproducible prompts.
But it's easy to assume everything underneath the model is already deterministic. Often it isn't. Retrieval. Ranking. Sampling. Data loading. Any non-deterministic stage in the pipeline can quietly invalidate your eval.
A fluctuating eval isn't just annoying. It's dangerous. Eventually you stop trusting the number, even when it's pointing at a real problem.
The lesson I took away
Before debugging the model, debug determinism. An evaluation can only be as deterministic as the pipeline feeding it. Same query. Same rows. Same order. Only then can you trust what your eval is telling you.
I build TextStack, an open-source reader for technical books, in .NET. This is from the retrieval layer behind its "Ask this Book" feature. Code on github.com/mrviduus/textstack.
Top comments (21)
the tie-break fix is right for exact search, but it papers over a second source that shows up the moment you scale: the ANN index itself. pgvector on a full scan is deterministic, but switch to HNSW or IVFFlat for speed and the candidate set becomes approximate, so your Top-K membership can drift run to run even with a perfect tiebreaker, because the recall set changed, not just the order. a deterministic ORDER BY stabilizes ranking, it doesn't stabilize recall once retrieval is approximate. for the eval pipeline i'd pin the index build, or use exact search for eval, so you're measuring the model and not ef_search variance. otherwise the drift comes back at scale wearing a different hat.
You got me here. I checked the code, my lexical leg is exact, but the vector leg runs over an HNSW index. So your point applies to me directly, not in theory. The per-book corpus is small, so for eval runs I can afford exact search and keep HNSW for production. Adding that to the follow-up list, thanks.
exact-for-eval, HNSW-for-prod is the clean split. one nice bonus once you have it: run the same query set through both and the delta IS your recall@k gap. the flat index gives you ground truth, so you can actually measure how much recall HNSW is trading away at your ef_search setting instead of guessing. then the eval catches two separate things: whether your retrieval logic is right (flat), and whether your ANN params are tuned tight enough (flat vs HNSW). cheap to track per release since the corpus is small.
The flat vs HNSW delta as a free recall@k gap metric is a great point. Two checks from one eval set. Stealing this for the eval suite, thanks.
Non-deterministic retrieval is a brutal source of false confidence. If the candidate set keeps changing, the eval is measuring a moving system, not just answer quality. Pinning retrieval conditions and logging the retrieved documents should probably be part of every serious RAG eval run.
Agreed. Mads suggested the same above: log a retrieval fingerprint with every run, index version, top-k ids, scores. Logging the retrieved set turns "the eval moved" into "here is what changed", and that is most of the debugging battle.
Yes. The retrieval fingerprint turns a vague trust problem into a diff. Once you can compare index version, ids, scores, and query shape, “the eval changed” stops being mysterious and becomes something the team can actually debug.
Silent non-determinism has a cousin: silent truncation. Ollama clamps the context window to the GGUF's native size without any error — my 32B was capped at 40960 despite requesting 65536, so session compression was failing invisibly and outputs looked flaky when they were actually being truncated. Before blaming retrieval order, I'd add one check to the list: verify the context you asked for is the context you actually got.
Good addition. Silent truncation looks the same as flaky retrieval from the outside, you blame the wrong layer. Adding a check that the context I asked for is the context I actually got. Thanks!
The tie-break is right, and the trap is that it has to ride through every ranking stage, not just the two leaf retrievers. RRF re-introduces the same non-determinism at the fusion output: with the usual k around 60 the per-list contributions flatten, so equal fused scores at the Top-K boundary are common, and a plain ORDER BY fused_score DESC will reorder those ties run to run even after both inputs are deterministic. So the deterministic id, or a stable doc key, needs to carry into the final fused sort, not stop at the retrievers. It is worth checking whether docs that appear in one list versus both are what produce your boundary ties, since the k constant makes those collisions likely. I hit this implementing RRF over hybrid retrieval and wrote the version I settled on here: gist.github.com/renezander030/41af...
Good catch. I went and checked my fusion code. I got lucky: the fused sort is a stable sort and ties keep first-seen order, so that step is deterministic. But only because the leaf lists arrive in a fixed order, the query-level tie-break still carries everything. Thanks for the gist, I will take a look.
This is the kind of RAG failure mode that looks like an eval problem until you trace the retrieval layer carefully.
I like separating three sources of instability:
If those are mixed together, a failing eval tells you almost nothing. You need to know whether the answer changed because the model reasoned differently, because different chunks were retrieved, or because the same chunks arrived in a different order.
For production systems, I’d also log the retrieval fingerprint with every answer: index version, embedding model, query rewrite, filters, top-k ids, scores, and reranker version.
Without that, “RAG quality” becomes too vague to debug.
Yes, for me it was only the third one. Same IDs, same scores, just a different order.
Your fingerprint idea is good. If I log the top-k IDs per run, this bug is obvious right away. I will add it thanks.
The tie-breaker catch is the right one. One thing worth adding though: deterministic is not the same as correct. ORDER BY score DESC, id makes both runs agree, but they can now agree on a worse tied doc, the one that happens to sort first by id. The eval goes green while a resolution problem in your scorer just gets frozen in place. The number I actually watch is how often a Top-K boundary was decided by the tie-breaker instead of the score. If that's high, the fix isn't the tie-breaker, it's that your scores don't separate the candidates.
And the SQL fix stops at the query. On an HNSW or IVF index the candidate set itself can shift between rebuilds or under concurrency, before any ORDER BY runs, so the id tie-breaker won't save you there. Cheapest way to catch either without reading code: run each eval query K times and diff the Top-K id sets. Identical inputs returning different ids means retrieval is non-deterministic, and no amount of model debugging will settle that score.
Good point. My tie-breaker makes runs equal, but it can freeze a worse doc and the eval stays green. I like your metric: how often the top-K boundary is decided by the tie-breaker, not the score. Adding it to my eval report. And you are right about ANN, I checked: my lexical leg is plain SQL, but the vector leg runs on HNSW, so the id trick alone will not save me there.
Building on the point Vinicius raised about the tie-break freezing a worse doc: the thing I keep coming back to is that
, idgives you a stable order but an arbitrary one, so the eval is now reproducibly maybe-wrong instead of randomly maybe-wrong. Curious whether you'd consider making the eval loud about it rather than just stable. Something like counting how many Top-K boundaries were settled by the id tiebreak instead of by the score, and failing the run if that number climbs. That way a green eval means the ranking actually decided the result, not the row order. The whole "order is data, not presentation" reframing is the part of this that'll stick with people.I like "reproducibly maybe-wrong instead of randomly maybe-wrong", that is exactly the trade. And making it loud fits how I run evals: every number I track eventually becomes a gate in CI. So yes: tie-break-decided boundaries as a counter first, then a threshold that fails the run. If the counter climbs, the problem is in the scorer, not in the sort. Adding it, thanks.
This reframes the problem exactly right for me. I chased "flaky" eval scores for weeks before realizing the retrieval layer was reordering chunks on every call, so the judge was scoring a different context each run. Do you pin the retrieved set when you eval, or accept the variance and run enough samples to see through it?
I went the third way: make retrieval itself deterministic, then there is nothing to pin. Tie-breaker in the query plus a stable fusion sort, and the judge sees the same context every run. Pinning would hide the bug, sampling would just pay for it on every run.
Some comments may only be visible to logged-in visitors. Sign in to view all comments.