DEV Community

Discussion on: Why RAG breaks on Persian text, and what I built to fix it: normalization, half-spaces, spelling variants, and a regression bench

Collapse
 
ahmetozel profile image
Ahmet Özel •

Weighting the original query at 2.0 is a useful preference, but it does not guarantee that exact matches stay ahead. Several highly correlated spelling or spacing variants can each contribute another RRF vote to the same competing document, eventually outweighing the original query's contribution.

Since matched_queries already records the provenance, I would inspect score contributions by variant family and test queries with many equivalent rewrites. Giving the expansion family a fixed total weight, or capping its contribution per document, would stop the number of generated variants from becoming an accidental ranking parameter.

Collapse
 
sadrmanesh profile image
Naser Sadrmanesh •

Fair point, and you're right about the current code. The original query gets 2.0, but each variant gets a flat 1.0 with no cap, and with max_variants=6 up to five correlated rewrites can together outvote the original for a competing chunk. So the number of variants really is a hidden ranking parameter right now. Normalizing the variant family to a fixed total weight (e.g. splitting 1.0 across variants) is a clean fix, and I'd like to test it on queries with many equivalent rewrites. matched_queries only lists which variants hit, so I'll add per-variant contributions too. Thanks for the careful read!

Collapse
 
sadrmanesh profile image
Naser Sadrmanesh • • Edited

Normalize RRF weight across the query-variant family
Raised in a DEV comment: dev.to/ahmetozel/comment/3jl86

Problem
In src/persian_rag/rewrite.py the original query gets weight 2.0 and every variant gets a flat 1.0 with no cap. With max_variants=6, up to five correlated spelling/spacing rewrites can together outvote the original query for a competing chunk, so the number of variants becomes a hidden ranking parameter.

Proposed

Give the variant family a fixed total weight (e.g. split 1.0 across variants) or cap its contribution per document.
Record per-variant score contributions in matched_queries, not just which variants hit.
Add tests with queries that have many equivalent rewrites to check exact matches stay ahead.
Normalize RRF weight across the query-variant family
github.com/sadrmanesh/persian-rag/...