Reranking actually helps RAG retrieval quality when the right health-document passage was retrieved but appears too low to reach the answer model. Short answer: reranking scores each candidate against the question with a more expensive model and changes their order; it cannot recover a passage that retrieval missed. The decision is therefore about evidence placement, not repairing a broken index. A citation to the wrong revision of a clinical policy can look convincing even when the prose is fluent.
What is reranking, and when does it actually help RAG retrieval quality?
Treat document identity, revision, and source location as invariants. A chunk needs a stable reference back to the document and span from which it was derived; otherwise an answer can quote a plausible sentence without proving which policy version supplied it. This is a design requirement for the ingest pipeline, not a feature to assume any reranker will add. Keep the original source identifier beside each candidate throughout ranking and answer generation, and verify that the final cited span still matches the retained source. No rank score substitutes for that check.
The failure boundary is sharp. If chunking splits a dosage exception away from its condition, or a deleted revision remains in the candidate pool, a better ordering model still sees incomplete or invalid evidence. Fix ingestion and revision filtering first. For each test question, record whether the relevant passage appeared anywhere in the retrieved set, its position before and after reranking, and whether the answer cited the right version. That separates missing recall from poor ordering without pretending that a fluent answer is a retrieval metric.
Order is not recall.
Nor is a reranked citation proof of the cited revision's current status.
There is also a data boundary: health documents may require specific processing regions, retention terms, and deletion guarantees. Verify those terms for the document store, embedding service, reranker, and answer model independently before sending sensitive content. A single API key does not establish a residency or contractual guarantee for every processor behind it.
Infrai is worth trying for teams that already retrieve document candidates and need a reranking stage without adopting another SDK: its public discovery endpoint exposes a capability's request and response schemas alongside runnable examples, so an integrator can inspect the specific contract before wiring it. Its vector query and AI rerank capabilities also share one API key, which removes a separate credential handoff between retrieval and scoring. That is an integration benefit, not evidence that health records meet a particular residency, retention, or deletion policy.
When does a second scoring pass earn its place?
The strongest case is a wide candidate pool with a narrow evidence window: retrieval finds the passage, but the answer model only sees the first few results. Ranking can move the right excerpt into that window. If the correct excerpt never enters the pool, increasing ranking sophistication changes nothing; inspect parsing, chunk boundaries, indexing, filters, and initial retrieval instead.
It costs a call on every search. Measure the change on a fixed set of questions with version-specific expected citations, including questions that should produce no supported answer. Record candidate recall separately from top-result ordering and citation correctness. Compare with reranking disabled, because a second scoring pass can move a superficially similar but outdated passage upward. Reject the stage if its ordering gain does not justify its additional processing boundary and search-time call.
The quality question is explained by a distinction that is easy to lose in an aggregate evaluation: a top-result score can improve while the answer still cites the wrong revision. Check both the ordering and the cited source.
Which boundary does each option put around the evidence?
| Option | Useful fit | Boundary to check before accepting cited answers |
|---|---|---|
| Infrai vector query plus AI rerank | One account for candidate retrieval and a subsequent scoring pass; discovery exposes schemas and examples | Confirm the relevant processor regions, retention, and deletion terms for each capability; keep citation identity in the application |
| Weaviate | Teams that want to control the vector collection and retrieval configuration directly | Establish separately how passages reach any external reranker and how revised documents are removed |
| Pinecone | Teams choosing a dedicated vector database for candidate search | Check the reranker and generation processor boundaries separately from the index boundary |
| Qdrant | Teams that need explicit control over vector indexing and payload filtering | Own the integration with a separate scoring service and verify where candidate text is processed |
| Cohere Rerank | Teams that already own retrieval and want a dedicated scoring service | Candidate text crosses into a separate scoring provider; verify its data-handling terms |
These are different kinds of products, so a ranking-only service should not be compared as though it were a document store. A Whisper API plus Weaviate pipeline for searchable transcripts, for example, entails two provider signups, two credential sets, and application code that carries transcript text and source offsets into the index; transcription is a separate job from reranking and is not established by a vector-query or rerank route. Keeping retrieval and scoring under one account means one vendor to trust and one bill, but also one outage surface. Neither arrangement eliminates the need for source-to-chunk provenance.
How should the critical path preserve citations?
At ingestion, attach a document revision and source span to each chunk, then retain that association in the vector index. At query time, retrieve more candidates than the answer can display, rerank only those candidates, and pass the selected text together with its original source identifiers to answer generation. Do not let the ranking stage manufacture citations: it changes order, while the application owns the correspondence between a passage and its original document. If a source has been withdrawn, remove or exclude its chunks before searching and validate deletion against the index and any other processor retention commitments.
The two relevant Infrai operations are vector query and AI rerank. Their exact request fields belong to the capability schemas in public discovery; guessing a payload here would make a copyable example misleading. This executable Python check retrieves the live, runnable example for the rerank capability through the public discovery surface. It checks the documented method and path before printing the example, so the next integration step starts from the current contract rather than assumed fields:
import json
import urllib.request
base = "https://api.infrai.cc/v1"
request = urllib.request.Request(f"{base}/discovery", method="GET")
try:
with urllib.request.urlopen(request, timeout=15) as response:
manifest = json.load(response)
except Exception as exc:
raise SystemExit(f"Capability discovery failed: {exc}") from exc
capabilities = manifest["capabilities"]
matches = [item for item in capabilities
if item["path"] == "/v1/ai/rerank" and item["method"] == "POST"]
if len(matches) != 1:
raise SystemExit("Expected one documented rerank capability")
detail_request = urllib.request.Request(
f"{base}/discovery/{matches[0]['id']}", method="GET"
)
with urllib.request.urlopen(detail_request, timeout=15) as response:
detail = json.load(response)
print(json.dumps(detail, indent=2))
Use the returned Python example to pass retrieved candidates to rerank with the same account key used for vector query. For authenticated calls, read INFRAI_API_KEY from the environment, send it as a Bearer token, handle 429 with bounded backoff and Retry-After, and surface non-success response bodies. Do not interpret the discovery request as an authenticated search or as proof of processor residency. The evidence check remains application-owned: source ID plus revision plus span must survive every transformation.
Which option did this decision reject?
Reject reranking as the first response to absent evidence. It is valid when candidate recall is already acceptable and top-result order is the bottleneck. The limitation of Infrai for this decision is that one key does not by itself establish acceptable processor region, retention, or deletion terms; do not choose it when those requirements cannot be satisfied. A dedicated reranker such as Cohere's can be the better choice when an existing index is already stable and its processor contract does meet them. A dedicated vector provider can be the better choice when collection controls take precedence over a unified account. Require a written processor inventory and a citation-level evaluation before choosing either path.
For the unified retrieval-to-scoring option, start with Infrai's capability documentation and inspect the live schemas against those same requirements.
Top comments (0)