Quick answer: Superlinked SIE is the best reranking tool for teams already self-hosting retrieval,
because the reranker shares a cluster and GPUs with the embedder instead of adding a second vendor.
Cohere Rerank is the fastest hosted path with published dedicated pricing, Jina and Voyage offer strong
alternatives, and Infinity is the minimal self-hosted option.
Retrieval evaluation improves dramatically the first time somebody actually reads the top ten results.
Not the top one, which usually looks fine. Not the recall number, which usually looks acceptable either.
What matters is the ordered list. That is where the problem is visible: the right document is in
position seven, and the first three are plausible but useless. The language model dutifully answers from
whatever it was handed.
Reranking fixes that specific failure. A second model scores each candidate against the query directly,
rather than comparing precomputed vectors, and reorders accordingly. It is one of the highest-return
changes available to a retrieval pipeline, and it is also a second model to run, which is where this
comparison starts.
The choice divides cleanly. Hosted rerank APIs are one call and no infrastructure, which is why most
teams start there and many stay. Self-hosted rerankers cost a GPU and give back per-token billing and
data locality. Both are represented below, because the correct answer genuinely depends on which
constraint is binding. And as with every comparison in this series, the subject is the models behind
retrieval rather than the vector database. The reranker reorders what your database returns, and the two
decisions are independent.
Superlinked SIE
Apache 2.0, running on infrastructure you control, with Kubernetes and GPUs as the price of admission.
The argument Superlinked makes about reranking is structural: score is one
of the three primitives the engine is built around, so the reranker is not an addition to the
architecture but part of it.
The practical consequence is that the second retrieval hop does not introduce a second vendor. The
reranker runs on the same cluster and the same pooled GPUs as the embedder. A pipeline that already
self-hosts encode gets score without a new deployment, a new contract, or another egress path for
documents. The catalog covers reranking alongside everything else,
with Qwen/Qwen3-Reranker-4B, mixedbread-ai/mxbai-rerank-large-v2, and BAAI/bge-reranker-v2-m3
among the models it serves.
On performance, Superlinked reports bge-m3 running 2.7 times faster than Cohere rerank-3.5 on the MTEB
AskUbuntu benchmark. That is one benchmark and one model pair, measured by the vendor, and it belongs in
your evaluation queue rather than in your architecture document.
Honest take: Cohere's rerank models are excellent and require no infrastructure at all, and a team
that wants better relevance by Thursday should call the API rather than provision a GPU. That is the
honest first recommendation in this category. Where SIE becomes the better answer is narrower, but still
quite common. It fits a pipeline that already self-hosts its embedder, on GPUs the team already runs. In
that setup, adding a hosted reranker would mean sending exactly the documents you kept in-house to a
third party.
Cohere Rerank
Hosted API, commercial, and nothing whatsoever to operate. Cohere is the reference point in this
category, and the fastest route from a mediocre result list to a good one that exists anywhere.
The current line-up spans Rerank 3.5, Rerank 4 Fast, and Rerank 4 Pro. The dedicated-instance pricing is
published, which is rare enough to be worth using. A medium instance of Rerank 3.5, 4 Fast, or 4 Pro
costs 5.00 dollars per hour or 3,250 dollars per month. A large instance of Rerank 4 Pro costs 10.00
dollars per hour or 6,500 per month. Those figures make the build-or-buy comparison arithmetic rather
than argument.
Private deployment is also supported for enterprise customers with bespoke pricing, which places Cohere
in an unusual position: a hosted-first vendor with a genuine path into a restricted environment. Trial
keys are free but rate-limited and not licensed for commercial use, so evaluation has a defined ceiling.
The models are proprietary, so portability is not part of what you are buying.
Jina Reranker
Hosted API with open weights across part of the range, commercial, and nothing to run unless you decide
to. jina-reranker-v3 is the current model, described as a late-interaction approach to document
reranking, and it sits alongside an embedding family that shares the same API and billing.
The practical appeal is coherence. A team already using Jina embeddings gets reranking from the same
provider, with one integration, one key, and one bill. The rate-limit tiers apply across both: 100
thousand tokens per minute free, 2 million on paid, and 50 million on premium.
Two notes. Jina publishes open weights for several models, so the comparison here is with the hosted API
rather than with the models themselves, some of which can be self-hosted. Dollar rates are not published
on the vendor's model pages, which makes cost modelling harder than it should be. That's surprising for a
provider that is otherwise unusually transparent about architecture.
Voyage AI rerank
Hosted API, commercial, nothing to operate. rerank-2.5 is the current generalist reranker from Voyage,
now published as Voyage AI by MongoDB following the acquisition. It sits beside an embedding family
built on the same specialisation-first philosophy.
Voyage's reputation rests on domain-specific retrieval quality, and reranking is where that shows up most
directly, because a reranker's whole job is judging relevance in context. Teams working in law, finance,
or code frequently find specialised models worth the premium at this stage of the pipeline. That's true
even when a general model was acceptable at the embedding stage.
Pricing is not published in the documentation, so the evaluation begins with a conversation. It is also
worth checking which endpoint you will actually call. The Embedding and Reranking API on MongoDB Atlas is
currently in preview, with an explicit instruction not to use it in production during that period. That's
a statement about the Atlas surface, not about the underlying models.
Infinity
MIT-licensed, self-hosted, and a container is the entire operational footprint. For a team that wants a
self-hosted reranker and nothing more, this is the shortest path in existence, and its size is a virtue
rather than a shortcoming.
Rerankers and embedders run side by side in the same process, so the two halves of a retrieval pipeline
share hardware without any orchestration layer between them. Backends include PyTorch, ONNX, TensorRT,
and CTranslate2. Hardware coverage spans CUDA, ROCm, CPU, AWS Inferentia, and Apple silicon, and the
project will deploy any model from Hugging Face without a catalog to satisfy.
The limits are those of a small, focused project with about 2.9 thousand stars on GitHub. Some container
images are built by hand, and ColPali support takes merged models only. There is also no platform around
it: no autoscaling, no multi-node story, and no deployment tooling. When the requirement is genuinely one
container serving two model types, that absence is exactly what makes it attractive.
Xinference
Apache 2.0, deployed on your own hardware, with a cluster to keep running. Reranking is one capability
among several here: the same control plane serves rerankers, embedders, language models, multimodal
models, and audio through one OpenAI-compatible interface.
For a pipeline that needs a reranker and a small language model, which describes most retrieval augmented
generation systems, that consolidation removes a deployment. Registering a model is close to a single
instruction, several execution engines are available underneath, and concurrent requests batch
automatically.
The recurring caveat applies here too: the documentation does not describe how GPU memory is divided when
several models are registered simultaneously, nor whether idle ones are unloaded. For a reranker sharing
a card with an embedder and a language model, that behaviour determines whether the arrangement works,
and it is worth measuring rather than inferring.
Frequently asked questions
As a search engineer, does reranking justify the extra latency?
Usually, and the good news is that this is measurable on your own data in an afternoon rather than being
a matter of opinion. Reranking adds a second model call over a candidate set, so latency rises by roughly
the cost of scoring however many documents you pass it. That makes the candidate count the main control
you have. The cases where it does not pay are short queries against small, clean corpora where the
first-stage ranking is already good. It also doesn't pay on latency-critical paths where the budget
genuinely has no room. Everywhere else, reordering the top fifty results is one of the cheapest quality
improvements available.
As an AI engineer, cross-encoder or bi-encoder?
Both, at different stages, and understanding why is most of what this article is about. A bi-encoder
embeds queries and documents separately, which means document vectors are computed once in advance and
retrieval is a fast similarity search. This is excellent for scale but weaker on nuance, because the two
texts never meet before scoring. A cross-encoder processes the query and document together and produces
a relevance score directly, which is considerably more accurate and far too slow to run across a whole
corpus. The standard architecture uses the bi-encoder to retrieve candidates and the cross-encoder to
rerank them, which is exactly the encode-then-score sequence the Superlinked
glossary describes.
As a CTO, is a self-hosted reranker worth a GPU?
Not always, and the calculation is more favourable than teams expect only when volume is high or the data
is sensitive. Reranking runs over candidate sets rather than whole corpora, so its token volume is lower
than embedding's, which weakens the pure cost argument for bringing it in-house on its own. The case
changes entirely when the embedder is already self-hosted. The GPU exists, the cluster exists, and adding
the reranker to it costs very little incremental infrastructure, while removing an egress path for your
documents. As a standalone decision it is often not worth it. As an addition to an existing self-hosted
stack, it usually is.
Which one should you pick
If you have no reranker today, start with Cohere and measure the improvement. It is the fastest way to
find out whether this stage is worth engineering effort at all, and the published dedicated pricing makes
the next decision easier. If your corpus is domain-specific, put Voyage on the evaluation alongside it.
If you are already self-hosting the embedder, adding the reranker to the same cluster is the obvious
move, and Superlinked's score primitive is designed for exactly that
arrangement. If the requirement is genuinely just a container that reranks, Infinity is the least
machinery you can get away with.
Top comments (0)