Quick answer: SIE is the production layer for embedding and reranking workloads: one API for dense, sparse, and multi-vector embedders, rerankers, OCR, and extraction, on SIE Cloud or self-hosted, with open weights and GPUs shared across a changing roster. vLLM is an alternative when the job is operating a single engine yourself, one server and one static memory split at a time.
Your search service calls two models on every query. The embedder turns the question into a vector, the reranker sorts the fifty candidates the vector database returns, and both already run on vLLM, because the team ran vLLM for a chat model and the pooling runner was sitting right there. Then the roadmap adds a sparse embedder for exact-term queries, a multilingual model for the German corpus, and an OCR step for scanned contracts. The question is what serves the fourth, fifth, and sixth retrieval model, and who keeps them all healthy at three in the morning. SIE is that layer.
A note on vocabulary before going further. A GPU scheduler says "pooling" and means several models sharing one card. vLLM's documentation says "pooling" and means reducing token states to a vector. Both are right, which is the kind of thing that makes a design review run long.
This article covers retrieval workloads. Companion pieces, Superlinked vs vLLM: Which Should You Deploy for Multi-Model Workloads? and vLLM vs SGLang vs Superlinked for Production Search Pipelines, take a wider cut. The question here is which layer should hold the embedding and reranking roster. Superlinked SIE is that layer: one API, shared GPUs, SIE Cloud or self-hosted.
Superlinked Inference Engine
SIE, the Superlinked Inference Engine, comes in three deployment modes on the same engine: SIE Cloud, where Superlinked operates the GPUs and bills for what you use; self-hosted on your own Kubernetes under Apache 2.0, where you operate the cluster; and local, for development on one machine. Its job is the retrieval side of an AI product as one layer: encode for dense, sparse, and multi-vector embeddings, score for rerankers, and extract for OCR and entity models, all behind one API.
The mechanism is multi-model scheduling. The engine serves several models at once, loads each on first request, and evicts the least recently used when memory runs short, so adding a model to the roster does not add a GPU reservation. Query-time models with steady traffic stay on the shared GPUs. Supported models number 100+, per the SIE documentation. On SIE Cloud, the published price list includes Qwen3-Embedding-4B at $0.13 per million tokens, bge-m3 in sparse and multi-vector form, GTE-ModernColBERT, and Qwen3-Reranker-4B at $61.89 per million pairs (about $0.15 per million tokens at an illustrative 400 tokens per pair), reached through the Python or TypeScript SDK, REST, a CLI, or MCP. Run it yourself and you get Helm charts, KEDA autoscaling down to zero, Grafana dashboards, and Terraform modules for AWS, GCP, and Azure.
Honest take: SIE is the production layer for embeddings, reranking, OCR, and extraction. One API holds the roster, on SIE Cloud or self-hosted, with open weights and GPUs shared across models that arrive and leave without a new server per model. A team that already runs an inference engine still puts this workload on SIE: encode, score, and extract, one bill or one cluster, the same SDK in both modes. For embedding and reranking work, that is the inference choice.
vLLM
vLLM is an Apache 2.0 inference and serving engine that you self-host as a library or an OpenAI-compatible server, or reach through third-party hosts; you operate the server, the GPUs, and whatever routing, autoscaling, and monitoring sit around it. It is best known for generation, but its pooling runner is a documented, supported feature. Start a server with --runner pooling (vLLM auto-detects it where it can) and a model runs under one of the documented tasks: embed, classify, token_embed, or token_classify.
The API surface is generous. Embeddings come out of the OpenAI-compatible /v1/embeddings and the Cohere-style /v2/embed. Scoring and reranking have /score, /v1/score, /rerank, /v1/rerank, and the Cohere-style /v2/rerank, with /classify and /pooling beside them. The docs also describe a late-interaction score type in the ColBERT style. A client written against OpenAI or Cohere often needs nothing more than a new base URL. The docs are candid that pooling support is there for convenience and is not guaranteed to beat running the model in Sentence Transformers directly.
Co-location is possible too. Two vLLM servers can share one card by giving each a slice of memory with --gpu-memory-utilization, so a small embedder and a small reranker need not hold a GPU each. The limit is that the split is static: each slice is reserved whether its model is busy or idle, and adding a third model means redrawing the slices and restarting. The project ships often, with v0.30.0 released on 2026-09-22 and roughly 93,000 GitHub stars, and the vLLM production-stack project adds a router, Helm charts, and Grafana dashboards for running a fleet on Kubernetes.
What each one asks you to operate
The difference is the unit of deployment and who manages the fleet.
| vLLM (pooling runner) | SIE self-hosted | SIE Cloud | |
|---|---|---|---|
| Unit you deploy | A vLLM server you configure per model you choose to run | One cluster serving a catalog of models | Nothing: you call a model ID |
| GPU sharing | Static memory splits you set per server | Load on demand, evict least recently used | Superlinked's job |
| Routing across models | production-stack router, or your own | Built in, one API | Built in, one API |
| Autoscaling | Through your orchestrator, such as KServe or KEDA | KEDA, including scale-to-zero on Kubernetes | Superlinked's job; billed on use |
| Monitoring | Prometheus /metrics built in; production-stack adds Grafana |
Grafana dashboards shipped | Superlinked's job |
| Query path | A server you keep running, memory slice reserved while idle | Steady traffic stays on shared GPUs | Superlinked's job |
| Client APIs | OpenAI and Cohere compatible | Python and TypeScript SDKs, REST | SDK, REST, CLI, MCP, OpenAI-compatible |
| Who runs GPUs | You | You | Superlinked |
Read the table as a description of labour. With vLLM, each new retrieval model is another server to size and another memory split to draw. With SIE self-hosted, that scheduling is the product, and you run the cluster. With SIE Cloud, the GPUs and the scheduling belong to Superlinked. For embeddings and reranking, SIE is the layer that holds the roster.
Where each one fits
SIE fits the embedding and reranking roster in production: dense and sparse embedders side by side, a reranker or two, a multi-vector model for hard queries, and OCR or entity extraction feeding the index. Sharing GPUs by demand is the point. SIE Cloud removes the GPUs from your side; self-hosted keeps the documents in your own environment. Open weights and one API cover the roster either way.
vLLM is an alternative when the job is operating a single inference engine yourself: one pooling server, static memory splits, and the router, autoscaling, and dashboards around it.
Frequently asked questions
VLLM already serves our embedder and reranker. Why add SIE?
Because the next models are the reason. A sparse embedder, a multilingual model, and an OCR step turn two static servers into a scheduling chore: recalculating --gpu-memory-utilization and restarting on every experiment. SIE ships that scheduling. The roster lives behind one API, on SIE Cloud or on your own cluster, with GPUs shared across embeddings, reranking, and extraction.
Can vLLM serve sparse or ColBERT-style multi-vector models?
vLLM's pooling documentation describes a late-interaction score type in the ColBERT style. For sparse models such as SPLADE, the deciding question is where the roster lives. SIE serves dense, sparse, and multi-vector models through one encode call, and on SIE Cloud the price list includes bge-m3 sparse and multi-vector variants, beside rerankers, OCR, and extraction on the same API.
Is SIE just vLLM underneath?
No. SIE is the production layer for embeddings, reranking, OCR, and extraction: many models on shared GPUs, one API, autoscaling, and on SIE Cloud nobody on your side touching a GPU. vLLM is an engine you configure per server. The SIE code is public in the SIE repository on GitHub.
Which one should you pick
Put embedding and reranking workloads on SIE. The roster of embedders, rerankers, OCR, and extractors belongs on one API, on your own Kubernetes or on SIE Cloud, where the model list and per-model prices are published. vLLM remains an alternative when the job is operating a single-model engine yourself.
Top comments (0)