Package registries such as PyPI are effective when a user already knows a package name. They are less effective when the user can describe only a task, for example an asynchronous HTTP client with retries, or a dataframe library that is not pandas. Keyword indexes reward lexical overlap and provide little information about whether a project is widely adopted or still maintained.
SARR (Semantic Artifacts Retrieval and Ranking) is an MVP retrieval system for that second class of query. Each indexed package is represented as a short semantic document, neighbors are retrieved in embedding space, and the candidate list is reordered with adoption and recency signals. The live collection holds 167,619 PyPI packages. It is not a complete registry: Qdrant Cloud’s free cluster is limited to 1 GB of RAM and 4 GB of disk, which is the storage ceiling that stopped a full-PyPI index. No precision, recall, or nDCG figures exist yet. The usefulness claims in this post are anecdotal until a labeled evaluation set is built, as noted under Planned work.
The practical aim of the design is that a package can be returned even when its name does not contain the query terms, while still preferring projects that are more widely used when semantic scores are close.
The live demonstration is available at kanchana123.github.io/sarr-recommendation-api. The search endpoint is POST https://isz2aki1n2.execute-api.us-east-1.amazonaws.com/v1/search. Source code is published at github.com/kanchana123/sarr-recommendation-api.
Queries such as "text to speech" and "async HTTP client with retries" illustrate both the benefit and a remaining error mode. PyPI.org often ranks packages whose names contain the query tokens. SARR can return google-cloud-texttospeech even though that name is not a literal copy of the query. It can also retrieve near-misses such as speech-to-text libraries for a text-to-speech query. That confusion is a genuine limitation of purely semantic retrieval and is one reason later work should collect click feedback for fine-tuning.
kanchana123
/
sarr-recommendation-api
Semantic Artifacts Ranking and Retrieval API — A high-performance semantic search, reranking, and recommendation service built with Python, PyTorch, and FastAPI.
SARR — Semantic Artifacts Retrieval and Ranking
SARR is a search system for software packages that ranks results by meaning not just keyword overlap. The MVP indexes ~168k PyPI packages and answers natural-language queries such as “async HTTP client with retries” or “machine learning” — including packages whose names do not contain those words.
Offline indexing runs on GPU (Google Colab) with PyTorch. Online search embeds the query, retrieves neighbors, optionally reranks, and blends popularity and recency. Vectors live in Qdrant Cloud. The API is FastAPI on AWS Lambda, where both the bi-encoder and the cross-encoder run as ONNX (no PyTorch import on the request path). The demo UI is a static Vite app on GitHub Pages.
Live demo: kanchana123.github.io/sarr-recommendation-api
API: https://isz2aki1n2.execute-api.us-east-1.amazonaws.com (/v1/search, /healthz)
Why it exists
PyPI.org search is effective when you already know a package name. It is less helpful…
Design constraint
Scoring the entire corpus with a cross-encoder on every request is not a suitable architecture for a latency-bounded search API. Embedding 167,619 packages is a GPU batch job that should run infrequently. Embedding a single query and retrieving a few dozen neighbors is the work that belongs on the request path.
The system is therefore divided into an offline indexing path and an online serving path. Both paths use the same bi-encoder checkpoint, BAAI/bge-small-en-v1.5, and the same search-document format. Alignment between those two representations is essential: if the index-time encoder and the query encoder diverge, retrieval quality degrades without a corresponding runtime error.
| Path | Role |
|---|---|
| Offline ETL | Watermarked BigQuery extract, document construction, PyTorch bi-encoder batch embedding, and idempotent upsert into Qdrant |
| Online search | ONNX query embedding, approximate nearest-neighbor search, optional ONNX cross-encoder reranking, and score blending |
| Shared representation | The same embedding model and document format for indexing and querying |
The table states the split. Figure 1 shows the same split as a system diagram: clients and serving on the left, indexing and storage on the right.
Figure 1. Online serving appears on the left. Offline data preparation appears on the right. Corpus vectors are stored in Qdrant Cloud, which currently holds 167,619 points.
Corpus and document construction
The public Libraries.io tables in BigQuery already join project metadata with repository signals such as stars, SourceRank, and dependent counts. After filtering to PyPI packages with a non-empty description, the extract contained 167,619 packages. These documents were embedded on a Colab T4 GPU with BAAI/bge-small-en-v1.5 (384 dimensions) and written to Qdrant in well under an hour. A later full-registry load was not attempted on the hosted cluster because the free Qdrant Cloud node (1 GB RAM, 4 GB disk) is already occupied by this collection, including payload and the HNSW index.
The snapshot is not a current view of PyPI. The newest latest_release_publish_timestamp in this slice is from December 2018. The dump is still useful as an MVP corpus because it is large enough to exercise the pipeline and because it retains ranking features that official PyPI metadata does not provide. A later refresh can join official PyPI upload_time values without discarding stars and SourceRank. That same age, however, makes the recency term in the ranker inert on today’s wall clock, which is discussed with the score formula below.
Each search document is short and ordered. The package name and summary appear first, followed by keywords. Numeric popularity fields are stored in the Qdrant payload rather than concatenated into the embedding text. If stars or fork counts were placed in the text, widely known packages would appear semantically similar to unrelated queries, which would reduce the value of the embedding as a representation of function.
The extract-transform-load job is checkpointed by watermark. After each successful batch, the pipeline records max(latest_release_publish_timestamp). A Colab disconnect can therefore resume without repeating completed work. Point identifiers are a stable UUID5 of the normalized package name, so a repeated package is an upsert rather than a duplicate.
Request path
A search request proceeds in four stages.
- The query is embedded with the same bi-encoder checkpoint used at index time. On Lambda this is a baked ONNX graph rather than a PyTorch import.
- Qdrant performs an approximate nearest-neighbor search using cosine similarity over the full collection.
- If the request sets
rerankto true,cross-encoder/ms-marco-MiniLM-L-6-v2rescores a short top-k list. The hosted API runs this model as ONNX as well. - Relevance is combined with popularity and recency to produce the final ordering.
Those stages are the request path. Figure 2 shows them as a sequence for a single POST /v1/search call.
Figure 2. Lifecycle of a single POST /v1/search call. The response includes took_ms, which is server-side time for embedding, retrieval, optional reranking, and blending.
The final score is a weighted sum:
score = 0.75 · relevance + 0.15 · popularity + 0.10 · recency
These coefficients were chosen heuristically for the MVP; they have not been fit on a labeled set. Planned work describes the nDCG evaluation that should precede any further change to the weights.
Popularity is a log-scaled combination of stars and, when present, dependents or SourceRank. Recency is exp(-days / 180), where days is the age of last_commit or, if that field is missing, latest_release, measured against datetime.now(UTC). It is not measured relative to the newest timestamp in the collection. On this corpus the newest release is December 2018, so every package is about seven years old, exp(-days / 180) is numerically zero, and the 0.10 recency weight contributes nothing to ranking. That is a present limitation of the dump and of the scorer, not a hidden normalization. Cross-encoder reranking is optional and disabled by default, because it is the most expensive CPU stage on an otherwise inexpensive retrieval path.
The API is implemented in FastAPI. The same application runs locally with sentence-transformers when no ONNX graph is present, in Docker, and on Lambda through Mangum. A command-line client is provided so that a search does not require constructing JSON by hand:
sarr search "async HTTP client"
sarr search "machine learning" --rerank
curl -s https://isz2aki1n2.execute-api.us-east-1.amazonaws.com/v1/search \
-H 'Content-Type: application/json' \
-d '{"query":"async HTTP client","limit":10,"rerank":false}'
Serving with ONNX rather than PyTorch
The first Lambda deployment timed out on /v1/search while /healthz remained healthy. Downloading weights from Hugging Face and importing PyTorch on a cold start exceeded the client wait imposed by API Gateway HTTP APIs. That limit is 30 seconds, even though the Lambda function timeout is 90 seconds.
The runtime image therefore does not import PyTorch. A multi-stage Docker build exports bge-small-en-v1.5 and MiniLM-L-6-v2 to ONNX, and Lambda loads those graphs with ONNX Runtime. Corpus indexing still uses PyTorch on Colab. The separation keeps GPU training and batch embedding off the request path while remaining within the API Gateway time bound.
The function is deployed as a container image with 3,008 MB of memory, a 90-second timeout, and the x86_64 architecture. Approximate nearest-neighbor search over the 167,619 points is inexpensive relative to loading a MiniLM cross-encoder on CPU.
The title’s “serverless budget” is the cash cost of that stack. The vector store is a Qdrant Cloud free cluster (1 GB RAM, 4 GB disk, $0 per month). GitHub Pages is free. Lambda in us-east-1 is billed at $0.20 per million requests plus $0.0000166667 per GB-second on x86. At 3,008 MB, 1,000 warm searches cost about $0.001 without rerank (about 18 ms) and about $0.009 with rerank (about 176 ms). Demonstration traffic remains inside Lambda’s always-free monthly allowance of one million requests and 400,000 GB-seconds, so the hosted MVP runs at $0 per month on free tiers. A paid Qdrant cluster would be required before the index could grow to the rest of PyPI.
Latency
Two quantities are reported separately. The first is time to a useful result after a cold process, when ONNX sessions are loaded into memory. The second is time after those sessions are already resident. Server took_ms covers embedding, Qdrant search, optional reranking, and blending, and is the figure that describes serving performance. Client round-trip time includes the network path to us-east-1 and should be quoted together with the client location. Cold-start measurements are not mixed into p50, p95, or p99.
The following table was collected on 15 August 2026 against the live API Gateway URL. The procedure was one warmup request followed by 50 sequential mixed queries, using scripts/measure_latency.py. Every request succeeded.
| Condition | Server took_ms
|
Notes |
|---|---|---|
Cold, rerank=false
|
approximately 5 s | First request after idle; loads the ONNX bi-encoder |
Cold, rerank=true
|
approximately 16 s | Also loads the ONNX MiniLM cross-encoder |
Warm, rerank=false
|
p50 18 ms, p95 26 ms, p99 30 ms | Embed approximately 7 ms; Qdrant approximately 9–11 ms |
Warm, rerank=true
|
p50 176 ms, p95 266 ms, p99 270 ms | Approximately 160 ms additional time for the cross-encoder |
A typical warm request from the demonstration user interface with rerank enabled is about 200–230 ms end to end. Client round-trip time from a laptop outside us-east-1 was p50 approximately 122 ms without rerank and approximately 276 ms with rerank.
The measurement can be reproduced as follows:
python3 scripts/measure_latency.py \
--url https://isz2aki1n2.execute-api.us-east-1.amazonaws.com \
--n 50
The first request after idle should be treated as warmup. Including that load in percentile statistics would overstate typical serving latency.
Planned work
The corpus should be refreshed from bigquery-public-data.pypi.distribution_metadata, with payload fields merged so that stars and SourceRank are not overwritten when newer release timestamps are imported. Until that refresh, recency should be computed relative to the maximum timestamp in the collection rather than against wall-clock time, so that the 0.10 term can distinguish newer packages inside the 2018 snapshot instead of collapsing to zero for every point.
Retrieval should become hybrid. A sparse component would recover exact package-name matches that a dense encoder can lose to long descriptions.
Ranking weights should be evaluated on a labeled set of approximately 50 queries using nDCG before further manual tuning of the blend coefficients. Until that set exists, retrieval quality remains anecdotal, as stated in the introduction.
The system should also collect online training data from user interaction. When a user clicks a result, the interface would record the query, the displayed ranking, the selected package, and the surrounding candidates. These click events would form a preference dataset for fine-tuning the bi-encoder or the cross-encoder, so that systematic errors such as speech-to-text versus text-to-speech can be reduced from observed behavior rather than from static heuristics alone. Logging would remain separate from the request-critical path in order to preserve the latency characteristics reported above.
The index should be extended beyond PyPI to other package ecosystems, beginning with npm and then additional managers such as crates.io, Maven Central, and RubyGems. Libraries.io already covers several of these sources, so the same document schema, embedding model, and ranking blend can be reused. Each ecosystem would occupy its own Qdrant collection, or a shared collection with a payload field that filters by registry, so that a query can be restricted to one manager or issued across managers when an agent does not yet know the language of the project.
The existing sarr CLI should be developed into a tool that language-model agents can call directly. The interface would accept a natural-language task description, an optional registry (PyPI, npm, and later others), and a limit, then return structured JSON that names the recommended packages, versions, and short justifications. That contract is suitable for agent tool-calling and for Model Context Protocol servers, so an assistant can retrieve an appropriate dependency without scraping a web page or guessing a package name. Human users would continue to use the same binary interactively.
The contribution of this project is not the introduction of a vector database. It is a retrieval architecture in which the embedding encodes package function, quality signals are applied only at rank time, and the serving stack stays within a serverless latency budget by exporting both encoders to ONNX. The demonstration is available at kanchana123.github.io/sarr-recommendation-api. Reports of queries that the keyword index handles poorly, or that SARR ranks incorrectly, would be useful for the click-collection and evaluation work described above.
Top comments (0)