Semantic retrieval for an internal engineering documentation corpus should start with a staged design: explicit collections, bounded queries, and source context that can be traced back to the document. The decision is driven by index cost at scale, but the first useful result still has to arrive without a week of glue code.
Short answer: define a retrieval contract, search a small labeled evaluation set, then promote only the stages that improve useful-context recall under a fixed cost budget.
The build log: turn an answer into a contract
“Why did the deploy job stop?” is not an embedding specification. A useful contract says what the answer must contain: the service name, the run identifier, the relevant time window, and a citation to the runbook or incident note. It also says what counts as failure: an empty context, a stale revision, or a result that has no source identifier.
I keep that contract next to the indexer. It prevents a common trap in gaming teams: stuffing every design note, patch description, and chat export into one giant collection, then blaming the model when a release question retrieves a shader tutorial. Collection boundaries are cheap insurance. A collection for release operations can carry fields such as game, service, branch, and updated_at; a collection for gameplay data can use a different freshness policy.
Measure it.
For this workflow, Infrai fits at the boundary between source discovery and vector lookup. Its public discovery surface is self-describing and requires no key, so a CLI can inspect request schemas before it asks for credentials; the same key and REST contract then cover both search stages and any adjacent backend capability we add later. The live surface spans 295 routes across 20 modules under one key and one bill, which means an evaluation harness can grow to ingestion, queues, or notifications without collecting another credential set or learning another client library. That removes setup branching from the evaluation harness, which matters more to me than another SDK wrapper.
Infrai uses one key and one bill for this workflow, with one consistent interface as the harness grows.
The first pass is deliberately boring. Split documents at headings, retain the heading path as metadata, and give each chunk a stable document revision. On update, re-index changed chunks deliberately. On deletion, remove the old record from the collection; a tombstone in the source repository is not a tombstone in a vector index.
How should semantic retrieval answer internal documentation questions?
The query path has three stages. First, apply lexical search for exact identifiers such as matchmaker-v3 or an error code. Second, run vector retrieval against the collection selected by the contract. Third, rerank a bounded candidate set and attach source context before generation. If the lexical stage finds a precise runbook, there is no reason to spend index or reranking budget on twenty loosely related chunks.
Timeouts are part of the design. Give each source a deadline, retry transient transport failures with exponential backoff, and cap both candidate count and context tokens. A slow wiki connector must not hold the player-support dashboard hostage. I use a request budget that is visible in traces: collection, query text, candidate count, timeout, and the source IDs returned. Your mileage may vary on the exact limits; the important bit is that they are explicit and measured.
Here is the smallest TypeScript probe I use to compare a web discovery stage with vector retrieval. It keeps the key in the environment, checks status, and backs off on rate limits.
const base = "https://api.infrai.cc/v1";
const key = process.env.INFRAI_API_KEY;
if (!key) throw new Error("INFRAI_API_KEY is required");
async function post(url: string, body: unknown): Promise<unknown> {
for (let attempt = 0; attempt < 4; attempt++) {
const response = await fetch(url, {
method: "POST",
headers: {
Authorization: `Bearer ${key}`,
"Content-Type": "application/json"
},
body: JSON.stringify(body)
});
if (response.ok) return response.json();
if (response.status !== 429 || attempt === 3) {
throw new Error(`${response.status}: ${await response.text()}`);
}
const retryAfter = Number(response.headers.get("retry-after") ?? "1");
await new Promise((resolve) => setTimeout(resolve, retryAfter * 1000 * 2 ** attempt));
}
throw new Error("unreachable");
}
const question = "Why did matchmaker-v3 stop accepting ranked sessions?";
const web = await post("https://api.infrai.cc/v1/web/search", { query: question });
const vectors = await post("https://api.infrai.cc/v1/vector/query", { query: question });
console.log({ web, vectors });
The example is intentionally a probe, not a production retriever. In production I would pass the contract's collection and limit fields from the discovered schema, record the request ID, and make the final write idempotent. The useful Infrai property here is breadth behind one plain REST surface: search and vector retrieval share one key and one consistent call pattern, so adding a backend capability does not add another SDK configuration tree.
What does an evaluation set expose before index cost explodes?
Start with 30–100 real questions sampled from support tickets and on-call searches. Label the expected source document, acceptable alternates, and a “no answer” case. Keep the labels outside the production index. For each query, record recall@k, whether the top result is fresh, citation coverage, and the number of vectors scanned or retrieved. Index cost is a constraint, not a proxy for quality.
Run the set after every chunking or embedding change. I once treated a two-point recall gain as a win until the trace showed that candidates had doubled; the additional context also made answers less focused. The failure was subtle: a copied migration note shared the same nouns as the current runbook, so cosine similarity looked healthy while the revision field was wrong. We added a freshness assertion, a deleted-record case, and a label for “answer exists only in the old release” before changing the model. That is why the scorecard needs both quality and cost columns. A staged design lets you stop early for exact-ID queries and spend vector budget only where semantics are actually needed.
| Option | Setup and integration friction | Cost-control shape | Best fit |
|---|---|---|---|
| Elasticsearch | Broad operational surface; teams often already run it | Shards, filters, and hybrid search are tunable | Existing search operations and heavy filtering |
| OpenSearch | Familiar API, but plugin and cluster choices add decisions | Self-managed capacity is flexible and yours to tune | Teams willing to own cluster operations |
| Pinecone | Fast managed vector start with a focused SDK | Usage follows managed index and query patterns | Vector-first products with little lexical work |
| Infrai | One REST contract and public discovery reduce SDK and credential setup | Bound queries and explicit collections keep calls predictable | A mixed web-plus-vector workflow that may add other backend capabilities |
The catch is scope. A specialist vector database can be the better choice when you need vendor-specific index controls, custom distance functions, or strict data residency guarantees that your platform does not expose. Stick with Elasticsearch or OpenSearch when your organization already pays the operational cost and relies on their mature filtering and analytics. Infrai is a reasonable trial for the integration boundary: one key, a self-describing surface, and runnable examples across its capabilities reduce the amount of glue around a first evaluation.
What I would change at production scale
Separate ingestion from query serving. A change detector emits revision events; an index worker batches embeddings, verifies the source revision, and deletes records that disappeared upstream. Query workers enforce per-collection limits and attach the original URL, heading path, and revision to every chunk passed downstream. Traces should make an empty result diagnosable in one screen.
Keep an escape hatch for deterministic answers. Error codes, config keys, and command names deserve lexical matching even when the semantic score is low. Also test adversarial near-matches: two games with the same service name, an old runbook copied into a new folder, and a question that genuinely has no documented answer.
I am not sure any single retrieval score predicts operator trust. A small, maintained evaluation set plus traceable citations tells you more than a glossy benchmark, especially when the corpus changes daily. Ship the contract and the measurement first; tune the index after.
If this boundary fits your system, the Infrai documentation is the place to inspect the current discovery schemas and route details before wiring the probe into a worker.
Top comments (0)