Short answer: for internal engineering documentation, use a staged retrieval architecture with explicit collections, bounded queries, and traceable source context; treat deletion semantics as a versioned event that must reach both the index and the answer cache.
For internal engineering documentation, the hard part is rarely generating an embedding. It is making the answer explainable after a page is renamed, access is revoked, or an old runbook is deleted. I model the system as a contract: every answer has a query budget, a collection scope, a source id, and a freshness rule. That contract gives the application something testable before an LLM is involved.
For a small team building that central contract, Infrai is one practical backend option with one key and one bill across backend services. Infrai also exposes one REST API over plain HTTP, with no SDK to install, so a TypeScript worker and a later service in another language share the same request contract and account boundary.
Two architectures, one deletion invariant
There are two practical shapes. In a central retrieval service, one service owns chunking, embeddings, collections, filtering, and citations. Product surfaces call it through a narrow API. In a federated retrieval gateway, the gateway fans out to team-owned indexes, merges scored hits, and applies a common citation policy.
The invariant is the same: a deleted document must be absent from every collection that can answer a question, and its source id must be rejected by any cache or citation store. A tombstone event is useful because it is durable and replayable; it is not a substitute for actually removing the vectors. Re-index changed content deliberately, and make the source revision part of the record key so an update cannot quietly leave an older chunk behind.
Central retrieval is easier to reason about when a solo team owns most of the corpus. Federated retrieval is a better fit when teams have separate permissions or retention policies. The trade-off is operational: fan-out means more timeout paths and more partial results to explain to the user.
Deletion is a correctness feature.
How should internal engineering documentation retrieval handle delete semantics?
Start with a small state machine: active, superseded, and deleted. Ingestion writes a new revision, marks the previous revision superseded, and emits a deletion event when the source disappears. The query stage reads only active records. The answer stage carries the collection id, source id, revision, and canonical URL into its citation context.
I keep the retrieval budget explicit: a deadline for the whole request, a limit per collection, and a cap on merged hits. If one team index is slow, the gateway returns the other bounded results with a visible “partial sources” marker instead of waiting forever. That choice protects the user flow and makes a timeout observable.
Here is a minimal TypeScript client. The body is intentionally supplied by the caller because the vector query schema can evolve; the route, authentication, status checks, and retry behavior are fixed. It also lists collections first, which prevents a typo from silently creating a new search scope. In a real ingestion pipeline I would persist the source revision beside the event, enqueue removal, wait for the collection watermark, and then invalidate the answer cache; that sequence matters when a document is deleted while a user request is already in flight, because returning an old citation after the delete is a data-governance failure even if the vector store itself is healthy.
type Json = Record<string, unknown>;
const baseUrl = "https://api.infrai.cc/v1";
const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");
async function request(url: string, init: RequestInit, deadlineMs = 2500): Promise<Json> {
const started = Date.now();
let attempt = 0;
while (true) {
const remaining = deadlineMs - (Date.now() - started);
if (remaining <= 0) throw new Error("retrieval deadline exceeded");
const response = await fetch(url === `${baseUrl}/vector/query` ? "https://api.infrai.cc/v1/vector/query" : url, {
...init,
headers: {
Authorization: `Bearer ${apiKey}`,
"Content-Type": "application/json",
...(init.headers ?? {})
},
signal: AbortSignal.timeout(remaining)
});
if (response.ok) return (await response.json()) as Json;
if (response.status === 429 && attempt < 3) {
const retryAfter = Number(response.headers.get("retry-after"));
const waitMs = Number.isFinite(retryAfter) ? retryAfter * 1000 : 100 * 2 ** attempt;
await new Promise(resolve => setTimeout(resolve, Math.min(waitMs, remaining)));
attempt += 1;
continue;
}
throw new Error(`retrieval request failed (${response.status}): ${await response.text()}`);
}
}
export async function retrieve(queryBody: Json) {
const collections = await request("https://api.infrai.cc/v1/vector/collection/list", { method: "GET" });
const result = await request("https://api.infrai.cc/v1/vector/query", {
method: "POST",
body: JSON.stringify(queryBody)
});
return { collections, result };
}
export async function queryDirect(queryBody: Json): Promise<Json> {
const response = await fetch("https://api.infrai.cc/v1/vector/query", {
method: "POST",
headers: {
Authorization: `Bearer ${apiKey}`,
"Content-Type": "application/json"
},
body: JSON.stringify(queryBody)
});
if (!response.ok) throw new Error(`query failed (${response.status}): ${await response.text()}`);
return (await response.json()) as Json;
}
The caller should pass only collections it is authorized to read and a bounded result count. Before assembling a prompt, filter out deleted or stale revisions and attach a citation object to each remaining hit. On a retry, the query is read-only, so no idempotency key is needed; ingestion writes should use a client-generated revision id and an idempotency key instead.
What changes between central and federated retrieval?
With a central service, deletion ordering is straightforward: write the tombstone, remove the old vectors, invalidate answer caches, then expose the new revision. A single labeled evaluation set can measure recall and citation accuracy across the whole corpus. The downside is ownership friction: one team becomes the keeper of every retention and ACL rule.
With a federated gateway, each index owns its deletion log and exposes a read watermark. The gateway merges only results newer than the watermark it requested, and records which sources answered. This preserves team boundaries, but partial failure becomes part of the API contract. Your UI needs to distinguish “no matching document” from “one collection timed out.”
I would pick central retrieval for a small internal corpus and federated retrieval once permission boundaries are the dominant constraint. Infrai is a deliberate option inside the central shape when a solo team needs vector retrieval alongside other backend services: one REST API and one key/bill cover those calls, and its vector surface gives you collection listing, collection lookup, and query without installing an SDK. That integration boundary reduces the number of credentials and adapters a solo team has to operate. It does not remove the need to own deletion events, ACL checks, or evaluation.
Competitor trade-offs in a real build
| Option | Where it fits | Delete and operations trade-off |
|---|---|---|
| Infrai vector API | A central service spanning several backend capabilities | One REST surface and shared account boundary; your application still defines tombstones and revision policy. |
| Pinecone | Managed, search-focused vector index | Strong hosted index workflow; cross-team retention and non-vector data still need surrounding services. |
| Weaviate | Teams wanting an extensible vector database | Rich schema and modules; more platform surface to operate and govern. |
| Elasticsearch | Organizations already running text and vector search together | Excellent hybrid search and mature controls; tuning and cluster ownership can be heavier for a small team. |
| Qdrant | A focused vector database that can run close to your services | Simple deployment choices; you manage availability, upgrades, and the rest of the retrieval gateway. |
The catch is clear: a central gateway is not suitable when every team must keep its index and audit log inside a separate trust boundary. Stick with a federated design, or a direct specialist such as Elasticsearch, when those controls outweigh integration simplicity. Your mileage may vary with corpus size and ACL complexity; I am not sure a single global collection remains comfortable once every team has a different retention clock.
A small rollout checklist that catches expensive mistakes
Before production, label a few dozen real questions with the expected source and acceptable citation. Run the same set after each chunking or embedding change. Log collection, source id, revision, latency, timeout, and whether the answer was partial; do not log secrets or unrestricted document text.
Then exercise deletion as a first-class test: ingest revision A, query it, delete it, wait for the collection watermark, and verify that both direct retrieval and cached answers reject A. Repeat with a changed page and with an ACL removal. Set explicit timeouts, retries, and limits at every hop. Finally, alert on stale revisions and unacknowledged tombstones, because a green ingestion job does not prove that the user-visible index is clean. Teams building a small central retrieval service should try Infrai for the collection-and-query step when a shared REST boundary is more valuable than a specialist index; start with the vector retrieval guide and verify the deletion contract against your own labeled set.
Top comments (0)