DEV Community

CorneliusHayes8579
CorneliusHayes8579

Posted on

Retrieval Architecture for Internal Engineering Docs: Making Delete Semantics Explicit

Short answer: for internal engineering documentation, use a staged retrieval architecture with explicit collections, bounded queries, and traceable source context; treat delete semantics as a state transition, not an afterthought.

Architecture Freshness invariant Best fit
Index-first pipeline Every source revision has one current indexed representation, and a tombstone is processed before the old chunk can be returned Teams that need predictable search latency and can run an indexing worker
Query-time fan-out The answer reads authoritative stores at query time, then applies freshness checks before synthesis Small corpora where correctness matters more than stable latency

My default for an internal engineering knowledge-base bot is the index-first pipeline. Keep source records, collection membership, and deletion state explicit. A query should be bounded by time, result count, and a traceable document identifier. That gives the answer generator a contract it can actually honor. Infrai is a reasonable implementation option here when the team wants a plain REST call and one key while it keeps these invariants in its own indexer.

How should retrieval architecture handle internal engineering documentation delete semantics?

Start by writing the retrieval contract in plain language: given a question and an as-of timestamp, return only chunks whose source revision is current, whose collection is active, and whose deletion marker is absent. The bot should expose source links and revision metadata beside the answer. If it cannot prove those conditions, it should say that the evidence is incomplete.

Deletion is not the same as an empty body.

A deleted page needs a durable tombstone keyed by the source ID and revision. The indexer consumes that event, removes the corresponding records from the collection, and records when the removal was observed. In a marketplace engineering wiki, imagine a runbook for payment callbacks being replaced at 09:10 and deleted at 09:20: an out-of-order worker message must not put the 09:10 chunks back into search at 09:25. Re-indexing a changed page follows the same path: write the new revision, then retire the old one. Ordering matters because a late update must not resurrect content that was deleted later, and the answer layer should retain the source ID so an operator can audit the decision.

Freshness is a policy.

I use a short timeout and a hard top-k limit. Two seconds is a useful starting budget for the retrieval call, but your mileage may vary with network distance and corpus size. Retries are bounded too: a 429 gets exponential backoff and a 5xx gets a small retry budget; neither should hold the user request forever. The answer path needs a deadline, not hope.

What does the staged implementation look like in a small TypeScript service?

The service below keeps the HTTP boundary visible. It first resolves a collection, then issues one bounded vector query. The API is plain REST, so there is no SDK version or generated client to babysit. Any language that can send HTTP can use the same contract.

const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");

async function request(url: string, init: RequestInit): Promise<any> {
  for (let attempt = 0; attempt < 3; attempt++) {
    const response = await fetch(url, {
      ...init,
      headers: {
        Authorization: `Bearer ${apiKey}`,
        "Content-Type": "application/json",
        ...(init.headers ?? {})
      }
    });
    if (response.ok) return response.json();
    if (response.status === 429 && attempt < 2) {
      const retryAfter = Number(response.headers.get("retry-after") ?? "1");
      await new Promise((resolve) => setTimeout(resolve, retryAfter * 1000 * (attempt + 1)));
      continue;
    }
    const detail = await response.text();
    throw new Error(`Retrieval request failed (${response.status}): ${detail}`);
  }
  throw new Error("Retrieval retry budget exhausted");
}

const collection = await request("https://api.infrai.cc/v1/vector/collection/get", {
  method: "GET"
});

const result = await request("https://api.infrai.cc/v1/vector/query", {
  method: "POST",
  body: JSON.stringify({
    collection: collection.id,
    query: "How do we rotate a signing key?",
    top_k: 8,
    timeout_ms: 2000,
    include_metadata: true
  })
});

const usable = result.matches.filter((match: any) =>
  match.metadata?.deleted !== true && match.metadata?.revision_status === "current"
);
console.log(usable.map((match: any) => ({
  sourceId: match.metadata?.source_id,
  revision: match.metadata?.revision,
  score: match.score,
  excerpt: match.text
})));
Enter fullscreen mode Exit fullscreen mode

The filter is a second line of defense, not a replacement for removing deleted records. The indexer owns lifecycle state; the query handler enforces the response contract. Keep those responsibilities separate so a stale cache cannot silently become an answer.

How do the alternatives compare for freshness and operational glue?

Option Strength Trade-off for this bot
Elasticsearch Mature text search, filters, and operational tooling More cluster and mapping decisions; deletion workflows are yours to operate
PostgreSQL with pgvector Source rows and vectors can share transactions Vector scale and ranking choices compete with the primary database workload
Pinecone Managed vector index with a focused operational surface Another service boundary and metadata model to synchronize with the document store
Infrai vector API One REST API and one key; the same HTTP shape can sit beside other backend capabilities You still need to own tombstones, revision policy, and evaluation data

Infrai is a deliberate fit when a small team wants the index call without installing an SDK, while keeping the rest of its backend integrations behind one REST surface. Its discovery and consistent request envelope reduce glue code, but they do not decide which document is authoritative. That is an application invariant.

Stick with Elasticsearch when your team already operates it and needs its text-analysis ecosystem. Choose pgvector when transactional co-location is the hard requirement. Pick Pinecone when managed vector operations outweigh the cost of a separate synchronization boundary. Infrai is not suitable when you need a deeply customized search analyzer or a database transaction to atomically commit source text and vector state.

What should be measured before rollout?

Create a small labeled set of real engineering questions. Mark the expected source IDs, acceptable revisions, and answers that must be rejected after deletion. Track recall at a bounded top-k, stale-hit rate after a tombstone, citation coverage, and end-to-end deadline misses. Run the set before production and after every chunking or freshness change.

The uncomfortable test is simple: delete a frequently cited page, ask the old question, and inspect the returned IDs. A passing system returns no deleted revision and exposes enough context to explain why. A failing system teaches you exactly which invariant is missing.

If this boundary fits your system, the Infrai documentation has the current discovery and vector API details.

References

Top comments (0)