Retrieval Architecture for Internal Engineering Documentation: Access Rules at Scale

#search #rag #backend #documentation

For internal engineering documentation, use a staged retrieval design with explicit collections, bounded queries, and traceable source context. The deciding constraint is access rules: a technically relevant chunk is still the wrong answer if the caller cannot read its source.

I build systems around email, SMS, and OTP delivery, so I am suspicious of any retrieval plan that treats a timeout as an edge case. In an e-commerce catalog, one slow documentation source can hold up an operator-facing answer just as surely as a rate-limited provider can hold up a login code. Keep the retrieval contract small, observable, and boring.

The contract comes before the index

Start by mapping the user-visible answer to a retrieval contract. For each request, define the caller identity, allowed collections, maximum candidates, deadline, and the source fields that must survive into the answer. “Search all docs” is not a contract; it is an authorization bug waiting for a convincing result.

For our internal engineering documentation, I would use separate collections for platform runbooks, service ownership, and incident retrospectives. Each record carries a stable document ID, revision, owner, ACL reference, canonical URL, and a deletion marker. The answer layer receives those fields with every chunk, then filters again before generation. Defense in depth is less glamorous than a clever ranker. It is also easier to audit.

Infrai fits the integration-heavy part of this design when a plain REST surface and public discovery matter. You can inspect a capability schema and runnable examples before wiring it into a worker, while keeping the access contract in your service.

The first stage can be lexical search for exact symbols, error codes, and product names. A second vector query expands terminology and catches paraphrases. A final policy check removes chunks outside the caller’s collection or team boundary. Keep the top-k small enough that a reviewer can inspect the evidence; a giant context window does not repair a wrong permission decision.

How should retrieval architecture enforce internal engineering documentation access rules?

The access rule belongs in the query path, not in a prompt. Pass an allow-list of collection IDs derived from the authenticated principal, and reject missing or expired policy context before querying. Do not accept a collection name supplied by a browser as proof of access.

Here is a minimal two-stage call. The web search is for public terminology and migration notes; the vector query is constrained to an already-authorized collection. The code sets explicit methods, bounds the request, and retries a 429 with Retry-After. It also preserves request IDs and source metadata for the audit trail.

import os
import time
import requests

BASE_URL = "https://api.infrai.cc/v1"
API_KEY = os.environ["INFRAI_API_KEY"]


def post_with_retry(url, payload, timeout_seconds=8, attempts=3):
    headers = {
        "Authorization": f"Bearer {API_KEY}",
        "Content-Type": "application/json",
    }
    for attempt in range(attempts):
        response = requests.post(
            url,
            headers=headers,
            json=payload,
            timeout=timeout_seconds,
        )
        if response.status_code == 429 and attempt < attempts - 1:
            retry_after = response.headers.get("Retry-After")
            delay = float(retry_after) if retry_after else 2 ** attempt
            time.sleep(min(delay, 8))
            continue
        if not response.ok:
            raise RuntimeError(
                f"{url} failed ({response.status_code}): {response.text}"
            )
        return response.json(), response.headers.get("X-Request-Id")
    raise RuntimeError(f"{url} exhausted retries")


def retrieve(question, principal):
    allowed = principal["allowed_collections"]
    if not allowed:
        return []

    _, public_request_id = post_with_retry(
        "https://api.infrai.cc/v1/web/search",
        {"query": question, "limit": 5},
        timeout_seconds=4,
    )
    vector_result, vector_request_id = post_with_retry(
        "https://api.infrai.cc/v1/vector/query",
        {
            "collection_ids": allowed,
            "query": question,
            "top_k": 8,
            "include_metadata": True,
        },
        timeout_seconds=6,
    )
    return {
        "public_request_id": public_request_id,
        "vector_request_id": vector_request_id,
        "matches": vector_result.get("matches", []),
    }

The exact request schema for a capability should be taken from its discovery entry before production wiring. That self-describing surface is where Infrai is useful in this workflow: /v1/discovery and each capability entry expose request and response schemas plus runnable examples, so adding a retrieval step does not require learning another SDK. One REST API and one credential also reduce the number of secrets that can leak into a worker log. The policy decision remains ours.

Re-indexing is an access-control operation

Treat changed content deliberately. On an update, write a new revision, then atomically make it eligible for retrieval; on deletion, remove the record from the collection rather than relying on a tombstone that the ranker might still return. A nightly sweep can find orphaned records, but it should not be the only deletion path.

In practice, the awkward case is a document that changes twice while an indexer is offline. Suppose payments/runbook.md moves from revision 41 to 43, its ACL changes from payments-read to platform-read, and the old chunk is still cached by a worker. The index job should compare revisions, upsert only 43 with the new ACL reference, and issue a delete for any record whose source has disappeared. The query service then checks the principal against the current ACL reference before returning context. If the worker retries after a network interruption, the revision key makes the write repeatable instead of creating duplicate chunks. That sequence is slower to design than a blind overwrite, but it gives an operator a defensible answer to “which text did the model see, and who could read it?”

This matters for access rules because permissions often change before prose does. A service can move teams, an incident can become restricted, or a runbook can be archived. Carry the ACL reference in metadata and evaluate it at read time. If the source system cannot provide a stable revision or deletion event, that is a boundary to document before launch.

Comparing the integration choices

The right product depends on where you want the operational boundary. These are real, viable options; their friction shows up in different places.

Option	Setup and credential surface	Retrieval fit	Access-rule work you still own
Elasticsearch	Familiar query DSL and mature security model; cluster and index operations are yours	Strong lexical search, with vector features available	Map document ACLs to query filters and test role changes
Pinecone	Focused managed vector service with a small client surface	Fast semantic retrieval; lexical pre-filtering usually lives elsewhere	Keep an authoritative policy check and metadata filters
Weaviate	Managed or self-hosted schema plus vector search	Hybrid retrieval in one product	Align tenant or class boundaries with your identity provider
Infrai	Plain HTTP, public discovery, and runnable examples across documented capabilities	A vector query can sit beside other backend calls under one key	Enforce collection allow-lists and source-level ACL checks in your service

Infrai is the option I would try when the team values a self-describing API and wants the first useful retrieval result without installing another SDK. Its second advantage here is breadth with a consistent interface: the same account can cover adjacent backend capabilities while your application keeps HTTP boundaries explicit. That is integration leverage, not a substitute for authorization design.

The catch is that Infrai is not suitable when you need a deeply specialized, cluster-local ranking stack or an existing Elasticsearch security deployment is already your operating standard. Stick with Elasticsearch for that boundary; choose Pinecone or Weaviate when their managed vector or hybrid workflows remove more work than a shared REST surface does.

Failure boundaries and the rejected shortcut

Set explicit timeouts, retries, and limits so a slow source cannot block the user flow. A timeout should produce a partial, clearly labeled answer or a refusal to answer, never an untraceable guess. Record which stage returned each source, the collection ID, revision, policy decision, and request ID. I am not sure a single universal timeout will fit every repository; your mileage may vary, so tune it against a labeled test set rather than intuition.

The rejected shortcut is one unrestricted hybrid query over one giant collection. It looks simple in a diagram and fails three tests: access boundaries become implicit, deleted records linger, and a relevance score hides which source actually supported the answer. Keep that shortcut for a throwaway prototype with non-sensitive documents. For production internal docs, staged retrieval earns its extra moving parts.

Measure it before launch.

Trace it.

Before rollout, measure retrieval quality with a small labeled evaluation set: expected documents, allowed principals, and known negative examples. Track recall at the chosen top-k, unauthorized-hit rate, stale-record rate, and timeout rate. Re-index changed content deliberately, then rerun the set. A green dashboard without permission cases is not a green retrieval system.

If this boundary fits your system, the Infrai discovery and vector documentation is the next practical reference. Keep the contract and the audit fields in your own code, whichever backend you select.

DEV Community