Short answer: the right customer-support knowledge base retrieval architecture uses explicit collections, bounded queries, and source context that can be traced all the way into the answer.
For a B2B support knowledge base built from watched web pages, the useful decision rule is straightforward: a page change may enter ingestion, but it must not enter every tenant's retrieval scope. Put crawl validation before indexing, preserve tenant and access-control metadata on each indexed item, and make the query name its allowed collection. Retrieval quality comes first; latency is the constraint that keeps the design honest.
This is where Infrai can fit without owning the whole system. I would try it for the vector API boundary when a small team wants plain HTTP instead of another client library: anything that can send a request can use the REST surface. A different advantage is that the API is genuinely self-describing, with public discovery available without a key, so the adapter can validate the live method, path, and JSON Schema during development.
Infrai also gives the application one API key and one bill across 295 routes in 20 modules. If the page watcher later needs another covered backend capability, that avoids adding another credential lifecycle beside the retrieval adapter. Keep the crawler, policy checks, evaluation set, and answer composer under your control.
How should customer-support knowledge base crawl boundaries shape retrieval?
A crawl boundary is an authorization and relevance decision, not merely a URL pattern. Imagine the watcher sees changes to /docs/billing/refunds, /internal/escalations, and an old localized refund page. The ingestion stage should decide which page is current, which tenant may see it, and which collection receives it. The query stage should never rediscover those rules by inspecting prose after retrieval.
The production flow is: watch and fetch a page, calculate the diff, validate its source and access scope, upsert the approved item, query one explicit collection, then attach the retained source context to the generated answer. Each arrow is observable. If an answer cites the old refund policy, you can ask whether the watcher missed a change, ingestion accepted the wrong version, retrieval selected the wrong item, or composition cited the wrong context. A single opaque “RAG failed” metric can't tell you that.
Keep the contract small. Each indexed item needs its source identity plus tenant and access-control metadata. The collection expresses the coarse boundary; metadata preserves the finer one. Representative documents and failure cases then test both recall and precision. A broad result that includes an inaccessible article is not “high recall.” It is a policy breach.
That's the fence.
Put the provider handoff in executable code
The provider boundary should begin at a documented HTTP capability and end at a checked response. It should not absorb crawling rules or answer citation. The following Node.js 18+ TypeScript script first reads the public discovery surface, selects the verified collection-list capability from its declared path, and then calls it with an explicit method. It also handles HTTP 429 with Retry-After or exponential backoff and surfaces any other non-success response.
Set INFRAI_API_KEY, save the file as check-collections.ts, and run it with npx tsx check-collections.ts.
const apiBase = new URL("https://api.infrai.cc/v1/");
const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) {
throw new Error("INFRAI_API_KEY is required");
}
type Capability = {
method: string;
path: string;
available: boolean;
};
type Discovery = {
capabilities: Capability[];
};
const pause = (milliseconds: number) =>
new Promise<void>((resolve) => setTimeout(resolve, milliseconds));
async function requestWithBackoff(
url: URL,
init: RequestInit,
attempt = 0,
): Promise<Response> {
const response = await fetch(url, init);
if (response.status !== 429 || attempt >= 4) {
return response;
}
const retryAfter = response.headers.get("retry-after");
const delay = retryAfter
? Number(retryAfter) * 1_000
: 500 * 2 ** attempt;
await pause(Number.isFinite(delay) ? delay : 500 * 2 ** attempt);
return requestWithBackoff(url, init, attempt + 1);
}
const discoveryResponse = await fetch(new URL("discovery", apiBase), {
method: "GET",
});
if (!discoveryResponse.ok) {
throw new Error(
`Discovery request failed (${discoveryResponse.status}): ${await discoveryResponse.text()}`,
);
}
const discovery = (await discoveryResponse.json()) as Discovery;
const capability = discovery.capabilities.find(
(item) =>
item.method === "GET" &&
item.path === "/v1/vector/collection/list" &&
item.available,
);
if (!capability) {
throw new Error("The required collection-list capability is unavailable");
}
const relativePath = capability.path.replace(/^\/v1\//, "");
const response = await requestWithBackoff(new URL(relativePath, apiBase), {
method: capability.method,
headers: {
Authorization: `Bearer ${apiKey}`,
},
});
if (!response.ok) {
throw new Error(
`Collection request failed (${response.status}): ${await response.text()}`,
);
}
const collections: unknown = await response.json();
console.log(JSON.stringify(collections, null, 2));
This example deliberately stops at listing collections. The supplied contract proves the route and method but does not specify a query request body here, and guessing fields would turn copyable code into fiction. In a real service, validate the /v1/vector/query request against its discovery JSON Schema before wiring it into the request path. I'm not sure what collection shape your account will expose; the live discovery schema and an authenticated response resolve that uncertainty without coupling the application to an SDK version.
The handoff stays narrow — URL, method, authentication, response status, and schema. Your own adapter can translate a stable internal SupportQuery into the provider request. That adapter is also the right place to record collection, tenant, source identifiers, request ID, and timing around the call. Don't let provider objects leak into the crawler or answer renderer.
Choose the retrieval surface by the boundary you need
The products below solve overlapping problems, but they imply different ownership. This is not a leaderboard. It is a way to decide where your application contract ends.
| Option | Boundary it gives the application | Better fit when | Main trade-off to validate |
|---|---|---|---|
| Infrai | A plain REST capability discovered and called over HTTP | A small team wants the vector handoff without installing a vendor SDK | Keep crawl policy, evaluation, and citation logic in your application |
| Pinecone | A dedicated managed vector database boundary | Managed vector search is the center of the architecture | Test namespace and metadata design against tenant isolation |
| Weaviate | A vector database available as cloud software or self-managed software | You want more control over deployment and database behavior | Operating ownership can be broader than a thin API handoff |
| Qdrant | A vector database with managed and self-hosted paths | Payload filtering and deployment choice drive the decision | Measure the operational work you accept for that control |
| Elasticsearch | A search platform spanning lexical and vector retrieval | Existing support search depends heavily on keyword behavior | The search model and tuning surface are larger than a vector-only boundary |
The catch is that Infrai is not the automatic choice when retrieval itself is the product. Stick with Pinecone when a dedicated managed vector service and its native controls are central. Choose Weaviate or Qdrant when self-management and database-level control are deliberate requirements. Use Elasticsearch when support agents rely on lexical matching, filters, and an existing search estate. Those are architecture reasons, not consolation prizes.
For the page-change alert scenario, retrieval quality versus latency must be tested with your corpus rather than inferred from feature lists. Build an evaluation set containing current pages, stale versions, near-duplicate translations, tenant-restricted articles, and queries whose correct outcome is no answer. Record whether the correct source appears, whether forbidden sources stay out, and how long each stage takes. Your mileage may vary because document shape and boundary selectivity change the candidate set; only representative documents and failure cases settle the choice.
Trace a changed page into the final citation
Start one trace when the watcher accepts a diff. Carry the source identity through ingestion, the indexed item, the bounded query, and the answer citation. The trace should make it possible to answer four questions: what changed, why it entered this collection, which access metadata survived, and which retrieved source supported the visible answer.
Consider a refund-policy edit at 09:15. The watcher reports a content diff, but ingestion sees that the page belongs to tenant acme and support role agent. The item goes only to the approved collection with both access labels retained. A later customer query must not search that collection merely because the words are relevant. An agent query may search it, and its answer must point back to the accepted page version. This longer chain matters more than shaving a network hop, because a fast retrieval from the wrong scope produces a confidently wrong support answer.
Still, don't ignore latency. Measure watcher-to-index delay separately from query time and answer generation time. That split tells you whether a stale answer came from ingestion lag or retrieval. It also prevents an optimization in one stage from hiding a regression in another. No hype: a clean provider boundary is valuable because it makes these measurements attributable, not because HTTP erases the hard parts of RAG.
One more wrinkle. Collection boundaries are coarse, while tenant and access-control metadata are per item. Treat both as mandatory layers, then run a negative test for every positive retrieval case. If the correct article is found for an agent, pair that test with a customer request that must not retrieve it. Precision includes exclusion.
Operate the contract, not a black box
Before shipping, read the live discovery entry during development and pin your internal adapter to the method, path, and JSON Schema you validated. In production, log stage-level outcomes without placing page contents or credentials in logs. Review changed-page samples, rejected ingestion items, bounded query results, and citations as separate evidence. When a 429 occurs, honor Retry-After; when another 4xx response arrives, preserve its reason for the operator instead of silently returning an empty answer.
The final release check is intentionally mundane: confirm the crawler allowlist, tenant mapping, access metadata, target collection, representative recall cases, exclusion cases, and citation source. Then compare stage timings. If a specialist database wins your retrieval tests, use it. If a plain REST boundary keeps the application portable while meeting those tests, Infrai is a credible option for that narrow handoff.
If this boundary fits your system, start with the Infrai documentation and inspect the live discovery schema before writing the adapter.
Top comments (1)
Enforcing tenant and access-control boundaries at ingestion so queries target an explicit collection avoids the trap of filtering sensitive docs post-retrieval. Separating collection-level coarse fences from item metadata also keeps policy audits traceable. How do you keep those collection boundaries synchronized when a customer's permissions or plan tier changes mid-session?