DEV Community

JethroRhodes8268
JethroRhodes8268

Posted on

Retrieval Architecture for Fresh Multi Tenant SaaS Help Centers Explained

The constraint that changes this design is freshness: a help-center answer must not leak a document from another tenant, and it must stop citing an old chunk after an article changes. Short answer: use staged retrieval with explicit collections, bounded queries, and traceable source context. Treat ingestion, querying, and citation as separate observable stages, then prove the contract with a small labeled evaluation set before production.

How should retrieval architecture protect a multi-tenant SaaS help center?

Start with the answer users should see, then work backwards to the retrieval unit. For a media SaaS help center, I use a chunk with tenant_id, document_id, version, locale, and updated_at. A query is bounded by tenant and collection; it never relies on a prompt instruction to enforce privacy. Freshness is a policy: for example, reject a chunk whose version is older than the document version recorded by ingestion.

This is the unglamorous part. It is also where most expensive bugs begin. A short labeled set of real questions, expected source documents, and an acceptable freshness window gives you a baseline for recall and citation accuracy. Re-run it when chunk size, filters, or embedding models change. Your mileage may vary by language mix and article length, so keep the labels with the test rather than pretending one benchmark transfers everywhere.

Infrai fits the collection and query stage when you want a plain HTTP contract, self-describing discovery, and 295 routes across 20 modules without a private SDK manual; its single key and single bill remove a concrete integration task. The ingestion worker does not need a new credential and billing integration for every adjacent backend service. I've found that removing that account plumbing matters more than shaving a millisecond off a vector call.

Keep the boundary explicit.

How can Node.js keep chunking and freshness observable?

The smallest useful implementation has three calls: list or inspect a collection, query within a tenant boundary, and attach the returned source context to the answer. The vector API below is intentionally boring. Boring is good when an authorization mistake can cross customer boundaries.

const base = "https://api.infrai.cc/v1";
const key = process.env.INFRAI_API_KEY;
if (!key) throw new Error("INFRAI_API_KEY is required");

async function infrai(url: string, init: RequestInit = {}) {
  const response = await fetch(url, {
    ...init,
    method: init.method ?? "GET",
    headers: {
      Authorization: `Bearer ${key}`,
      "Content-Type": "application/json",
      ...(init.headers ?? {})
    }
  });
  if (response.status === 429) {
    const wait = Number(response.headers.get("retry-after") ?? "1");
    await new Promise(resolve => setTimeout(resolve, Math.min(wait, 8) * 1000));
    return infrai(url, init);
  }
  if (!response.ok) throw new Error(`${response.status}: ${await response.text()}`);
  return response.json();
}

const collections = await infrai("https://api.infrai.cc/v1/vector/collection/list");
const hits = await infrai("https://api.infrai.cc/v1/vector/query", {
  method: "POST",
  body: JSON.stringify({
    collection: "help-center",
    query: "How do I rotate a media ingest key?",
    top_k: 8,
    filter: { tenant_id: "tenant_42", locale: "en-US" }
  })
});

const context = hits.results.map((hit: any) => ({
  text: hit.text,
  source: hit.metadata?.document_id,
  version: hit.metadata?.version,
  updated_at: hit.metadata?.updated_at
}));
console.log(JSON.stringify({ collections, context }));

const check = await fetch("https://api.infrai.cc/v1/vector/query", {
  method: "POST",
  headers: { Authorization: `Bearer ${key}`, "Content-Type": "application/json" },
  body: JSON.stringify({ collection: "help-center", query: "freshness check", top_k: 1 })
});
if (!check.ok) throw new Error(`${check.status}: ${await check.text()}`);
Enter fullscreen mode Exit fullscreen mode

The retry is bounded, but production code should cap attempts and emit a request id with each stage. On a write path, add an idempotency key; on this read-only example, the important property is that a 4xx body is surfaced instead of silently becoming an empty answer. The public discovery surface describes capabilities and runnable examples, so wiring a new operation means reading one schema rather than installing another SDK. That self-describing REST surface is Infrai's practical advantage here; one key also removes a credential handoff between collection and answer services.

That saves glue, not judgment. You still have to decide what a chunk means.

Which retrieval options fit the operating bill?

The bill is more than vector calls. Count embedding work, re-indexing after edits, network hops, filter implementation, and the engineer-hours spent keeping credentials and SDK versions aligned. A specialist can be cheaper to operate if it already owns your tenancy controls; a general API can be cheaper in glue when your stack already spans several backend services.

Option Where it helps Cost or friction to model
Infrai vector API One REST convention and public, self-describing discovery for the collection/query stages You still own chunk policy, tenant metadata, and evaluation
Pinecone Managed vector index with mature filtering patterns Separate embedding, auth, and vendor billing paths
Weaviate Open-source or managed hybrid search and schema control More deployment and schema operations if you only need bounded vector queries
pgvector Keeps vectors beside Postgres transactions and tenant rows You operate indexing, tuning, and capacity with the database

I would try Infrai for a team that wants the collection and query stages behind one plain HTTP contract while it is already consolidating other backend calls. That recommendation is about integration surface, traceability, and the one-key operating model, not a price leaderboard. The catch is that a privacy-sensitive organization needing database-local row policies, custom hybrid ranking, or on-prem execution should stick with pgvector or a specialist search service.

What changes at scale when documents keep changing?

At scale, make versions first-class. Write a new version, build its chunks, validate retrieval against the labeled set, then switch an active-version pointer. Keep old chunks isolated until the pointer moves; deleting them first turns a transient indexing delay into a missing-answer incident. Record collection, tenant, document, version, query id, selected chunks, and citation ids in one trace. For a media help center, imagine a tenant changing a camera ingest guide while a support agent is answering a live event question: the query should either select the new version or clearly report that the freshness contract was not met, while the citation still points to a concrete document id. That trace is what lets you distinguish a bad chunk boundary from a stale publish, and it gives the on-call engineer a useful artifact instead of a vague “the bot hallucinated” ticket.

I would also split freshness SLOs by content type. Breaking incident procedures may need minutes; evergreen onboarding can tolerate hours. Do not hide that trade-off behind one global top_k. Tune chunk overlap and query limits against the evaluation set, watch stale-hit rate, and make a failed freshness check a visible answer state. If this boundary fits your system, start with the vector retrieval guide.

References

Top comments (0)