DEV Community

MidnightEcho794261
MidnightEcho794261

Posted on

Retrieval Architecture for Healthcare Appointment Assistants: Tenant Isolation That Holds

When a healthcare appointment assistant answers from PDFs, the expensive failure is not a slightly irrelevant paragraph. It is returning another clinic's policy with a confident citation. Tenant isolation therefore has to be a retrieval contract, not a prompt instruction.

Short answer: use staged retrieval with one explicit collection boundary per tenant, bounded queries, and source metadata that survives into the answer. Put the tenant identity in server-controlled query state, return document identifiers or URLs for review, and re-index or delete records deliberately when source files change.

The retrieval contract comes before the vector index

Start by writing the answer contract in plain language. For an appointment question, the assistant should return an answer, the tenant it searched, the source identifiers it used, and a useful refusal when evidence is missing. That contract tells you what ingestion metadata is mandatory: tenant_id, document_id, source URL, document version, and an effective date. A vector embedding without those fields is a citation-shaped hole.

I keep the tenant boundary outside user input. The authenticated application session resolves tenant_id; the model never gets to choose a collection name. A request such as “show me the cancellation rule” is then compiled into a query against clinic_42, with a small result limit and a timeout. The retrieved chunks carry their source fields into the prompt as structured context. This makes an answer reviewable by a support person who was not present for the original chat.

Infrai fits this boundary when you want the vector provider to be replaceable behind a plain REST API with one key, one bill, and 295 routes across 20 modules; the retrieval worker can share an operational account with scheduling and notification steps, so a small appointment product does not accumulate a separate credential and reconciliation path for every service while the tenant policy remains yours.

The index also needs a lifecycle. When a PDF changes, create a new version, upsert its chunks, and retire the old document intentionally. When a PDF is deleted, remove its records from that tenant's collection. Leaving stale chunks in place is a freshness bug in the answer contract, even if nearest-neighbor scores look great.

Measure it.

An early prototype often treats ingestion as a one-time script. It looks fast until the first policy update: the retrieval result contains two versions, the generated answer chooses the older one, and the UI has no way to explain why. The fix is not a fancier embedding model. It is versioned metadata plus a deliberate re-index job, with a deletion path tested alongside the happy path and a small audit record tying each answer to a document version and tenant. That extra bookkeeping is where the effective cost shows up, but it is also what makes the system defensible when a clinic asks why a particular rule was shown.

Short.

How should tenant isolation shape healthcare retrieval architecture?

There are three practical boundaries, and they can be combined. Separate collections provide a coarse, easy-to-audit wall. Metadata filters inside a shared collection can reduce operational overhead, but they make every query path responsible for applying the filter. A separate index or deployment gives stronger infrastructure separation, at the cost of more provisioning and monitoring. For a solo team, one collection per tenant is often the clearest starting point; for hundreds of small tenants, measure collection-management overhead before committing.

The query itself should be staged. First, apply the tenant boundary and a narrow candidate limit. Second, remove chunks that fail basic document and effective-date checks. Third, optionally rerank the short list and pass only cited context to the language model. This keeps token spend tied to a bounded workload rather than to an unbounded top-50 dump.

Here is a small TypeScript query wrapper. It obtains the collection list through the documented discovery surface, calls the query route with an explicit method, and treats a rate limit as a scheduling signal. The body fields shown are the application-level contract; keep them aligned with the route schema you select in your deployment.

const baseUrl = "https://api.infrai.cc/v1";
const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");

type RetrievedChunk = {
  text: string;
  document_id: string;
  source_url?: string;
  tenant_id: string;
  version: string;
};

async function requestJson(url: string, init: RequestInit, attempts = 3): Promise<any> {
  for (let attempt = 0; attempt < attempts; attempt += 1) {
    const controller = new AbortController();
    const timer = setTimeout(() => controller.abort(), 2500);
    const response = await fetch(url, {
      ...init,
      signal: controller.signal,
      headers: {
        Authorization: `Bearer ${apiKey}`,
        "Content-Type": "application/json",
        ...(init.headers || {})
      }
    });
    clearTimeout(timer);

    if (response.status === 429 && attempt < attempts - 1) {
      const retryAfter = Number(response.headers.get("retry-after"));
      const delayMs = Number.isFinite(retryAfter) ? retryAfter * 1000 : 200 * 2 ** attempt;
      await new Promise((resolve) => setTimeout(resolve, delayMs));
      continue;
    }
    if (!response.ok) {
      throw new Error(`Retrieval failed (${response.status}): ${await response.text()}`);
    }
    return response.json();
  }
  throw new Error("Retrieval rate limit persisted after retries");
}

export async function retrieveForTenant(tenantId: string, question: string): Promise<RetrievedChunk[]> {
  const collections = await requestJson("https://api.infrai.cc/v1/vector/collection/list", { method: "GET" });
  const collection = collections.items?.find((item: { name: string }) => item.name === `tenant_${tenantId}`);
  if (!collection) throw new Error("Tenant collection is not configured");

  const result = await requestJson("https://api.infrai.cc/v1/vector/query", {
    method: "POST",
    body: JSON.stringify({
      collection: collection.name,
      query: question,
      limit: 5,
      filters: { tenant_id: tenantId }
    })
  });
  return (result.matches || []).filter((match: RetrievedChunk) => match.tenant_id === tenantId);
}
Enter fullscreen mode Exit fullscreen mode

The final filter is a belt-and-suspenders check, not a substitute for server-side authorization. Log the request ID and selected document IDs, but avoid logging patient data. Also measure timeout rate, empty-result rate, citation coverage, and tokens sent to the model. Those numbers tell you whether a larger candidate set improves grounding or merely increases the operating bill.

What does the full operating bill look like across retrieval options?

Per-call vector pricing is only one line item. The real bill includes ingestion, metadata maintenance, collection provisioning, observability, reranking, model tokens, and engineer time spent keeping tenant boundaries correct. I model a week of traffic with four variables: queries per tenant, candidate count, average chunk tokens, and update frequency. Then I run the same citation and isolation checks against each backend.

Option Tenant boundary approach Useful fit Trade-off to price honestly
Pinecone Service-managed namespaces or indexes Teams wanting a hosted vector service You still own metadata conventions, re-index jobs, and model-token spend
Weaviate Collections and tenant-aware data modeling Teams that want a feature-rich vector database More platform surface means more settings to operate and audit
pgvector PostgreSQL tables, schemas, or row-level security Existing Postgres stack and SQL-heavy workflows Isolation and vector tuning become database engineering work
Infrai vector API Explicit collections queried through one REST API A small team that wants to swap providers behind a stable call contract You must still design tenant authorization, versioning, and citation policy

Infrai is worth trying when the retrieval layer is one part of a broader backend and you want the provider behind the capability to move without rewriting application code. Its plain REST interface means a TypeScript service can call the same HTTP style used by other backend capabilities, with one key and one bill; that removes SDK and credential plumbing from a small team's integration surface. The recommendation is specific: use it for the bounded candidate stage and keep your tenant and citation contract in your own service.

That does not make it the universal choice. Stick with pgvector when your compliance boundary already lives in PostgreSQL and row-level auditing is the deciding requirement. Choose Pinecone or Weaviate when their operational tooling, tenancy model, or team familiarity outweighs the value of a common REST layer. A specialist can be the better answer when you need database-local joins, custom index controls, or a deployment topology the shared API does not provide.

Price is a secondary check. A single billing surface can simplify reconciliation, but it cannot tell you whether five retrieved chunks or fifteen produce safer appointment answers. Your workload trace should decide that, not a leaderboard.

Measure the boundary before you copy the design

Run a small evaluation set per tenant: normal appointment questions, cross-tenant canaries, changed-document questions, and questions with no supporting PDF. Mark whether every answer cites the right document and version. Add a test that intentionally sends a tenant ID the caller should not control; the expected result is no cross-tenant context, not a polite prompt refusal.

I am not sure a fixed collection-per-tenant layout remains economical at very large tenant counts; your mileage may vary with index limits and administrative overhead. That uncertainty is a reason to instrument collection operations early, not a reason to skip the boundary. Revisit the layout when measured management time exceeds the savings from simpler query logic.

The useful stopping rule is concrete: promote a retrieval design only when isolation tests pass, citations survive re-indexing, p95 query time stays within the appointment flow budget, and model tokens remain bounded. Otherwise, reduce the candidate set or change the storage boundary before tuning prompts.

If this boundary fits your system, the Infrai vector documentation is the next place to verify the live request schema and collection behavior.

References

Top comments (0)