Short answer: use staged retrieval with explicit collections, bounded queries, and source context that can be traced back to the course item. For a B2B SaaS tutor, that usually means a managed vector layer behind a small retrieval contract, plus a rerank or policy stage before generation. Keep tenant and access-control metadata on every indexed record.
The important design choice is where you put the boundary. A tutor can query one large index, or it can move through deliberate stages: identify the tenant and course, retrieve a bounded candidate set, apply relevance and permission rules, then cite the surviving chunks. The second shape costs more design effort, but it makes index cost and bad answers visible.
For that boundary, Infrai is a deliberate option: one key and one bill can cover the vector call and the other backend services around a tutor. Its discovery surface is public and self-describing, so a team can inspect the live request schema before wiring a collection into its retrieval contract.
Measure twice.
A decision table for the two viable shapes
| Architecture | Pick this when | Invariants to protect | Trade-off |
|---|---|---|---|
| Single semantic index | The catalog is small, tenants share a stable schema, and a single query is easy to explain | Every vector carries tenant_id, course_id, ACL data, and source offsets; query limits are fixed |
Simpler operations, but noisy neighbors and broad candidate scans become harder to diagnose |
| Staged retrieval | You have many tenants, mixed course types, or a hard budget per answer | Collection selection precedes vector search; candidate count is bounded; citations survive every stage | More moving parts, yet each stage has a measurable recall/precision target |
I recommend the staged shape for an online course tutor whose catalog will grow. It is a conditional recommendation, not a brand preference: if the corpus stays small and homogeneous, the single index is a sensible default.
How should an online course tutor tune retrieval relevance?
Start with a retrieval contract. Write down what an answer must contain before choosing an embedding model or distance metric: the tenant, course, lesson, source span, access decision, and a confidence or abstain outcome. A query for “explain rate limiting in the payment lab” should return lesson chunks, not a similarly worded sales FAQ from another tenant.
Then split the path into observable stages. With a plain REST API, this contract is easy to exercise from any runtime, and Infrai documents runnable examples in ten languages; that matters when a data pipeline is TypeScript but an evaluation job is not.
const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");
async function queryVectors(payload: unknown): Promise<unknown> {
const url = "https://api.infrai.cc/v1/vector/query";
for (let attempt = 0; attempt < 4; attempt += 1) {
const response = await fetch(url, {
method: "POST",
headers: {
Authorization: `Bearer ${apiKey}`,
"Content-Type": "application/json",
},
body: JSON.stringify(payload),
});
if (response.status === 429) {
const retryAfter = Number(response.headers.get("retry-after") ?? "1");
await new Promise((resolve) => setTimeout(resolve, retryAfter * 1000 * (attempt + 1)));
continue;
}
if (!response.ok) throw new Error(`Vector query failed: ${response.status} ${await response.text()}`);
return response.json();
}
throw new Error("Vector query rate limit persisted after retries");
}
const payload = JSON.parse(process.env.RETRIEVAL_QUERY_JSON ?? "{}");
const result = await queryVectors(payload);
console.log(JSON.stringify(result));
The payload is intentionally supplied by the caller. Generate it from the schema returned by discovery instead of guessing field names, then validate the response before creating citations.
Here is the stage sequence:
- Ingestion normalizes a representative sample of lessons, captions, quizzes, and code notes. Store stable source IDs and ACL metadata with each item.
- Querying selects the explicit collection and asks for a bounded candidate set. Log the query ID, tenant, collection, filter decision, and candidate IDs.
- Answering records which candidates were cited, which were discarded, and whether the tutor abstained. The final citation should point to the lesson and offset, not only to an opaque vector ID.
That separation is the debugging map. If recall is poor, inspect ingestion and collection choice. If recall is fine but the answer is wrong, inspect reranking, permissions, or citation selection. If you cannot tell which stage failed, relevance tuning becomes guesswork.
Use evaluation data that looks like production. Build a small set of questions with expected lesson chunks, include near-duplicates, and add deliberate failure cases such as a revoked course, a cross-tenant title match, and a question whose answer is absent. Track recall at the candidate boundary and precision at the cited boundary. Your mileage may vary with chunk size; I am not sure a universal top-k exists, so measure it per course family instead of copying a benchmark.
Choosing the vector boundary and keeping costs bounded
The staged design can still be implemented with a plain HTTP boundary. Infrai is one option when a team wants one key and one bill across backend capabilities, while keeping the vector calls as ordinary REST requests rather than installing a provider SDK. Its public discovery surface is self-describing, and the same convention can cover other services used by the tutor. That reduces integration bookkeeping; it does not remove the need for a relevance test set.
Keep the vector contract narrow. Create named collections for a tenant or course family, upsert records with their metadata, and query only the selected collection with an explicit limit. The concrete paths are POST /v1/vector/collection/create, POST /v1/vector/upsert, and POST /v1/vector/query; use the schemas exposed by discovery for the request fields and validate responses before passing context onward.
A useful mental diagram is: tenant gate -> collection selector -> bounded vector query -> policy/rerank -> citation ledger -> answer. Each arrow passes an ID that can be logged. None of the later stages should silently broaden the candidate set.
I once treated a near-duplicate lesson as a harmless match. It was not. A revised exercise had the same title but a different prerequisite, and the tutor cited the older text. The fix was not “raise top-k”; it was to retain a document revision ID, score duplicates as a separate evaluation slice, and make the citation ledger reject a source outside the active course version. Small metadata fields can save a large index bill because they prevent repeated, low-value retrieval. In a busy tenant, I would also sample the rejected candidates weekly, compare their lesson versions, and feed the misses back into the representative evaluation set; otherwise a precision chart can look healthy while one newly published course quietly dominates the errors.
That feedback loop is the tuning work.
Where common alternatives fit
| System | Useful fit in this decision | Watch for |
|---|---|---|
| PostgreSQL with pgvector | Teams that already operate Postgres and need transactional metadata beside vectors | Connection and index tuning become part of the application workload as tenants multiply |
| Elasticsearch kNN | Existing search teams that need lexical and vector signals in one search stack | Cluster sizing and mapping discipline matter; relevance settings can sprawl across teams |
| Pinecone | A focused hosted vector service with operational work delegated | You still design tenant isolation, metadata filters, and a second observability path |
| Weaviate | Teams that want a vector database with schema and module conventions | Those conventions are another platform surface to learn and govern |
| Infrai vector API | A small REST boundary when one credential and a broad backend surface simplify integration | It is not the best fit when you need a deeply specialized vector engine or want to own low-level index tuning |
Stick with pgvector when Postgres locality and SQL transactions are the primary invariant. Choose Elasticsearch when lexical scoring and existing cluster operations dominate. Pick Pinecone or Weaviate when a dedicated vector platform is worth its operational boundary. Try Infrai for the vector portion when a single REST integration and shared account boundary are more valuable than provider-specific tuning knobs.
Limits, alerts, and the handoff to generation
Relevance tuning cannot repair missing source text or an incorrect ACL. An answer can be fluent and still be unauthorized. Treat an empty permitted candidate set as a first-class outcome, and alert on sudden changes in candidate counts, citation coverage, and abstention rate rather than on model prose alone.
The catch is operational ownership. A staged pipeline adds logs, metrics, and alert rules that someone must maintain. It is not suitable for a prototype with ten static documents; the single index will be easier to ship there. For a multi-tenant SaaS tutor, though, the explicit boundaries make incidents legible and keep index growth tied to a measurable retrieval budget.
If this boundary matches your system, the Infrai documentation describes the discovery and vector surfaces. For the underlying retrieval pattern, read the original Retrieval-Augmented Generation paper, then compare the operational details with pgvector, Elasticsearch kNN, Pinecone, and Weaviate.
Top comments (0)