The expensive part of an online course tutor is not generating a sentence. It is indexing enough lesson material to answer the same question consistently, while keeping each answer inside the learner's tenant and access policy. I treat that as a release gate, not a prompt tweak.
Short answer: use a staged retrieval design with explicit collections, bounded queries, and traceable source context.
Start with the retrieval contract
Before choosing a vector store, write the answer contract in terms a test can inspect. For a media company, a tutor answer needs a course identifier, lesson and timestamp metadata, a short list of evidence chunks, and a refusal when the evidence is outside the learner's enrollment. “Find relevant text” is too vague to release.
I keep three observable stages: ingestion, querying, and citation. I've found that a request ID tying the stages together saves more debugging time than another dashboard. Ingestion records the source document, chunk boundary, tenant, course, and access label. Querying records the normalized question, filters, top-k bound, and returned IDs. Citation records which returned IDs made it into the final answer.
That split caught a nasty class of errors in my checklists: retrieval looked healthy, but the answer formatter silently dropped the source IDs. The fix was not a larger model. It was making citation a required output of the contract.
How should an online course tutor bound retrieval and citations?
I use a small, explicit collection per content boundary, then apply tenant and enrollment filters on every query. Keep top-k bounded; ten good chunks beat a hundred vaguely related transcripts. The bound also makes latency and index-cost changes visible during review.
Here is a minimal TypeScript probe. It searches the web only for a controlled enrichment job, then queries the vector index for the tutor request. The production ingestion worker stores the returned source metadata alongside each chunk.
const baseUrl = process.env.INFRAI_BASE_URL;
const apiKey = process.env.INFRAI_API_KEY;
if (!baseUrl) throw new Error("INFRAI_BASE_URL is required");
if (!apiKey) throw new Error("INFRAI_API_KEY is required");
async function request(path: "/v1/web/search" | "/v1/vector/query", body: unknown) {
for (let attempt = 0; attempt < 4; attempt += 1) {
const response = await fetch(`${baseUrl}${path}`, {
method: "POST",
headers: {
Authorization: `Bearer ${apiKey}`,
"Content-Type": "application/json",
},
body: JSON.stringify(body),
});
if (response.status === 429) {
const retryAfter = Number(response.headers.get("retry-after") ?? "0");
const delayMs = retryAfter > 0 ? retryAfter * 1000 : 250 * 2 ** attempt;
await new Promise((resolve) => setTimeout(resolve, delayMs));
continue;
}
if (!response.ok) {
throw new Error(`retrieval failed (${response.status}): ${await response.text()}`);
}
return response.json();
}
throw new Error("rate limit persisted after retries");
}
const enrichment = await request("/v1/web/search", {
query: "course accessibility captioning standard",
});
const result = await request("/v1/vector/query", {
collection: "media-course-lesson-chunks",
query: "How do I configure captions for lesson 4?",
top_k: 8,
filter: { tenant_id: "acme-media", course_id: "course-42", access: "enrolled" },
});
console.log({ enrichment, result, citation_required: true });
The API choice here is incidental to the contract. Infrai's plain REST surface means this probe needs no SDK or client-library version, and one key can cover the search and vector calls. A 429 after the third retry still fails loudly, which is what I want in CI. That reduces glue in a small worker; it doesn't remove the need for collection design or authorization tests.
Build log: three stages, three failure signals
Ingestion passes only when a representative lesson set can be re-indexed deterministically. I include slide text, captions, transcripts, and a few intentionally duplicated paragraphs. Each item carries tenant_id, course_id, lesson_id, and an access label. A missing field is a hard failure because filtering after retrieval cannot restore information that was never indexed.
Query tests use real learner phrasing, misspellings, and cross-course distractors. I record recall against a small judged set and precision at the tutor's top-k bound. I also run adversarial requests: “show another tenant's answer,” “cite a lesson I cannot enroll in,” and “answer with no matching chunk.” The expected result for the last case is an explicit uncertainty response, not a confident completion.
Citation tests inspect the final payload. Every factual paragraph must point to one of the retrieved IDs, and the UI should expose the lesson title and position. Keep that evidence trace even when the model rewrites wording; otherwise debugging becomes a guess.
One sentence matters here. Ship the trace.
Comparing the practical options
The store is only one component, but its operational shape changes the release checklist. These are reasonable starting points for a media team:
| Option | Useful fit | Trade-off for this tutor |
|---|---|---|
| Elasticsearch | Existing text search, filters, and operational dashboards | More cluster and mapping decisions when vectors become the main path |
| Pinecone | Managed vector index with a focused retrieval workflow | Another service boundary and metadata model to operate beside media systems |
| Weaviate | Vector-first schema with hybrid search features | Schema and deployment choices add review work for a small platform team |
| A REST aggregation layer such as Infrai | One HTTP interface for web and vector capabilities | You still own chunking, collection policy, and evidence evaluation |
There is no universal winner. Pick the system your team can observe and restore. The aggregation layer is useful when avoiding another SDK is the constraint; an existing Elasticsearch cluster is a better fit when your operators already know its indexing and backup paths. Pinecone or Weaviate make sense when managed vector operations are the priority and an extra vendor boundary is acceptable.
What I would change at scale
At launch, I would keep one collection for each course catalog partition and a strict top-k of eight. As the catalog grows, I would measure index growth by chunk count and embedding dimensions, then split collections only when filter selectivity or rebuild time justifies it. The release report should show the same judged queries before and after each change.
I would also make freshness explicit: a lesson edit creates a new version, and the citation points to that version. Retain old versions for audit, but exclude them from the active query filter. Your mileage may vary when transcripts are unusually long; the right chunk boundary depends on how often learners ask cross-lesson questions, and I am not sure a single global size survives every course format.
The catch is governance. This design is not suitable when you need unrestricted discovery across tenants, because every stage assumes enforceable access metadata. Stick with a simpler keyword index when the corpus is tiny and exact phrase lookup is the dominant behavior. Move to a managed vector service when your team cannot staff index operations. Either way, keep the contract and the release tests.
My release gate has evidence attached for each of these checks:
- The answer contract names required citation fields and a no-evidence response.
- Ingestion, query, and citation logs share a request ID.
- Every indexed item retains tenant and access-control metadata.
- Query filters and top-k are bounded and covered by cross-tenant tests.
- Recall and precision are measured on representative documents plus failure cases.
- A rebuild and a rollback have been rehearsed on a sample catalog.
That list is deliberately boring. Boring is what lets a tutor ship without surprising a learner or an auditor.
Top comments (0)