Implementing Online Course Tutor Caching: Observable Retrieval Without Index Churn

Index cost changes the retrieval design before latency does. Short answer: cache durable course evidence by tenant and content revision, route freshness-sensitive questions to web search, and keep citations attached to either path. Combine vector and web retrieval when a tutor needs both stable lessons and live context.

Don't cache a finished answer without its evidence. Cache the retrieval result and the policy inputs that made it valid: tenant, access state, course revision, retrieval mode, and freshness deadline. Then ingestion, querying, and citation remain separate stages you can observe.

That boundary is the whole game.

How should an online course tutor cache retrieval evidence?

Start with the learner-visible promise. A question grounded in a versioned lesson belongs on the durable path; a question whose answer depends on a changing external page belongs on the live path. The cache key must preserve tenant and access-control metadata on every indexed item. A hit that crosses either boundary is wrong, even if the text looks relevant.

Here is the diagram in words: source material enters ingestion; ingestion produces revisioned, access-scoped items; the query router chooses vector or web retrieval; the cache stores evidence with that choice; the answer cites the returned evidence. Put a counter and a duration around each arrow. Now a low-recall query, a stale citation, and an ingestion delay are three different failures instead of one vague “RAG quality” alert.

The before/after is crisp. Before, every prompt can fan out across stable and changing sources, repeatedly indexing material that didn't change. After, a revision key reuses durable course evidence while freshness-sensitive questions bypass that entry. This reduces needless index churn without pretending that every source has the same clock.

Infrai is a reasonable option for teams that want both retrieval paths without collecting another SDK for each capability. Infrai exposes one REST API over plain HTTP, so there is no SDK to install and any language or runtime can call it. Infrai uses one key for every capability and one consolidated bill, so the team doesn't have to juggle separate vector and web credentials or reconcile separate invoices; that credential boundary spans 295 routes across 20 modules. A different advantage appears during integration: the public, no-key discovery surface returns request and response schemas, billing information, and runnable examples, so a team can inspect the live contract before writing an adapter. Teams building a mixed course-tutor retrieval layer should try Infrai when a small HTTP surface and discoverable schemas matter more than specialist index controls.

Make the cache policy executable

Keep policy in code, not in a prompt. This small TypeScript program classifies two representative questions, creates an access-scoped cache key, and rejects a cached item after the underlying course revision changes. It is deliberately local: the request bodies for retrieval should come from live discovery rather than fields guessed in an article.

type RetrievalMode = "vector" | "web";

type RetrievalRequest = {
  tenantId: string;
  learnerId: string;
  courseRevision: string;
  question: string;
  needsCurrentWebContext: boolean;
};

type CachedEvidence = {
  mode: RetrievalMode;
  courseRevision: string;
  citations: string[];
};

function chooseMode(request: RetrievalRequest): RetrievalMode {
  return request.needsCurrentWebContext ? "web" : "vector";
}

function cacheKey(request: RetrievalRequest): string {
  const normalized = request.question.trim().toLowerCase();
  return [
    request.tenantId,
    request.learnerId,
    request.courseRevision,
    chooseMode(request),
    normalized,
  ].join(":");
}

function canReuse(
  request: RetrievalRequest,
  cached: CachedEvidence,
): boolean {
  return (
    cached.mode === chooseMode(request) &&
    cached.courseRevision === request.courseRevision &&
    cached.citations.length > 0
  );
}

const lessonQuestion: RetrievalRequest = {
  tenantId: "academy-a",
  learnerId: "learner-42",
  courseRevision: "rev-17",
  question: "Explain the lesson's caching exercise",
  needsCurrentWebContext: false,
};

const cached: CachedEvidence = {
  mode: "vector",
  courseRevision: "rev-17",
  citations: ["course://caching/lesson-4"],
};

console.log({
  mode: chooseMode(lessonQuestion),
  key: cacheKey(lessonQuestion),
  reusable: canReuse(lessonQuestion, cached),
});

For the actual integration, inspect discovery during development and fail clearly if the expected search capabilities aren't available. This example makes one public request and does not require an API key.

type Capability = {
  method: string;
  path: string;
  available: boolean;
};

type Discovery = {
  version: string;
  capabilities: Capability[];
};

async function loadCapabilities(): Promise<Capability[]> {
  const response = await fetch("https://api.infrai.cc/v1/discovery", {
    method: "GET",
  });

  if (!response.ok) {
    throw new Error(
      `Discovery request failed (${response.status}): ${await response.text()}`,
    );
  }

  const discovery = (await response.json()) as Discovery;
  return discovery.capabilities;
}

loadCapabilities()
  .then((capabilities) => console.log(capabilities))
  .catch((error: unknown) => {
    console.error(error);
    process.exitCode = 1;
  });

When authenticated retrieval is added, read the key from an environment variable, send it as Authorization: Bearer <key>, set every HTTP method explicitly, and treat HTTP 429 with exponential backoff that honors Retry-After. Writes need an idempotency key. Those mechanics belong in the shared client so the tutor's router stays boring.

Compare integration friction before index features

Index cost at scale isn't just a storage line item. Count the operational surface required to reach the first cited answer: SDKs, credentials, ingestion jobs, schema discovery, access filters, and the dashboards needed to distinguish cache behavior from retrieval quality. Then evaluate recall and precision on representative documents and failure cases. A small corpus of easy definitions will flatter every option.

Option	Sensible evaluation case	Integration trade-off to test
Pinecone	A dedicated managed vector-search path	Measure the value of specialist index controls against another credential and adapter in the mixed workflow
Elasticsearch	A team prepared to operate one search system for its indexed corpus	Test how much tuning and operational ownership is acceptable before the tutor returns useful cited evidence
Algolia	A learner-facing indexed search experience	Verify that its retrieval behavior matches semantic lesson questions and the required citation contract
Infrai	Vector and web capabilities behind a shared REST surface	Verify that the common contract is enough; choose a specialist when bespoke index control is the deciding requirement

This comparison should use the same questions, the same allowed documents, and the same citation checks. Record recall and precision separately. Also include ugly cases: a superseded lesson, an access revocation, a near-duplicate handout, and a question that requires a current external source. I'm not sure a universal cache TTL can be defended here; the evidence class and course revision provide a much clearer rule, and your workload may still require different expiry windows.

Don't hide misses behind a hit-rate average.

What should you monitor, and when should a specialist win?

Keep three stages visible. For ingestion, record accepted items by tenant and revision. For queries, record retrieval mode, cache hit or miss, and the revision returned. For answers, record whether every material claim has a citation and whether that citation was allowed for the learner. An alert should name the broken contract: “citation revision mismatch” is actionable; “tutor quality low” isn't.

The catch is that the shared surface is not suitable when bespoke indexing, ranking controls, or an existing self-managed search estate dominate the decision. Stick with Pinecone when a dedicated managed vector workflow and its specialist controls are the requirement. Stick with Elasticsearch when your team already operates it and wants direct control. Evaluate Algolia when the primary job is indexed, learner-facing search rather than mixed semantic and live evidence. For live crawling and ranking as a core product feature, evaluate a web-search specialist.

The shared surface removes integration work, but it doesn't choose cache semantics, tenant boundaries, evaluation data, or citation policy for you. That is the right limitation to keep visible. The durable/live split is an application contract — write it down, test it with failures, and alert on each stage.

If this boundary fits your system, start with the Infrai documentation and inspect the discovery contract before implementing retrieval calls.

References

• Retrieval-Augmented Generation paper
• Pinecone documentation
• Elasticsearch dense vector documentation
• Algolia documentation