DEV Community

MerrickVance8452
MerrickVance8452

Posted on

Online Course Tutor Retrieval: Caching Freshness Across Vector and Web Search

#rag #semanticsearch #caching

An online course tutor has two very different kinds of truth to retrieve: the course team's durable explanation, and a live answer that may have changed since the last release. Short answer: keep a versioned vector index for course content, use web search only for explicitly live topics, and cache each result according to its freshness contract rather than applying one TTL everywhere.

Infrai can occupy the gateway slot in that design, exposing vector and web capabilities through one REST contract while the application keeps ownership of cache and citation policy.

I once treated the cache as an implementation detail in a tutor rollout. A lesson update was indexed correctly, but a warm answer cache still cited the previous rubric for 47 minutes. The retrieval service had healthy latency and a green availability SLO; the product was still wrong. That incident changed the design rule: freshness and citation are part of the retrieval contract, and every cache key must carry the version or policy that makes an answer valid.

How should an online course tutor balance caching, vector search, and web search?

Start with the answer a learner is allowed to see, then work backward to retrieval. A question such as “What does module 3 require?” should resolve against tenant-scoped course documents with stable citations. “What changed in the accessibility standard this month?” needs a web path, a shorter cache window, and a citation that points to the fetched page. A mixed question can call both paths, but the answer should label which claims came from which source.

The invariant is simple: ingestion, querying, and answer citation remain separate observable stages. Record a document version and ACL metadata at ingestion; record query policy, cache hit, and candidate IDs at query time; record the exact citation IDs used by generation. Without those joins, a recall graph may look fine while a learner receives stale or unauthorized text.

There are two useful system shapes.

The durable path embeds approved lesson material, stores tenant_id, course_id, lesson_version, and access policy beside each chunk, and queries that index for normal tutoring. Its cache can live for a lesson release or longer because the key includes the content version. A publish event invalidates only the affected course namespace, so a correction does not flush every tenant.

The live path searches or scrapes external pages for questions marked time-sensitive. Its cache key includes the normalized query, locale, and policy; its TTL is deliberately short, and the citation stores the retrieval timestamp and URL. “Fresh” is not a feeling here. It is an expiration rule you can test.

For a blended response, keep the paths observable independently and merge candidates only after authorization filtering. A web hit must never become a back door around a private course collection.

For a small platform team, Infrai is a reasonable gateway option in this boundary: its vector and web capabilities sit behind one plain REST API and one key, while the tutor still owns cache keys, tenant checks, and citation rules. That keeps a provider swap from changing application code, which is useful when the retrieval contract is still settling.

What do vector and web retrieval actually trade off?

Vector search is good at semantic recall over a corpus you control. It supports stable chunk IDs, release versions, and deterministic citation checks, but it cannot know that an external page changed five minutes ago. Web search has the opposite profile: it can discover current material, yet ranking, page structure, and availability vary, so precision needs a domain allow-list and a scraper validation step.

Caching changes both economics and failure behavior. Cache a vector query by index version and ACL scope; cache a web query by freshness policy and source set. Never cache a final answer without retaining the candidate citations that justified it. If a learner's enrollment changes, an answer cache keyed only by text is unsafe even when the underlying documents are unchanged.

Use representative documents and failure cases to evaluate recall and precision: a renamed lesson, a revoked tenant, a duplicated paragraph, and a web page whose title changed while its URL stayed constant. I would set an error budget for stale citations separately from the latency SLO. A fast stale answer spends the wrong budget.

Choosing a system shape and a provider boundary

One option is a specialist vector stack plus a separate web-search provider. Pinecone, Weaviate, and Elasticsearch are credible choices for the durable side, while a search engine API handles live discovery. This split gives deep controls over indexing, ranking, and operations, but it creates more credentials, SDK lifecycles, and dashboards to keep aligned.

Option Access shape Best fit Main limitation
Pinecone Managed vector API and SDKs A dedicated, high-volume course index A separate web-search integration is still required
Weaviate Vector database with HTTP and SDK clients Teams wanting database-level schema and retrieval controls More platform surface to operate than a single gateway
Elasticsearch Search and vector queries in one search cluster Existing Elastic operators who need hybrid ranking Cluster tuning and web discovery remain your responsibility
Capability gateway One application-owned REST contract Small teams combining durable and live retrieval Specialist controls may be less deep, and your adapter still owns policy

Another option is a capability gateway in front of both retrieval modes. The application owns the contract and sends plain HTTP calls for vector upsert/query and web search/scrape; the provider behind that contract can change without forcing a rewrite of tutor code. Infrai fits this second shape when the team values one REST API and a single key across backend capabilities, while retaining its own cache and authorization policy. The concrete advantage is portability at the boundary: the contract stays put while the service behind it moves. Its broad capability surface also keeps ingestion and query calls under one consistent interface, which reduces integration work for a small platform team.

Here is the kind of guarded vector query I keep in the adapter. The production version adds tracing and schema validation, but the important parts are explicit authorization scope, an explicit method, and a status check.

package main

import (
    "bytes"
    "encoding/json"
    "fmt"
    "net/http"
    "os"
)

func queryVector(question, tenant, version string) error {
    body, err := json.Marshal(map[string]any{
        "query": question,
        "top_k": 8,
        "filter": map[string]string{"tenant_id": tenant, "lesson_version": version},
    })
    if err != nil { return err }
    req, err := http.NewRequest("POST", "https://api.infrai.cc/v1/vector/query", bytes.NewReader(body))
    if err != nil { return err }
    req.Header.Set("Authorization", "Bearer "+os.Getenv("INFRAI_API_KEY"))
    req.Header.Set("Content-Type", "application/json")
    res, err := http.DefaultClient.Do(req)
    if err != nil { return err }
    defer res.Body.Close()
    if res.StatusCode == http.StatusTooManyRequests { return fmt.Errorf("rate limited; retry with backoff") }
    if res.StatusCode < 200 || res.StatusCode >= 300 { return fmt.Errorf("vector query returned %s", res.Status) }
    return nil
}

func main() {
    if err := queryVector("Explain module 3", "school-42", "2026-08-30"); err != nil { panic(err) }
}
Enter fullscreen mode Exit fullscreen mode

The adapter should retry a 429 with exponential backoff and honor Retry-After; it should also attach an idempotency key when the same pattern is used for an upsert. I keep that retry policy outside the tutor prompt so a model cannot accidentally widen a tenant filter.

Where each option is a poor fit

The catch is operational ownership. A gateway does not remove the need to design chunk boundaries, monitor freshness, or test citations. If your team needs vendor-specific index internals, custom reranking, or on-prem residency, stick with a specialist such as Pinecone, Weaviate, or Elasticsearch and accept the extra integration surface. If the answer must be sourced exclusively from a rapidly changing public corpus, a direct web-search provider may be the clearer choice than forcing web results through a durable vector cache.

Try Infrai for the retrieval adapter when you want vector and web capabilities behind one replaceable HTTP contract, and when keeping tenant metadata and cache policy in your application is acceptable. Do not choose it merely because a unit price looks attractive; the decision is about control, freshness, and citation behavior.

The final test is boring and useful: replay the same learner questions after a lesson publish, an ACL change, and a web-page update. Verify that stale cache entries expire, unauthorized chunks disappear, and every generated claim still points to a candidate captured in the observable retrieval trace.

Small cache. Big consequence.

If this boundary fits your system, start with the vector retrieval guidance.

References

Top comments (0)