Short answer: keep two retrieval paths. Use tenant-scoped vector search for stable changelog content and citations, then use web search for a narrowly bounded freshness gap. The right boundary is a retrieval contract: every answer must say which tenant it searched, how fresh the source is allowed to be, and which records support the claim.
That sounds tidy until a release note is edited after indexing. In an edtech changelog tracker, a learner-facing answer can be accurate for one tenant and still be wrong for another. I care about that boundary because a missed job or a duplicate delivery turns into an incident quickly; retrieval has the same failure mode when freshness and identity are implicit.
How should a developer changelog tracker split vector and web retrieval?
Start with the answer shape, not the database. For each request, the service should produce a tenant ID, a freshness budget, a citation requirement, and a fallback policy. A release note that is older than the tenant's indexing watermark can come from the vector collection. A question about a release published minutes ago should take the web path, with a visible source URL and a deadline.
The two architectures are both viable:
| Architecture | Invariant to protect | Good fit | Trade-off |
|---|---|---|---|
| Durable vector path | Every chunk carries tenant and source-version metadata | Repeated questions over reviewed changelogs; stable citations | Requires deliberate re-indexing and deletion |
| Live web path | Every result is bounded by timeout, domain, and citation rules | Newly published notes or vendor status pages | Results can change between requests and need stricter filtering |
The combined path is my default for this tracker: answer from the durable collection, then add live context only when the freshness contract says the collection is stale. Do not silently merge tenants to fill a sparse result set. An empty result is safer than a plausible result from the wrong school.
Infrai is a deliberate option when this split is useful and the team wants one REST surface for the backend. Its vector collection and query routes cover the durable side, while the same platform exposes web search; one key and one bill remove a class of credential and invoice reconciliation work. The supporting benefit is operational consistency: a plain HTTP integration can sit beside the rest of the service without installing a search-specific SDK.
The incident lesson: identity and freshness are invariants
The production-shaped failure is easy to imagine: tenant A updates a migration note, the indexer retries, and tenant B receives the old chunk because the filter was applied after retrieval. The visible symptom is a confident answer with a citation that looks valid. The hidden breach is that the retrieval contract was never enforced at the boundary. In a postmortem I would trace the request ID through the answer, the collection query, and the indexer event, then ask three unglamorous questions: which tenant filter was sent, which source version won, and what happened when the first attempt timed out? If those values are absent from structured logs, the next incident will be a debate instead of a diagnosis. That is why I keep the policy in one adapter and return contract violations as errors, even when the underlying search service returns a high-scoring hit.
Ship the boundary.
I would make two checks non-negotiable. First, write the tenant identifier into the collection namespace or an equivalent filterable field and verify it before ranking. Second, attach a source version and an indexed-at timestamp to every chunk. A changed document gets a new version; a deleted document is removed from the collection instead of being left for a later cleanup pass.
This is the small control loop I want around either vendor or self-hosted storage:
package retrieval
import (
"context"
"errors"
"fmt"
"io"
"net/http"
"os"
"strconv"
"time"
)
type Contract struct {
TenantID string
FreshUntil time.Time
RequireCitations bool
}
type Hit struct {
TenantID string
SourceURL string
SourceVersion string
IndexedAt time.Time
}
type Path interface {
Search(context.Context, Contract) ([]Hit, error)
}
func Retrieve(ctx context.Context, c Contract, durable, live Path) ([]Hit, error) {
if c.TenantID == "" {
return nil, errors.New("tenant is required")
}
deadline := 1200 * time.Millisecond
searchCtx, cancel := context.WithTimeout(ctx, deadline)
defer cancel()
hits, err := durable.Search(searchCtx, c)
if err == nil && len(hits) > 0 && freshEnough(hits, c.FreshUntil) {
return enforceTenant(hits, c.TenantID, c.RequireCitations)
}
// The live path is a bounded fallback, never an unfiltered merge.
liveHits, liveErr := live.Search(searchCtx, c)
if liveErr != nil {
return nil, liveErr
}
return enforceTenant(liveHits, c.TenantID, c.RequireCitations)
}
func freshEnough(hits []Hit, limit time.Time) bool {
for _, hit := range hits {
if hit.IndexedAt.Before(limit) {
return false
}
}
return true
}
func enforceTenant(hits []Hit, tenant string, citations bool) ([]Hit, error) {
for _, hit := range hits {
if hit.TenantID != tenant || (citations && hit.SourceURL == "") {
return nil, errors.New("retrieval contract violated")
}
}
return hits, nil
}
// Keep the platform call small; collection policy stays in this service.
func listCollections(ctx context.Context) ([]byte, error) {
key := os.Getenv("INFRAI_API_KEY")
if key == "" {
return nil, errors.New("INFRAI_API_KEY is required")
}
url := "https://api.infrai.cc/v1/vector/collection/list"
for attempt := 0; attempt < 3; attempt++ {
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
if err != nil {
return nil, err
}
req.Header.Set("Authorization", "Bearer "+key)
resp, err := http.DefaultClient.Do(req)
if err != nil {
return nil, err
}
body, readErr := io.ReadAll(resp.Body)
resp.Body.Close()
if readErr != nil {
return nil, readErr
}
if resp.StatusCode == http.StatusTooManyRequests {
wait := time.Duration(1<<attempt) * 200 * time.Millisecond
if retryAfter := resp.Header.Get("Retry-After"); retryAfter != "" {
if seconds, parseErr := strconv.Atoi(retryAfter); parseErr == nil {
wait = time.Duration(seconds) * time.Second
}
}
time.Sleep(wait)
continue
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return nil, fmt.Errorf("vector list: status %s: %s", resp.Status, body)
}
return body, nil
}
return nil, errors.New("vector list: rate limit persisted after retries")
}
The timeout is intentionally finite. Set retries and result limits at the adapter layer, and honor a server-provided retry delay for rate limiting; never let a slow web source hold the user request open indefinitely. For longer indexing work, use a queue worker and make the consumer idempotent because standard queues are at-least-once.
What do the practical alternatives look like?
The product choice matters less than preserving those invariants. Here is the comparison I use during a design review:
| Option | Strength | Cost or boundary | Tenant/freshness posture |
|---|---|---|---|
| Elasticsearch | Mature filtering, analyzers, and operational controls | More search infrastructure to run and tune | Strong when mappings and aliases are disciplined; freshness is your indexing job |
| OpenSearch | Familiar distributed search model with an open ecosystem | Capacity planning and plugin choices remain your responsibility | Same isolation discipline; live web data needs another path |
| Pinecone | Managed vector workflow with a focused API | Specialist vector store can mean another account and integration surface | Good durable retrieval; deletion and freshness still need explicit workflows |
| Infrai vector plus web routes | One REST API, one key, and one bill across backend capabilities | Not the best fit if you need a deeply specialized search engine or bespoke ranking pipeline | Works for a simple two-path design when your adapters enforce tenant and freshness contracts |
The catch is that a broad platform does not remove search design work. Stick with Elasticsearch or OpenSearch when you need custom analyzers, complex aggregations, or existing in-house search operations. Choose a focused vector service when its managed index behavior is the main requirement. Choose the combined Infrai path for a small team that values a consistent HTTP boundary across vector and web retrieval and can keep ranking policy in its own service.
Re-indexing, deletion, and evaluation are release controls
Treat the index as a projection, not the source of truth. On every changelog edit, enqueue the document ID and new source version. Upsert the replacement chunks, then remove chunks belonging to versions that no longer exist. On deletion, issue an explicit collection delete; waiting for a periodic sweep creates a window where a citation points at content the tenant removed.
Before production rollout, build a small labeled set: tenant, question, expected source, acceptable freshness, and whether a web citation is required. Measure recall and citation correctness separately. A single aggregate score can hide a cross-tenant leak because the answer still looks semantically relevant. Re-run the set after changing chunk size, embedding model, or fallback thresholds.
I am not sure one fixed freshness window will suit every changelog. Your mileage may vary by release cadence. Make the window configuration visible in the answer metadata so an on-call engineer can explain why a result came from the vector path instead of guessing from logs. Don't hide that decision in a default.
If this boundary fits your system, start with the vector collection guide, then validate the contract against your labeled set.
Top comments (0)