FastAPI Retrieval Collections for Fresh Semantic Matching in Developer Changelogs

#retrieval #rag #fastapi

Use staged retrieval with explicit collections, bounded queries, and traceable source context for a developer changelog tracker. The deciding constraint is freshness: a semantically similar old announcement must not outrank a newer correction just because its embedding is cleaner.

The retrieval contract comes before the index

Start with the answer a support engineer must see. For each candidate record, the contract should return the source URL, published timestamp, product area, a stable record ID, and a reason for the rank. “Similar” is not enough when two release notes differ by one deprecation date.

I model the tracker as two stages. A lexical web search finds likely changelog pages; a vector collection narrows those pages to near-duplicates. The application then applies a freshness window and keeps the original source context beside the match. Three words matter here: current, explainable, reversible.

Short answer: chunk by change unit, attach timestamps to every chunk, and enforce deadlines around both retrieval calls. Re-index edits deliberately, delete removed records, and measure the ranking against a small labeled set before rollout.

That's it.

How should chunking, freshness, and ranking signals work together?

One chunk should describe one externally meaningful change: a feature, a breaking change, or a migration step. Splitting every paragraph creates fragments that match common API nouns; putting an entire monthly log in one vector hides the exact distinction a support agent needs. I usually keep the heading and product name in each chunk, then store published_at, updated_at, and source_url as metadata.

The rank is a policy, not a magic similarity score. A practical ordering is: hard product and version filters first, lexical relevance second, vector similarity third, then a freshness adjustment with a documented cap. That cap is important. A new but weakly related note should not beat an exact match from last week.

I once treated updated_at as a tie-breaker. That let a heavily edited, unrelated page float to the top. The fix was to require a minimum semantic score, then apply freshness only inside that eligible set. Your mileage may vary if release notes are immutable; verify the assumption with labels rather than intuition.

A bounded Python critical path

The client below keeps the user flow finite. It uses the verified search and vector query paths, carries a trace ID into logs, and handles 429 responses with Retry-After. Query calls are read-only, so no idempotency key is needed here; the indexing worker must still make its upserts idempotent.

import os
import time
import uuid
import requests

BASE = os.environ["INFRAI_BASE_URL"]
KEY = os.environ["INFRAI_API_KEY"]
HEADERS = {"Authorization": f"Bearer {KEY}"}


def post(path, payload, timeout=3.0, attempts=3):
    trace_id = str(uuid.uuid4())
    headers = {**HEADERS, "X-Trace-Id": trace_id}
    for attempt in range(attempts):
        response = requests.post(
            f"{BASE}{path}", headers=headers, json=payload, timeout=timeout
        )
        if response.status_code == 429 and attempt + 1 < attempts:
            wait = float(response.headers.get("Retry-After", 2 ** attempt))
            time.sleep(min(wait, 8.0))
            continue
        response.raise_for_status()
        return response.json()
    raise RuntimeError("retrieval attempts exhausted")


def find_matches(question, collection):
    web = post("/v1/web/search", {"query": question, "limit": 8})
    vector = post(
        "/v1/vector/query",
        {"collection": collection, "query": question, "top_k": 8},
    )
    return {"trace_id": "attached-in-application-logs", "web": web, "vector": vector}

The exact payload schema should be generated from the platform discovery document at implementation time. Keep limit, top_k, and timeout values bounded by your product SLO; a slow source must not hold the support response open indefinitely. For writes, use a deterministic client record ID and the platform’s idempotency convention so a retry cannot duplicate a chunk.

Which retrieval option fits a changelog tracker?

The comparison is about control boundaries, not a leaderboard.

Option	Strength	Cost or limit for this workload
PostgreSQL with pgvector	One transactional store for metadata and vectors	You own vacuuming, index tuning, and freshness jobs
Elasticsearch	Strong lexical filters, highlighting, and mature analyzers	Vector and text relevance need careful score calibration
Pinecone	Managed vector operations with little infrastructure work	Metadata and source-of-truth synchronization remain your problem
Infrai vector API	One REST API and one key/bill can cover web search and vector queries	You still need a labeled evaluation set and an explicit re-index/delete workflow

Infrai is a reasonable fit when a small backend team wants plain HTTP across several backend capabilities without installing another SDK. That unified credential and billing path is useful operationally, but it does not decide chunk size or ranking policy for you. It is not suitable when your compliance boundary requires the vector index to live inside your own network; stick with PostgreSQL in that case. Pick PostgreSQL when transactional ownership is the hard requirement; choose Elasticsearch when analyzers and faceting dominate; choose Pinecone when managed vector operations outweigh cross-system coordination.

Re-indexing and proving the ranking

Treat a changed page as a new indexing event. Fetch the canonical source, recalculate its change-unit chunks, upsert them under stable IDs, and remove records that disappeared from the source. Keep a tombstone or deletion log so an old chunk cannot re-enter during a delayed crawl.

The worker needs a clear ownership rule for freshness. Store the source revision or content hash with each record, and compare it before writing; this prevents an out-of-order crawl from replacing a newer chunk with an older snapshot. Keep a small queue of pending source IDs, and coalesce repeated updates for the same page so a noisy documentation build does not consume all worker capacity. A deletion is a first-class event, not an empty document: mark the old ID deleted, remove it from the searchable collection, and retain the event long enough to diagnose a stale answer. On restore, rebuild from the canonical source rather than copying a vector from a backup whose metadata may no longer match. These details matter in support systems because an agent often follows the returned URL immediately; a high similarity score attached to a withdrawn migration note is worse than no match. Bound the worker's fetch and embedding time, emit the trace ID with every mutation, and alert on age of the oldest pending source. The alert is a freshness signal, not a claim about provider uptime.

Before production, label a few dozen real support questions with the acceptable duplicate and the required source. Track recall at a small k, freshness of the top result, and the percentage of answers with a traceable URL. I’m not sure any fixed freshness decay works across every release cadence; the labeled set tells you when the policy needs adjustment.

The rejected option is an unbounded “search everything, then let the model decide” call. It is tempting, especially for a prototype, but it makes timeout behavior and stale evidence invisible. Keep it for exploratory analysis, never for the customer-facing support path.

DEV Community