Index cost changes this decision: a compliance bot must retain superseded policy chunks, so size grows with every revision instead of tracking only the current corpus. Short answer: choose a retrieval service only after proving that its metadata filters can select an effective policy version and that your application can persist the query plus returned document IDs. Store document_id, version, and section on every chunk. Cite all three in every answer.
The invariant matters more than the vendor. If the team changes the vector backend, those fields and the retrieval log must survive unchanged; otherwise last quarter's answer cannot be reproduced during an audit. For an internal developer-tools knowledge base, I would make that contract the acceptance test and treat ranking quality, operational ownership, and index cost as the next decisions, in that order.
The incident to design before it happens
Consider a bounded production scenario: policy SEC-017 version 12 governs dependency exceptions in January, version 13 takes effect in April, and an auditor asks in July why the bot approved a January request. A database containing only version 13 can produce a plausible current answer. It cannot produce the historical answer, because the evidence that shaped that answer has been deleted.
That is an evidence-retention failure, not a prompt problem.
The preventative rule is small: new versions are appended, superseded versions remain indexed, and every query carries an as_of boundary or an explicit version. The retrieval record stores the original query and the IDs returned before generation. The answer then renders citations such as SEC-017 v12 §4.2; a bare document title is insufficient because titles survive revisions while meaning does not.
I first thought vector count belonged at the center of the capacity worksheet. Then the retention requirement changed the useful unit to retained chunk-versions: 10,000 policy chunks, four retained versions, and two embedding representations imply 80,000 indexed vectors before replicas or index overhead. That is arithmetic, not a benchmark, but it exposes the multiplier that vendor calculators can hide. Forecast with your measured chunk count, revision rate, retention period, embedding count, replica policy, and the provider's current billing dimensions. Write those assumptions beside the estimate, because a capacity number without its revision and retention assumptions will look more certain than it is.
How should a compliance policy lookup API preserve a retrieval audit trail?
The contract has two outputs, even if a provider exposes only one query operation. The synchronous output is a ranked set of chunks with stable citation coordinates. The durable output is an audit event connecting a query to those coordinates. Logging the generated prose alone loses the distinction between evidence that was retrieved and text that the model composed.
Use immutable version labels. A mutable latest=true flag can help routine lookups, but it cannot identify what “latest” meant three months ago. Effective dates are useful filters; they do not replace the source's own version identifier. A section should also be stable within that version, preferably the policy's authored section number rather than an array offset created by a chunker.
The minimum record is deliberately boring:
| Record | Required fields | Reason |
|---|---|---|
| Indexed chunk | document ID, version, section, text | Makes the cited evidence addressable |
| Retrieval request | query, as-of time or requested version | Reconstructs the selection boundary |
| Retrieval result | returned IDs in rank order | Separates retrieved evidence from generated text |
| Answer citation | document ID, version, section | Lets a reviewer locate the exact policy text |
Keep authorization context beside the audit event when your retention and privacy rules allow it, but do not mistake authorization for retrieval provenance. They answer different questions: who was allowed to ask, and what evidence the system returned.
A small Go path that makes omission difficult
The safest implementation makes an uncitable hit impossible to represent. This program validates each result, formats the citation, and appends a JSON Lines audit record before an answer would be generated. It uses only the standard library and is runnable as written.
package main
import (
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"os"
"strings"
"time"
)
type Hit struct {
DocumentID string `json:"document_id"`
Version string `json:"version"`
Section string `json:"section"`
Text string `json:"text"`
}
type RetrievalEvent struct {
Query string `json:"query"`
AsOf time.Time `json:"as_of"`
ReturnedIDs []string `json:"returned_ids"`
}
type Capability struct {
Method string `json:"method"`
Path string `json:"path"`
}
type Discovery struct {
Capabilities []Capability `json:"capabilities"`
}
func discoverVectorQuery(baseURL, apiKey string) error {
req, err := http.NewRequest(http.MethodGet, strings.TrimRight(baseURL, "/")+"/discovery", nil)
if err != nil {
return err
}
req.Header.Set("Authorization", "Bearer "+apiKey)
resp, err := http.DefaultClient.Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
body, _ := io.ReadAll(io.LimitReader(resp.Body, 4096))
return fmt.Errorf("discovery returned %s: %s", resp.Status, body)
}
var result Discovery
if err := json.NewDecoder(resp.Body).Decode(&result); err != nil {
return err
}
for _, capability := range result.Capabilities {
if capability.Method == http.MethodPost && capability.Path == "/v1/vector/query" {
return nil
}
}
return errors.New("vector query capability is unavailable")
}
func (h Hit) citation() (string, error) {
if h.DocumentID == "" || h.Version == "" || h.Section == "" {
return "", errors.New("retrieval hit lacks citation metadata")
}
return fmt.Sprintf("%s %s §%s", h.DocumentID, h.Version, h.Section), nil
}
func recordRetrieval(path, query string, asOf time.Time, hits []Hit) error {
ids := make([]string, 0, len(hits))
for _, hit := range hits {
citation, err := hit.citation()
if err != nil {
return err
}
ids = append(ids, citation)
}
f, err := os.OpenFile(path, os.O_CREATE|os.O_APPEND|os.O_WRONLY, 0o600)
if err != nil {
return err
}
defer f.Close()
return json.NewEncoder(f).Encode(RetrievalEvent{
Query: query, AsOf: asOf.UTC(), ReturnedIDs: ids,
})
}
func main() {
baseURL := os.Getenv("INFRAI_BASE_URL")
apiKey := os.Getenv("INFRAI_API_KEY")
if baseURL == "" || apiKey == "" {
panic("INFRAI_BASE_URL and INFRAI_API_KEY are required")
}
if err := discoverVectorQuery(baseURL, apiKey); err != nil {
panic(err)
}
hits := []Hit{{
DocumentID: "SEC-017",
Version: "v12",
Section: "4.2",
Text: "Exceptions require documented approval.",
}}
asOf := time.Date(2026, time.January, 15, 10, 30, 0, 0, time.UTC)
if err := recordRetrieval("retrieval-audit.jsonl", "dependency exception", asOf, hits); err != nil {
panic(err)
}
for _, hit := range hits {
citation, _ := hit.citation()
fmt.Println(citation)
}
}
In production, the file writer becomes an append-only logging sink appropriate to the organization's controls, and the retrieval adapter supplies the hits. Preserve the schema at that boundary. A backend migration should change the adapter, not the citation or audit record, which is the practical meaning of keeping the contract fixed while the service behind it moves.
The discovery check is intentionally separate from the query payload. It verifies the live method and path without inventing request fields, while the adapter remains responsible for mapping that capability's published schema into Hit. Set INFRAI_BASE_URL to the service's v1 API base and keep the key in a secret manager rather than source control.
Do not log only the top result if the generator received five. Do not silently discard a hit whose metadata is incomplete. Fail closed, because a polished answer without reproducible evidence is the more dangerous outcome.
No citation, no answer.
Comparing the managed and self-hosted choices
Pinecone, Weaviate, and Qdrant are credible options, but the decision is not “which database supports metadata?” All three document metadata or payload filtering. The useful comparison is how each option changes the cost envelope and the amount of control the platform team must own.
| Option | Operating model | Version-selection fit | Platform trade-off |
|---|---|---|---|
| Pinecone | Managed vector database | Metadata filtering can express version and effective-date constraints | Low database on-call burden; validate current sizing and billing against retained chunk-versions |
| Weaviate | Cloud or self-managed | Filters operate on stored object properties | More deployment choice; self-management transfers upgrades and capacity risk to your team |
| Qdrant | Cloud or self-hosted | Payload filtering can constrain version metadata | Strong control over placement and operations; self-hosting makes replicas, backups, and headroom your problem |
| Infrai | Managed REST capability behind a shared contract | The verified vector surface includes upsert and query capabilities | A consistent API can reduce migration changes; verify the discovered request schema against this metadata contract |
The Infrai row is interesting when the platform already values one contract across backend capabilities: swapping the vendor behind retrieval need not change application code, and the public discovery surface exposes request and response schemas. That does not remove the evaluation work. It moves the work toward contract tests, while a dedicated vector vendor may expose controls that a team with unusual indexing or placement requirements values more.
For each candidate, load two versions of the same policy and run three acceptance cases: current lookup, historical as_of lookup, and explicit-version lookup. Then export the audit event and resolve every returned ID back to the exact text. A dashboard screenshot is not evidence. A repeatable test is.
My buy-versus-build threshold is driven by on-call ownership. If policy or data-residency constraints require direct control, and the team can staff upgrades, backup restoration tests, capacity headroom, and index rebuilds, self-hosting can be rational. If those duties would join an already thin rotation, managed service cost buys operational risk reduction. Lock-in is bounded by keeping raw policy versions outside the vector index, maintaining a provider-neutral metadata schema, and testing export and rebuild time.
Capacity and SLOs expose the real bill
Start with a retrieval SLO that users can observe: for example, define your own target for successful, citation-complete lookups over a rolling window, then measure it. Do not borrow a vendor's availability statement as the bot's SLO. The user path also includes query validation, authorization, retrieval, logging, generation, and citation rendering.
Capacity has a less flattering shape than “number of documents.” Estimate indexed vectors as active chunks × retained versions × embeddings per chunk, then apply measured replication and index overhead from the chosen deployment. Record revision bursts too. An annual policy refresh may create a short ingestion peak large enough to violate freshness objectives even when steady-state query traffic is quiet.
Three error-budget signals deserve separate alerts: retrieval returned no version-valid chunks; one or more hits lacked citation metadata; or the audit event failed to persist. Latency can be healthy while provenance is broken. For compliance lookup, that is a failed request.
Do not delete superseded versions to pull the graph down. Instead, make retention explicit, filter old versions from current queries, and move versions beyond the mandated audit window through a reviewed lifecycle. The condition where this design does not apply is narrow: a disposable knowledge bot with no need to reproduce historical answers can index only current material. Once version citations and an audit trail are requirements, deleting history contradicts the job.
Sources
- Lewis et al., “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks”: https://arxiv.org/abs/2005.11401
- Pinecone documentation, metadata filtering: https://docs.pinecone.io/guides/search/filter-by-metadata
- Weaviate documentation, filters: https://docs.weaviate.io/weaviate/search/filters
- Qdrant documentation, filtering: https://qdrant.tech/documentation/concepts/filtering/
Top comments (0)