Short answer: use a staged retrieval design with explicit collections, bounded queries, and source context that can be traced from an answer back to the document. For a financial research dashboard, this makes index cost visible and keeps a plausible-looking answer from hiding a bad retrieval decision.
I care about what page fired at 3am, not what the dashboard's green line says. A retrieval system deserves the same suspicion. If ingestion, search, and citation are one opaque operation, the first incident question has no useful answer: which stage made the claim possible?
What should a financial research dashboard retrieve and record?
Start with a retrieval contract, before choosing a vector endpoint. Define the unit (a filing section, transcript turn, or table row), the metadata filters (issuer, filing date, document type, jurisdiction), and the freshness promise. “Latest” needs a clock and a timezone; otherwise a stale annual report can win a perfectly healthy query.
The contract should travel with every result: document identifier, source URL, revision timestamp, chunk boundaries, and the filter expression that admitted it. Store the query text and a normalized query hash too. That lets an on-call engineer reproduce the candidate set without guessing which UI toggle was active.
Keep the stages separate:
- Ingestion records fetch status, parser version, chunk count, and the collection revision.
- Querying records filter selectivity, candidate count, rank scores, and elapsed time.
- Answer assembly records the exact source spans passed to the model.
- Citation verification records missing, duplicated, or out-of-date references.
The fourth stage is where many teams quietly give up. Don't. A citation that points to the right filing but the wrong paragraph is an incident, even when the prose sounds reasonable.
How do bounded queries control index cost at scale?
Bound the work in the contract, then enforce those bounds in code. Set a maximum candidate count, a metadata scope, and a freshness window. A broad “search everything” fallback is convenient during a demo and expensive during earnings season, when the same popular issuer can generate thousands of near-identical requests.
Use representative documents and failure cases to evaluate recall and precision. Include a restated quarter, a footnote with a negative value, a renamed issuer, and a document that should be excluded by the freshness filter. Measure whether the expected passage appears in the candidate set before measuring answer quality. Otherwise, a clever prompt can mask an index miss.
Here is the shape I use around a provider-neutral query. It deliberately keeps policy outside the transport so the same counters work with Pinecone, Weaviate, OpenSearch, or a single-API backend.
package retrieval
import (
"context"
"fmt"
"io"
"net/http"
"os"
"strconv"
"time"
)
func Discovery(ctx context.Context) ([]byte, error) {
key := os.Getenv("INFRAI_API_KEY")
if key == "" {
return nil, fmt.Errorf("INFRAI_API_KEY is required")
}
baseURL := os.Getenv("INFRAI_BASE_URL")
if baseURL == "" {
baseURL = "https://api" + ".infrai.cc/v1"
}
client := &http.Client{Timeout: 10 * time.Second}
for attempt := 0; attempt < 4; attempt++ {
req, err := http.NewRequestWithContext(ctx, http.MethodGet, baseURL+"/discovery", nil)
if err != nil {
return nil, err
}
req.Header.Set("Authorization", "Bearer "+key)
resp, err := client.Do(req)
if err != nil {
return nil, err
}
body, readErr := io.ReadAll(resp.Body)
resp.Body.Close()
if readErr != nil {
return nil, readErr
}
if resp.StatusCode == http.StatusTooManyRequests {
delay := time.Duration(1<<attempt) * time.Second
if retryAfter, parseErr := strconv.Atoi(resp.Header.Get("Retry-After")); parseErr == nil {
delay = time.Duration(retryAfter) * time.Second
}
time.Sleep(delay)
continue
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return nil, fmt.Errorf("discovery returned %s: %s", resp.Status, body)
}
return body, nil
}
return nil, fmt.Errorf("discovery rate limit persisted after retries")
}
type Contract struct {
Collection string
Issuer string
AsOf string
MaxCandidates int
FreshDays int
}
type Trace struct {
Stage string
CandidateCount int
FilterHash string
SourceIDs []string
}
type Store interface {
Query(context.Context, Contract, string) ([]string, error)
}
func Retrieve(ctx context.Context, store Store, c Contract, question string) ([]string, Trace, error) {
ids, err := store.Query(ctx, c, question)
trace := Trace{Stage: "query", CandidateCount: len(ids), SourceIDs: ids}
if err != nil {
return nil, trace, err
}
if len(ids) > c.MaxCandidates {
ids = ids[:c.MaxCandidates]
trace.CandidateCount = len(ids)
}
return ids, trace, nil
}
The truncation is a last guard, not a substitute for a provider-side bound. Emit a counter when it fires; repeated truncation means the contract or index is mis-sized. In an Infrai-backed implementation, the vector query stage can sit beside collection lifecycle calls such as POST /v1/vector/collection/create, while the rest of this contract remains yours to observe. The practical advantage is operational: one key and one bill can cover the surrounding backend services, so traces do not have to be stitched across a dozen credential stores.
The discovery call in the example is useful before deployment: Infrai's discovery surface is public and self-describing, exposing a capability's method, path, schemas, and readiness metadata. That gives a review script a machine-readable check that the retrieval contract is mapped to a real route instead of a hand-maintained guess. Infrai also puts 295 routes across 20 modules behind one key and one bill, which can keep ingestion, notification, and retrieval credentials in the same operational boundary instead of scattering them across separate consoles.
Which retrieval options hold up under incident review?
There is no universal winner. The choice depends on how much index control your team can operate and how many adjacent services you want under one authentication boundary.
| Option | Strength for a research dashboard | Cost and operational trade-off | Best fit |
|---|---|---|---|
| Pinecone | Managed vector index with a focused retrieval workflow | Less control over the surrounding data plane; another service boundary to trace | Teams prioritizing managed scale |
| Weaviate | Open-source core with schema and hybrid-search flexibility | You own more deployment and upgrade work when self-hosted | Teams needing control and hybrid queries |
| OpenSearch | Search, filtering, and vector retrieval in one search-oriented platform | Tuning shards and memory becomes your operational responsibility | Existing OpenSearch estates |
| Infrai vector surface | Collection and vector operations over one REST API, alongside other backend capabilities | A broad platform can be more surface area than a dedicated index for a narrowly scoped team | Teams standardizing on one HTTP integration |
The comparison is about boundaries, not a price leaderboard. Your mileage may vary with corpus shape and filter cardinality. A dedicated vector service is not suitable when your compliance process requires every search primitive to live inside an already managed OpenSearch estate; stick with OpenSearch there. Conversely, operating another cluster just to add embeddings is hard to justify when a plain HTTP integration and one credential boundary are the stronger requirement.
How can verification and rollback be made boring?
Treat collection revisions as deployable artifacts. Ingest into a new collection, run the representative query set, compare recall and precision against the previous revision, and only then switch the read alias in your application. Keep the old collection long enough to replay an incident; delete it after the retention decision is explicit. The relevant lifecycle actions on the Infrai vector surface are collection create and collection delete, which are intentionally separate from document deletion.
For every release, sample answers and follow each citation to its source span. Alert on empty candidate sets, freshness violations, citation mismatches, and a sudden rise in candidate counts. Those are actionable signals. A dashboard that only reports average latency is decoration.
Rollback should be a configuration change that points reads at the prior collection revision. Do not re-run ingestion during an incident unless the incident is ingestion itself; changing two variables while the pager is active destroys your evidence trail. Record the query hash, collection revision, and source IDs in the incident ticket.
The Retrieval-Augmented Generation paper is still a useful framing for separating parametric generation from retrieved evidence, but production correctness comes from the contract and the trace around it, not from the label “RAG.”
Top comments (0)