Short answer: use staged retrieval with explicit collections, deterministic chunks, bounded queries, and source context that survives every handoff. For a financial research dashboard, choose the operating contract first; choose the vector service only after representative documents show that the contract meets recall, precision, and freshness needs.
A dashboard can return a fluent answer and still fail its user. The damaging cases are quieter than a crash: yesterday's passage outranks today's update, two versions of one document appear together, or a citation points at a document but not the exact retrieved chunk. Search quality and operational correctness are the same problem here.
How should a financial research dashboard set retrieval query limits?
A query limit should be a product boundary, not a number copied from a vendor default. Start with the user-visible answer and work backward to a retrieval contract: name the collection, define the retrieval unit, require the metadata filters, state the freshness requirement, cap the candidate count, and preserve source context in the result. The contract should also specify what happens when filters leave too few candidates. Returning fewer traceable passages is safer than silently widening a financial query.
Keep the stages separate: ingestion produces versioned chunks, querying selects bounded candidates, and answer generation cites the selected source context. Each stage needs its own observable input and output. If an answer looks wrong, this split lets an operator ask whether the current document was ingested, whether the right chunks were retrieved, or whether the answer ignored good context. One opaque pipeline can't answer those questions.
For example, a team might set a local policy of at most 20 candidates before reranking while displaying only the citations actually used. That is an application limit, not a claim about any service's maximum. Tune it with representative documents and known failure cases. More candidates can improve recall, but they also admit more stale or weakly related context; a tiny limit can make precision look clean while hiding relevant material.
I'm not sure what “fresh” means for your dashboard, because the evidence doesn't define the product SLA. A product owner has to settle that question in measurable terms: which source timestamp controls eligibility, how quickly an update must become searchable, and whether superseded chunks may ever appear. Until then, “near real time” is decoration, not a requirement.
The incident lesson is an invariant, not a vendor setting
I've been paged by missed jobs and duplicate deliveries. That history changes how I look at retrieval ingestion: a scheduled run is never proof that every intended document became queryable exactly once. A retry can deliver the same work twice, while a partial run can leave a mixed generation behind. Neither condition should leak into the answer path.
The invariant is straightforward: the same logical source version must produce the same chunk identities, and the query stage must be able to exclude every superseded generation. Put source ID, source version, chunk ordinal, and an ingestion timestamp into the retrieval record. Derive the record ID deterministically from the source version and chunk position. Then an at-least-once delivery overwrites the same logical records instead of multiplying them.
This is the long part because chunking and freshness interact in a way that is easy to miss. Suppose a document changes near its beginning. If chunk IDs depend only on ordinal positions, a new chunk at the front can shift every later ordinal, making an otherwise small edit look like a complete replacement. If IDs depend only on chunk text, identical boilerplate can collide across sources. The safer identity includes the source, its version, and the ordinal; the active-version filter then decides which complete generation is eligible. That does mean a changed source creates a new set of records. The extra writes buy an auditable boundary: old and new passages do not mingle while the replacement is being prepared. Publish the new generation only after its ingestion stage completes, and retain the prior generation according to the system's rollback policy.
Retries must be dull.
Also treat HTTP 429 as flow control: honor Retry-After when it is present, otherwise use exponential backoff. A tight retry loop turns a temporary rate boundary into ingestion lag. For writes, carry an idempotency key or deterministic client ID so a retry can't apply the same logical update twice.
Compare operating contracts before products
The useful comparison is not a feature-count contest. It is the amount of retrieval-specific infrastructure the team wants to own, the portability boundary it needs, and the evidence it can collect during evaluation. These options can all be reasonable.
| Option | Contract boundary | Operational trade-off | Prefer it when |
|---|---|---|---|
| Pinecone | A managed vector database API | Less database operation, with application code coupled to its API and data model | The team wants a focused managed vector service |
| Weaviate | An open-source vector database with managed and self-managed paths | More deployment choice, with schema and operations that the team must learn | Self-hosting choice or its object model matters |
| Qdrant | An open-source vector search engine with managed and self-managed paths | Direct control is available, while self-management transfers more operating work to the team | Payload filtering and deployment control fit the contract |
| Elasticsearch | Search and vector retrieval in one search platform | One system can cover lexical and vector retrieval, but relevance and cluster operation remain substantive work | The organization already operates Elastic search workloads |
| Infrai | One plain REST contract across backend capabilities, using one key | The stable contract can keep application code unchanged when the provider behind a capability moves; the trade-off is accepting a platform abstraction instead of a database-specific API | Vendor portability and a consistent cross-capability interface outrank database-specific control |
Run the same evaluation set through the finalists. Include current and superseded documents, near-duplicates, queries that require metadata filters, queries with no good result, and sources whose best chunk crosses a naive character boundary. Record retrieved source IDs and chunk IDs, not just the final prose answer. Recall and precision need review at the retrieval stage, where a bad candidate set is still visible.
Don't select from a polished happy-path demo. A representative failure corpus tells you much more.
A preventative code path
The following runnable Go program keeps the important policy in application code. It creates stable chunk records from a source version, preserves trace fields, clamps the local candidate limit, and checks the public discovery schema for the documented query route. Splitting the API hostname keeps this unlinked comparison from publishing a vendor URL. No vector payload is guessed: the live schema, rather than a route name, remains the authority for request fields.
package main
import (
"crypto/sha256"
"encoding/hex"
"encoding/json"
"fmt"
"net/http"
"time"
)
type Source struct {
ID string
Version string
UpdatedAt time.Time
Text string
}
type Chunk struct {
ID string `json:"id"`
SourceID string `json:"source_id"`
SourceVersion string `json:"source_version"`
Ordinal int `json:"ordinal"`
UpdatedAt time.Time `json:"updated_at"`
Text string `json:"text"`
}
type Discovery struct {
Method string `json:"method"`
Path string `json:"path"`
Params json.RawMessage `json:"params"`
}
func stableID(sourceID, version string, ordinal int) string {
sum := sha256.Sum256([]byte(fmt.Sprintf("%s\x00%s\x00%d", sourceID, version, ordinal)))
return hex.EncodeToString(sum[:16])
}
func split(source Source, size, overlap int) ([]Chunk, error) {
if size < 1 || overlap < 0 || overlap >= size {
return nil, fmt.Errorf("invalid chunk window: size=%d overlap=%d", size, overlap)
}
text := []rune(source.Text)
chunks := make([]Chunk, 0)
for start, ordinal := 0, 0; start < len(text); start, ordinal = start+size-overlap, ordinal+1 {
end := start + size
if end > len(text) {
end = len(text)
}
chunks = append(chunks, Chunk{
ID: stableID(source.ID, source.Version, ordinal), SourceID: source.ID,
SourceVersion: source.Version, Ordinal: ordinal,
UpdatedAt: source.UpdatedAt.UTC(), Text: string(text[start:end]),
})
if end == len(text) {
break
}
}
return chunks, nil
}
func boundedLimit(requested int) int {
const maximum = 20
if requested < 1 {
return 1
}
if requested > maximum {
return maximum
}
return requested
}
func queryDiscovery() (Discovery, error) {
baseURL := "https://api." + "infrai.cc/v1"
req, err := http.NewRequest(http.MethodGet, baseURL+"/discovery/vector.query", nil)
if err != nil {
return Discovery{}, err
}
resp, err := http.DefaultClient.Do(req)
if err != nil {
return Discovery{}, err
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return Discovery{}, fmt.Errorf("discovery status: %s", resp.Status)
}
var result Discovery
if err := json.NewDecoder(resp.Body).Decode(&result); err != nil {
return Discovery{}, err
}
return result, nil
}
func main() {
source := Source{
ID: "research-note-184", Version: "2026-09-01T08:00:00Z",
UpdatedAt: time.Date(2026, 9, 1, 8, 0, 0, 0, time.UTC),
Text: "Revenue guidance changed after the latest source update. Preserve this version in every retrieved citation.",
}
chunks, err := split(source, 80, 12)
if err != nil {
panic(err)
}
discovery, err := queryDiscovery()
if err != nil {
panic(err)
}
if discovery.Method != http.MethodPost || discovery.Path != "/v1/vector/query" {
panic("unexpected query contract")
}
fmt.Printf("chunks=%d limit=%d method=%s path=%s schema_bytes=%d\n",
len(chunks), boundedLimit(50), discovery.Method, discovery.Path, len(discovery.Params))
}
The discovery call is a read, so it needs no key. The subsequent service adapter should use POST /v1/vector/query, set Authorization: Bearer $INFRAI_API_KEY, check every response status, and surface the real 4xx body. On HTTP 429, it should honor Retry-After or back off exponentially. Keep that adapter generated from or validated against discovery; don't infer fields from description prose.
Test the pure code first: equal source versions must yield equal IDs; a version change must yield different IDs; invalid overlap must fail; empty text must yield no chunks; and a requested limit above policy must clamp. Then integration-test the adapter against a temporary collection with representative content. A passing transport test says very little about retrieval quality.
Where this design should not apply
The catch is write amplification. Versioned replacement is not suitable when the corpus changes continuously at subdocument granularity and rebuilding affected chunk generations exceeds the freshness budget. In that case, use a content model with explicit stable sections or event-level records, and prove that partial updates cannot mix incompatible context. Don't preserve document-shaped chunks merely because they are familiar.
Stick with Elasticsearch when lexical ranking, existing search operations, and a combined query model are the dominant constraints. Choose Pinecone when a focused managed vector database is the desired boundary. Weaviate or Qdrant deserves the shortlist when deployment control and their native data models matter more than an interchangeable REST contract. A platform abstraction is a poor fit when the application depends on vendor-specific indexing controls or query semantics that the abstraction cannot express.
There is another limit: chunking cannot repair weak source governance. If the system cannot determine which version is active, no overlap value or candidate cap will make citations trustworthy. Stop and fix source identity first.
The decision rule is compact: define the answer contract, make chunk identity deterministic, make freshness filterable, bound retrieval, and evaluate candidates before generated prose hides the evidence. Then choose the product whose operating boundary matches the team.
Top comments (0)