Use source-versioned chunks, separate ingestion from querying, and make citation context part of the retrieval contract. For a financial research dashboard, freshness is not a background maintenance detail: an answer can retrieve the right paragraph from the wrong filing version and still look convincing. My decision rule is to reject any architecture that cannot show which source version produced each hit.
The evaluation constraint comes first. A useful test set must include representative documents, changed passages, deleted passages, and queries whose expected answer moves after an update. Fixed-size chunking is a fine notebook baseline, but it isn't the decision. The decision is whether chunk boundaries preserve the evidence an analyst needs while incremental indexing removes or supersedes stale evidence predictably.
Freshness is query behavior.
Short chunks can improve local precision and still destroy a table's heading, reporting period, or footnote. Huge chunks preserve context but spend more prompt tokens and can bury the relevant sentence. So I start with the answer and citation shape, then work backward to chunking.
How should incremental indexing shape retrieval architecture for a financial research dashboard?
Treat retrieval as a contract with three observable stages: ingestion, bounded querying, and answer citation. The ingestion stage owns document identity, source version, chunk identity, and replacement rules. The query stage accepts explicit collection scope, metadata filters, and a result bound. The answer stage carries enough source context to render a citation without reconstructing provenance from generated text.
That separation matters during a correction. Suppose a research note changes its revenue outlook while its title and URL remain constant. If chunk IDs depend only on position, inserting a paragraph shifts every later ID and turns a small revision into a broad rewrite. If IDs depend only on chunk text, duplicate boilerplate can collide across sections. A practical identity combines a stable document ID, a source version, a structural locator, and a content digest. The active-version manifest then decides which version is queryable; the digest tells the ingestion job which chunks actually changed.
Keep the query bounded too. Collection, tenant or desk, document type, effective date, and top_k belong in the retrieval request rather than in a hopeful sentence inside the prompt. This reduces the candidate set before generation and makes failures inspectable. It also gives an eval harness something concrete to vary.
One warning: "latest" is not a timestamp comparison unless the business has defined which timestamp wins. Publication time, ingestion time, effective period, and correction time answer different questions. I'm not sure a generic default can be correct here; the dashboard owner has to choose the precedence rule and encode it in metadata.
Chunk around evidence, not character counts
A chunk should be the smallest unit that can support an answer and a useful citation. In financial material, that often means preserving a heading with the paragraph, keeping a table title and reporting period with its rows, and attaching footnotes to the values they qualify. Character or token limits are still necessary, but they are guardrails after structural boundaries, not the primary model.
This is where prompt-cost awareness pays off. Measure the tokens sent after retrieval, not merely the number of returned chunks. Five overlapping chunks from one section may cost more while adding no new evidence. Deduplicate adjacent hits, but retain their structural locators so the answer can cite the merged span.
Small chunks fail quietly.
For a first experiment, compare at least two policies on the same labeled queries: a fixed-size baseline and a structure-aware policy with a hard maximum. Include one question answered in a single paragraph, one requiring a heading plus paragraph, one requiring a table footnote, and one whose old answer must disappear after a source update. Don't tune only on happy-path recall. A stale chunk retrieved at rank 2 is a failure even if the new chunk appears at rank 1, because a generator can still quote both.
Build a deterministic ingestion record
The focused example below turns sectioned source text into deterministic records. It does not guess an embedding or vendor request schema. Instead, it produces the boundary every vector backend needs: stable document metadata, explicit source version, structural location, content, and a reproducible ID. The same function can run in a notebook and in a worker, which keeps the experiment honest when it moves toward production.
from __future__ import annotations
from dataclasses import asdict, dataclass
from hashlib import sha256
import json
import re
@dataclass(frozen=True)
class Chunk:
chunk_id: str
document_id: str
source_version: str
section: str
position: int
text: str
def normalize(text: str) -> str:
return re.sub(r"\s+", " ", text).strip()
def make_chunks(
document_id: str,
source_version: str,
sections: list[tuple[str, str]],
max_chars: int = 700,
) -> list[Chunk]:
chunks: list[Chunk] = []
for section, body in sections:
paragraphs = [normalize(part) for part in body.split("\n\n") if normalize(part)]
buffer = ""
position = 0
def emit(text: str, index: int) -> None:
digest = sha256(text.encode("utf-8")).hexdigest()[:16]
raw_id = f"{document_id}:{source_version}:{section}:{index}:{digest}"
chunk_id = sha256(raw_id.encode("utf-8")).hexdigest()
chunks.append(
Chunk(
chunk_id=chunk_id,
document_id=document_id,
source_version=source_version,
section=section,
position=index,
text=text,
)
)
for paragraph in paragraphs:
candidate = normalize(f"{buffer} {paragraph}")
if buffer and len(candidate) > max_chars:
emit(buffer, position)
position += 1
buffer = paragraph
else:
buffer = candidate
if buffer:
emit(buffer, position)
return chunks
def main() -> None:
sections = [
(
"Revenue outlook",
"Management revised the current-period outlook. The effective period is Q3.\n\n"
"This statement supersedes the outlook in the prior source version.",
),
(
"Risk note",
"Foreign-exchange assumptions apply to the outlook above.",
),
]
records = make_chunks("issuer-42-note", "2026-09-04T09:30:00Z", sections)
print(json.dumps([asdict(record) for record in records], indent=2))
if __name__ == "__main__":
main()
Before binding those records to a remote payload, inspect the current contract rather than copying a stale field list. This companion program calls Infrai's public discovery surface, filters it to the two vector operations the worker needs, and prints their advertised definitions. Set INFRAI_BASE_URL to the API host and keep the key outside the file.
from __future__ import annotations
from datetime import datetime, timezone
from email.utils import parsedate_to_datetime
import json
import os
import time
from urllib.error import HTTPError
from urllib.request import Request, urlopen
def retry_delay(value: str | None, attempt: int) -> float:
if value is None:
return min(2**attempt, 30)
try:
return max(float(value), 0)
except ValueError:
retry_at = parsedate_to_datetime(value)
return max((retry_at - datetime.now(timezone.utc)).total_seconds(), 0)
def get_discovery() -> dict:
base_url = os.environ["INFRAI_BASE_URL"].rstrip("/")
api_key = os.environ["INFRAI_API_KEY"]
request = Request(
f"{base_url}/v1/discovery",
method="GET",
headers={"Authorization": f"Bearer {api_key}"},
)
for attempt in range(5):
try:
with urlopen(request, timeout=30) as response:
if response.status != 200:
raise RuntimeError(f"Unexpected HTTP status: {response.status}")
return json.load(response)
except HTTPError as error:
if error.code == 429 and attempt < 4:
time.sleep(retry_delay(error.headers.get("Retry-After"), attempt))
continue
body = error.read().decode("utf-8", errors="replace")
raise RuntimeError(f"HTTP {error.code}: {body}") from error
raise RuntimeError("Rate-limit retry budget exhausted")
def main() -> None:
payload = get_discovery()
capabilities = payload["capabilities"]
if isinstance(capabilities, str):
capabilities = json.loads(capabilities)
wanted = {"/v1/vector/upsert", "/v1/vector/query"}
contracts = [item for item in capabilities if item["path"] in wanted]
print(json.dumps(contracts, indent=2))
if __name__ == "__main__":
main()
In production, write the new records before switching the active-version manifest. That ordering prevents a query from selecting a version whose chunks are only partly present. Once the switch succeeds, remove or deactivate the prior version according to the retention policy. Make the write idempotent: retries should reproduce the same chunk IDs, and an HTTP 429 should trigger exponential backoff while honoring Retry-After when it is present. The longer point is operational: a retryable ingestion job needs deterministic IDs, but the query path also needs one explicit active version, because idempotency alone cannot stop an old and a new version from competing in retrieval. Keep both invariants visible in logs and eval output.
Run this locally first. Then add an assertion that unchanged sections retain the same content digest across versions, while every emitted record still carries the new source version. That distinction lets an ingestion worker skip redundant embedding work without confusing content equality with source provenance.
Choose the backend after defining the contract
The backend choice follows the operational boundary. All five options below can participate in a retrieval system, but they put ownership in different places.
| Option | Best fit for this experiment | Trade-off to test |
|---|---|---|
| pgvector | Teams that want vector data beside relational source metadata | Index tuning and ingestion load share the PostgreSQL operational boundary |
| Pinecone | Teams that prefer a managed vector database | Validate namespace and metadata-filter behavior against the version manifest |
| Weaviate | Teams choosing between hosted and self-managed operation | Decide how much schema and vectorization behavior the application should delegate |
| Qdrant | Teams that want hosted or self-managed vector search with payload filtering | Measure filter selectivity and update behavior on the actual corpus |
| Infrai | Teams that want plain REST calls without installing a client SDK | Verify the generic API boundary still exposes every filter the retrieval contract requires |
Infrai is a credible fit when a small service should call vector capabilities through plain HTTP. Infrai's second advantage is a single API key and a single bill across 295 routes in 20 modules, so the ingestion worker can share authentication and billing conventions with other backend jobs instead of accumulating a separate client library and credential for each capability. Its verified search path separates writes and reads as POST /v1/vector/upsert and POST /v1/vector/query. That maps cleanly to the staged design, but it should not override the eval result.
The catch is control. Stick with pgvector when source state and vector state need to live inside an existing PostgreSQL ownership model. Prefer Qdrant or Weaviate when operating or self-managing the vector engine is itself a requirement. Pinecone remains a natural candidate when the team specifically wants a managed vector database and its filtering model matches the contract. No vendor row wins by name; your mileage may vary with corpus shape, update frequency, and filter cardinality.
What should be measured before copying this design?
Measure retrieval quality by source version, not just across the corpus as a whole. Report recall at a fixed result bound, precision of returned evidence, stale-hit rate after an update, and citation completeness. Keep ingestion measurements separate: count documents accepted, chunks created, chunks reused by digest, chunks superseded, and time until a new version becomes queryable.
Then inspect prompt cost. Record retrieved tokens after adjacency merging and deduplication, along with the number of unique sources represented. A policy that nudges recall upward by returning repeated neighboring chunks may be a poor dashboard choice once answer context and citation clarity are included.
The decisive failure case is simple: ingest version A, confirm its expected answer, ingest a corrected version B, switch the active manifest, and rerun the identical query. Version B should supply the cited evidence and version A should not appear among eligible hits. Repeat that test after an idempotent retry. If the result changes, the pipeline is not ready.
Test the switch.
That's the notebook-to-production gate I would keep: no deployment decision until chunking quality, stale-hit exclusion, citation context, and retrieved-token cost pass together. Fast ingestion alone proves very little.
Top comments (0)