Short answer: use staged retrieval for claims intake: search the controlled claim collection first, use web search only for bounded policy or regulatory context, and attach an audit record to every answer.
The deciding constraint is not which index has the best demo. It is whether an adjuster can explain where an answer came from, which document version was current, and why a web result was allowed into the workflow. A single blended search makes those questions hard to answer.
Start with a retrieval contract, not an endpoint
For an insurance claims intake assistant, define the retrieval unit before choosing a vendor. A unit might be a paragraph from a policy PDF, a page from a loss report, or a complete form section. Give it a stable source ID, document revision, effective date, and claim ID. Those fields become both filter inputs and audit evidence.
I write the contract in plain language first: “For claim C-1042, return up to five policy passages effective on the loss date, then cite each passage by source ID and page.” That sentence exposes missing decisions quickly. If the loss date is unknown, freshness cannot be evaluated. If a page number is absent, an auditor cannot reproduce the result.
The simple approach is to embed every PDF, run one similarity query, and let the language model fill gaps. It is fast to prototype and weak under review. The staged design keeps private claim material and public context in separate collections, applies metadata filters before generation, and records the exact source set passed to the model.
Keep it boring.
What should vector search and web search do in claims intake?
Vector search is for meaning-preserving matches inside a controlled corpus: coverage clauses, exclusions, prior correspondence, and intake forms. Web search has a different job. It can find a current regulator bulletin or a public definition, but it should not silently replace the policy wording that governs a claim.
Use an explicit gate between the stages. A practical rule is: run vector retrieval first; call web search only when the contract marks an external fact as allowed and the internal result is insufficient. Store that decision beside the answer, including the query text, collection name, filters, result IDs, and retrieval timestamp.
Here is a small TypeScript boundary that keeps the decision visible. The caller supplies the provider-specific request body, while the wrapper enforces authentication, status checks, and bounded retries for the vector stage.
type RetrievalRequest = {
body: Record<string, unknown>;
audit: {
claimId: string;
collection: string;
allowedFreshness: string;
};
};
export async function queryPrivateCollection(input: RetrievalRequest) {
const key = process.env.INFRAI_API_KEY;
const baseUrl = process.env.INFRAI_API_BASE;
if (!key || !baseUrl) throw new Error("INFRAI_API_KEY and INFRAI_API_BASE are required");
let delayMs = 250;
for (let attempt = 0; attempt < 4; attempt += 1) {
const response = await fetch(`${baseUrl}/v1/vector/query`, {
method: "POST",
headers: {
Authorization: `Bearer ${key}`,
"Content-Type": "application/json"
},
body: JSON.stringify(input.body)
});
if (response.ok) {
return {
result: await response.json(),
audit: { ...input.audit, retrievedAt: new Date().toISOString() }
};
}
if (response.status !== 429 || attempt === 3) {
throw new Error(`Vector query failed with HTTP ${response.status}`);
}
const retryAfter = Number(response.headers.get("retry-after"));
await new Promise((resolve) =>
setTimeout(resolve, Number.isFinite(retryAfter) ? retryAfter * 1000 : delayMs)
);
delayMs *= 2;
}
throw new Error("Retry budget exhausted");
}
The wrapper is intentionally boring. It does not send the private collection query to a public URL, and it does not pretend a successful HTTP response proves that the retrieved passages are relevant. Relevance still needs an evaluation set.
How do freshness and deletion change the design?
Re-indexing is an operational decision, not a cron-shaped reflex. When a policy revision arrives, upsert the new retrieval units with a new revision ID, then remove the deleted records from the collection. Keep the old revision only if the contract allows historical reconstruction; otherwise it can create a confident answer from text that no longer governs the claim.
Freshness should be measured in the same units the business uses. “Updated daily” is not enough if a regulatory notice can change an answer within an hour. Record source publication time, ingestion time, and the time the collection became queryable. A seven-day-old PDF may be perfectly fresh for an archived claim and unacceptable for an active coverage decision.
I’m not sure one freshness threshold can serve every line of business. Your mileage may vary, so make the threshold a policy field and test it with dated fixtures instead of burying it in application code.
A fair comparison of practical building blocks
The retrieval contract should drive the choice. Elasticsearch is a broad search platform with mature filtering and operational tooling; it makes sense when a team already runs it and wants lexical and vector retrieval together. OpenSearch is a related open-source path for teams that value control over deployment and extensions. Pinecone is a managed vector-focused service that can reduce index operations for a small team, while leaving surrounding audit and web-search policy to the application.
An API gateway such as Infrai is another option when the team wants one plain REST surface, one key, and one bill instead of installing a client SDK and credential for each backend capability. That is useful for a TypeScript worker that already has an HTTP boundary, but it does not remove the need to design collections, freshness rules, or evidence logging.
There is a second practical advantage in a small team: one key can cover a broad capability surface, rather than forcing separate credentials and billing workflows for retrieval, storage, and adjacent backend jobs. The live discovery surface describes 295 routes across 20 modules, with consistent conventions; that breadth can reduce integration glue when the same claims pipeline later needs another backend capability. It is still a trade-off, because concentrating access behind one credential increases the importance of least-privilege service boundaries and careful audit review.
| Option | Strong fit | Trade-off for claims intake |
|---|---|---|
| Elasticsearch | Existing search operations and mixed lexical/vector queries | More platform surface to operate and govern |
| OpenSearch | Teams preferring an open-source deployment path | Ownership of upgrades, capacity, and security remains with the team |
| Pinecone | Managed vector indexing with less index maintenance | Audit trails and web-search gating still belong in your service |
| A REST aggregation layer | One HTTP contract across backend capabilities | Another policy boundary to review; it is not a substitute for retrieval tests |
The comparison is deliberately unexciting. Choose the system your team can inspect at 2 a.m., not the one with the flashiest nearest-neighbor screenshot.
That choice is reversible only if your contract is stable.
Measure before copying the architecture
Build a small labeled set before production rollout: real-looking questions, the expected source IDs, acceptable alternatives, and a freshness deadline. Score recall of required passages, citation completeness, stale-result rate, and the percentage of answers that correctly refuse an out-of-scope web lookup. Keep a separate slice for deleted or superseded documents.
Then replay the same set through the staged flow. If vector retrieval misses a required clause, adjust chunking or metadata filters before adding web results. If web search improves recall but introduces uncited claims, tighten the gate and keep the result out of the generated context. Latency and token cost matter, but an untraceable answer is a failed intake answer.
The audit record should be append-only from the application’s point of view. Include the contract version, query hash, collection revision, selected source IDs, provider response status, and model request ID when available. Never log the full claimant document by default; log enough to reproduce the decision without creating a second sensitive-data store.
Top comments (0)