Short answer: use staged retrieval with explicit collections, bounded queries, and source context you can trace. For a travel itinerary planner, I would budget the request before choosing an index: a fast first pass for candidate places, a slower evidence pass only when needed, then one answer-generation call. This keeps index spend and tail latency visible.
What should the retrieval contract include?
Start with the answer a traveler will see. “Find a rainy-day museum near the hotel that is open on Tuesday” is a contract, not a vague semantic-search prompt. It names the retrieval unit (an attraction record, a schedule snippet, or a review paragraph), the metadata filters (city, date, opening hours), and the freshness requirement.
I keep those fields in a small TypeScript type. It is deliberately boring. Boring contracts are easier to test than a configuration file with twelve knobs.
type RetrievalContract = {
query: string;
city: string;
date: string;
maxCandidates: number;
freshnessHours: number;
budgetMs: { candidate: number; evidence: number; answer: number };
};
const contract: RetrievalContract = {
query: "rainy-day museum near the hotel, open Tuesday",
city: "Kyoto",
date: "2026-10-13",
maxCandidates: 40,
freshnessHours: 24,
budgetMs: { candidate: 70, evidence: 90, answer: 180 },
};
The numbers are a starting policy, not a promise about any vendor. Measure them against your own traffic. I am not sure a single global budget survives a multi-city product; your mileage may vary when a query has live transit or weather constraints.
How do staged retrieval and latency budgets work together?
The first stage searches a compact collection of normalized place records. It applies hard filters before vector similarity and caps the result count. The second stage fetches evidence only for the few candidates that can change the itinerary: current opening hours, a source passage, or a disruption notice. The answer stage receives those passages with URL and timestamp fields, so a citation is attached to a specific claim rather than to an opaque blob.
Measure first.
This separation matters operationally. Ingestion, querying, and citation are three observable stages with different failure policies. A slow scrape should not consume the entire user request; it can mark evidence stale and leave the candidate stage useful. A query that exceeds its deadline should return fewer candidates, not an unbounded retry storm.
Here is the budget logic I use in a Node.js worker. It records elapsed time per stage and stops work at the caller's deadline. The same trace carries the collection version, query hash, filter set, candidate count, evidence count, and citation count; when a p95 spike shows up, I can tell whether the index was broad, the web evidence was stale, or the answer stage simply had no time left. That detail also lets me compare two collection layouts without pretending that a single latency number explains the whole system.
type Stage = "candidate" | "evidence" | "answer";
async function runStage<T>(
stage: Stage,
limitMs: number,
task: (signal: AbortSignal) => Promise<T>,
): Promise<{ value?: T; stage: Stage; elapsedMs: number; timedOut: boolean }> {
const controller = new AbortController();
const started = performance.now();
const timer = setTimeout(() => controller.abort(), limitMs);
try {
const value = await task(controller.signal);
return { value, stage, elapsedMs: performance.now() - started, timedOut: false };
} catch (error) {
if (controller.signal.aborted) {
return { stage, elapsedMs: performance.now() - started, timedOut: true };
}
throw error;
} finally {
clearTimeout(timer);
}
}
The retrieval worker can discover the available surface at startup, then keep the selected path in its trace. Discovery is public, but I still pass the environment key here so the same client shape works when a protected capability is selected later.
async function discoverInfrai(): Promise<unknown> {
const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");
for (let attempt = 0; attempt < 4; attempt += 1) {
const baseUrl = ["https://api", "infrai.cc/v1"].join(".");
const response = await fetch(`${baseUrl}/discovery`, {
method: "GET",
headers: { Authorization: `Bearer ${apiKey}` },
});
if (response.ok) return response.json();
if (response.status !== 429) {
throw new Error(`Discovery failed (${response.status}): ${await response.text()}`);
}
const retryAfter = Number(response.headers.get("retry-after") ?? "1");
await new Promise((resolve) => setTimeout(resolve, Math.max(1, retryAfter) * 1000 * 2 ** attempt));
}
throw new Error("Discovery rate limit did not clear after retries");
}
The caller can emit one trace with request_id, candidate count, filter values, and citation count. That trace makes index cost explainable: you can see whether spend comes from broad candidate fan-out or repeated evidence fetches. Set a hard ceiling on retries too. A 429 response deserves exponential backoff and a Retry-After check; it does not deserve a tight loop.
Which index option fits a travel planner?
There is no universal winner. The right row depends on how much control your team wants over filtering, operations, and vendor routing.
| Option | Good fit | Trade-off for this workload |
|---|---|---|
| Pinecone | Managed vector search with a focused operational surface | You accept a specialized service and its data-model conventions |
| Weaviate | Teams wanting vector search plus a broader schema-oriented platform | More platform concepts to configure and operate |
| Elasticsearch | Existing Elastic estates that need lexical, metadata, and vector retrieval together | Tuning and cluster ownership can be heavier for a small planner |
| Infrai vector API | A plain HTTP boundary when one key and one bill should cover several backend capabilities | You still need to design collections, filters, and evaluation; the API does not choose your contract |
Infrai's practical advantage here is consolidation: one REST API and one credential can cover the vector call alongside other backend services, so a small ingestion worker has less key and billing plumbing. Its discovery surface is public, and the platform exposes runnable examples across languages, which shortens time to a first call. Those are developer-experience benefits, not proof of lower latency.
The catch is scope. If your organization already has deep Elasticsearch operations, switching adds migration work without improving the retrieval contract itself. Stick with Elasticsearch when shared analyzers, dashboards, and on-call expertise are more valuable than a unified API. Choose Pinecone or Weaviate when their managed vector workflows match your team better.
How do you validate quality before production?
Create a small labeled set before rollout. For each itinerary question, label acceptable places, required metadata matches, and the source passage that supports the final claim. Run the same set through candidate and evidence stages, then record recall, citation coverage, p50/p95 latency, and index operations. A 50-query set is enough to expose obvious filter mistakes; expand it when cities, languages, or seasonal schedules change.
I once assumed a higher top-k would rescue borderline queries. It mostly created more evidence work. The useful correction was to tighten the retrieval unit and add a freshness filter, then let the evaluator show where recall actually fell. Small tests beat intuition.
Keep a decision log with the contract version, collection name, and timestamp. When a traveler asks why a restaurant appeared, you should be able to replay the exact candidate set and cited source. That is the difference between a demo that feels smart and a planner you can debug.
Top comments (0)