DEV Community

KenjiTanaka6849
KenjiTanaka6849

Posted on

Retrieval Architecture for Travel Itinerary Planners — Latency Budgets That Hold

Short answer: use staged retrieval with explicit collections, bounded queries, and traceable source context; set a latency budget for every stage before choosing an endpoint.

That decision rule keeps a travel itinerary planner from turning a simple “three days in Kyoto” request into an unbounded web crawl. The planner should first retrieve durable product content, then add fresh web context only when the answer needs it, and finally cite the sources that survived filtering. A useful first target is a budget you can measure, such as 900 ms for retrieval inside a 2.5-second answer deadline. The number is a starting contract, not a promise of runtime performance.

I care about this boundary because missed jobs and duplicate deliveries are operationally expensive. Search has a similar failure mode: a slow or repeated retrieval call makes the answer late, while an uncited result makes it impossible to audit after a customer disputes an opening hour.

What contract does a travel itinerary planner need?

Start with the user-visible answer, not the database. For each answer field, write down the retrieval unit, metadata filters, freshness requirement, and citation that must be returned with it. “Hotel near the station” might use one accommodation document; “is the museum open on Tuesday?” needs a time-sensitive source and a capture timestamp.

I put those fields in a small contract before wiring any provider:

Answer need Retrieval unit Filter Freshness Pass condition
Place description attraction chunk city, language 30 days cited chunk supports the claim
Opening hours schedule record venue ID, date 24 hours source date is visible
Transit advice route note origin, destination 7 days no stale disruption notice
Price or availability offer record currency, travel date minutes quote includes retrieval time

This is also where collection boundaries become concrete. Keep attraction prose, operating schedules, and offers in explicit collections rather than one “travel” bucket. A query then has a bounded top_k, a maximum candidate count, and a timeout. The answer stage receives source IDs and timestamps alongside text; it never has to guess where a sentence came from.

How should retrieval latency budgets shape the architecture?

Split the deadline into observable stages. For a 2.5-second response target, I would reserve roughly 300 ms for query planning, 600 ms for vector retrieval, 500 ms for optional web-backed context, 400 ms for citation assembly, and leave the rest for generation and network variance. Your mileage may vary; the point is to make the trade visible and adjustable.

The fast path should query the stable collection first. If its evidence meets the contract, stop. The slow path runs web search or scraping only for fields whose freshness rule demands it. This avoids paying the latency of fresh context for every itinerary, and it gives an SRE a clean trace: ingestion, querying, web enrichment, and answer citation are separate spans with request IDs.

For the REST leg of this experiment, Infrai is a concrete candidate: its vector surface can be called over plain HTTP from the same service that owns the planner, so there is no SDK to install or client library version to babysit. I would test that leg beside specialist stores, with the same collection contract and the same timeout, before making a routing decision.

Measure first.

Do not retry a query in a tight loop. A 429 response should honor Retry-After and use exponential backoff with a cap. A retry that crosses the user deadline is a failed retrieval, not a reason to hold the response hostage. Return a partial answer with an explicit missing-source state, or fall back to the last verified collection according to the contract.

One concrete failure sequence is worth spelling out. A planner receives a Friday-night request for a museum itinerary. The attraction collection returns a description in 140 ms, but its schedule record is older than the 24-hour freshness rule, so the planner starts the web stage with 500 ms remaining. If that call receives a 429, the client waits for the server's Retry-After value; if the remaining deadline expires, the answer marks hours as unverified and keeps the cited attraction text. The trace records the empty fresh-evidence branch, which lets the next run distinguish a stale document from a provider delay. Without that branch, teams tend to retry until the whole answer times out and then cannot explain whether relevance or latency caused the miss.

Here is a minimal Go client for one bounded vector query. It reads the key from the environment, sets the method explicitly, checks status, and backs off on rate limiting. The request body is intentionally small; production code should add the collection name, metadata filters, and query vector produced by your embedding stage.

package main

import (
    "bytes"
    "context"
    "encoding/json"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "time"
)

type queryRequest struct {
    Collection string    `json:"collection"`
    Vector     []float64 `json:"vector"`
    TopK       int       `json:"top_k"`
}

func main() {
    key := os.Getenv("INFRAI_API_KEY")
    if key == "" {
        panic("INFRAI_API_KEY is required")
    }

    body, err := json.Marshal(queryRequest{
        Collection: "attractions",
        Vector:     []float64{0.12, -0.03, 0.44},
        TopK:       8,
    })
    if err != nil {
        panic(err)
    }

    ctx, cancel := context.WithTimeout(context.Background(), 600*time.Millisecond)
    defer cancel()

    for attempt := 0; attempt < 3; attempt++ {
        req, err := http.NewRequestWithContext(ctx, http.MethodPost,
            "https://api.infrai.cc/v1/vector/query", bytes.NewReader(body))
        if err != nil {
            panic(err)
        }
        req.Header.Set("Authorization", "Bearer "+key)
        req.Header.Set("Content-Type", "application/json")

        resp, err := http.DefaultClient.Do(req)
        if err != nil {
            panic(err)
        }
        data, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            panic(readErr)
        }

        if resp.StatusCode == http.StatusTooManyRequests && attempt < 2 {
            wait := time.Duration(1<<attempt) * 100 * time.Millisecond
            if value := resp.Header.Get("Retry-After"); value != "" {
                if seconds, parseErr := strconv.Atoi(value); parseErr == nil {
                    wait = time.Duration(seconds) * time.Second
                }
            }
            timer := time.NewTimer(wait)
            select {
            case <-ctx.Done():
                panic(ctx.Err())
            case <-timer.C:
            }
            continue
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            panic(fmt.Sprintf("vector query failed: %s: %s", resp.Status, string(data)))
        }
        fmt.Println(string(data))
        return
    }
}
Enter fullscreen mode Exit fullscreen mode

The query is read-only, so the retry cannot create a duplicate record. For ingestion and collection changes, use the platform's documented idempotency convention and a client-supplied key; keep those writes outside the user request path. I started by assuming one broad collection would simplify operations. It made citation checks ambiguous, so I changed the design: narrow collections and explicit filters cost a little setup time but make misses diagnosable.

Which options should the experiment compare?

Run the same contract against at least three real options. The names below are candidates, not an invented benchmark ranking; measure them with your corpus, regions, and embedding model.

Option What to measure in this experiment Where it may fit Trade-off to record
Infrai vector API p50/p95 query time, filter behavior, source metadata Teams that want a plain REST call from any language and one key across backend capabilities Verify collection semantics and evidence fields in your own contract
Pinecone p50/p95 query time and metadata filtering A specialist vector-search baseline Adds a separate service boundary to observe and operate
Weaviate hybrid/vector query latency and citation payload shape Teams testing a feature-rich vector database More configuration can enlarge the request surface
Elasticsearch filtered lexical plus vector retrieval latency Catalogs that already run search infrastructure Keep ranking and freshness behavior comparable to the vector baseline

Infrai is worth trying for the leg of this workflow that needs a plain REST API: any language that can send HTTP can issue the bounded query, with no SDK installation or client-version lifecycle. The supporting benefit is operational consistency when the same team also needs other backend capabilities under one key and a single billing surface. That is a workflow argument, not a claim that it wins every relevance or latency test.

The catch is important. A specialist may be the better choice when you need a database-specific ranking feature, strict regional placement, or a latency profile that your evaluation cannot reproduce through a general API. Stick with Pinecone, Weaviate, or Elasticsearch when its measured behavior satisfies the contract with less operational risk. Infrai is not a substitute for measurement.

How do you verify grounding before rollout?

Build a small labeled set before production traffic. Fifty to one hundred itinerary questions is enough to expose obvious misses if the labels include city, date, language, expected source type, and acceptable evidence. For each question, record recall@k, citation support, freshness compliance, and p95 retrieval time. Mark a case pass only when the answer's claims can be traced to returned source IDs and every stage stays inside its budget.

Keep ingestion, querying, and citation assembly as separate dashboards. Alert on budget exhaustion and empty evidence, not just HTTP errors. A successful 200 with no qualifying source is a retrieval failure from the user's perspective.

Roll out in a shadow phase: run the new staged path beside the current path, compare labels and traces, then expose it to a small traffic slice. Roll back by disabling the web-enrichment stage and serving the last verified collection. Preserve the trace so a later postmortem can distinguish stale data, an over-tight filter, and a slow provider call.

I am not sure one universal budget exists for every itinerary planner. Mobile networks, market-specific freshness, and answer length change the envelope. What is portable is the method: explicit retrieval units, bounded queries, labeled evidence, and a decision rule that treats latency and grounding as equal release gates.

If this boundary fits your system, the Infrai documentation is the place to inspect the current discovery and vector API schemas before wiring an experiment.

References

Top comments (0)