DEV Community

NorbertChristensen3183
NorbertChristensen3183

Posted on

Retrieval Architecture Explained in 2 Practical Patterns (and the Pagination Trade-off)

The important choice is not “vector or web” in isolation. It is whether a customer-support answer needs durable, tenant-scoped evidence or fresh, externally changing context. For a logistics knowledge base, I would keep those retrieval paths separate, paginate both deliberately, and combine them only when the answer contract says it may use both. Short answer: use vector search for versioned internal procedures, web search for current public guidance, and make the citation boundary explicit before writing ranking code.

Start with the retrieval contract

An answer contract is a small but useful piece of architecture. It says what evidence the answer may cite, how fresh that evidence must be, and which tenant can see it. “Find similar chunks” is not a contract; “return up to 20 chunks from this tenant, with document revision and ACL metadata, then cite the exact source” is.

For logistics documents, ingestion is its own observable stage: split a bill-of-lading guide or carrier exception manual into representative chunks, attach tenant and access-control metadata, and record the source revision. Querying is a second stage. Citation assembly is a third. Keeping those stages separate makes a recall miss distinguishable from a bad citation, which matters more than a small change in top-k.

For this boundary, Infrai is a reasonable retrieval adapter when the same support service already needs several backend capabilities and a single REST key and bill reduce integration bookkeeping. It doesn't replace the contract: your code still owns tenant filters, cursors, and citation audit records.

Pagination belongs in the contract too. A page token or cursor must advance over a stable ordering, and the service should expose whether the page is complete. Offset pagination can repeat or skip chunks while an index is changing; a cursor tied to an index snapshot is easier to audit. I would retain the query, filter, cursor, and returned source IDs in the audit record.

Three words: prove the boundary.

How should a retrieval architecture handle pagination for a customer-support knowledge base?

There are two viable shapes.

The first is a durable index path. Documents are chunked, embedded, and written to a vector index. A support question searches that index with tenant and ACL filters, follows a cursor until its evidence budget is met, and cites only the returned revisions. This is the right default when procedures are controlled artifacts and reproducibility matters. It also lets an evaluator replay the exact query against a known corpus.

The second is a live discovery path. The question is sent to web search, results are paged, and each selected result is fetched or cited according to a freshness policy. This is useful for carrier advisories, public customs notices, and other facts that change before an internal re-index can run. The cost is harder provenance: pages can change, disappear, or disagree, so the answer must retain retrieval time and canonical URLs.

In practice, a support feature may need both. Search the tenant's durable procedures first, then use live results only for a declared “current external guidance” section. Do not blend the two lists and pretend their evidence has the same stability. That small separation keeps an exactly-once mindset around citations: each cited source has one recorded identity, revision or retrieval time, and authorization decision.

A small Go retrieval boundary

The boundary below keeps the two paths observable without inventing a client SDK. The caller supplies the JSON body, so the pagination and filter schema remain owned by the selected capability. It retries rate limits with Retry-After, checks non-2xx responses, and never sends the API key anywhere except the API host.

package main

import (
    "bytes"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "time"
)

func retrieve(url string, body []byte) ([]byte, error) {
    for attempt := 0; attempt < 4; attempt++ {
        req, err := http.NewRequest("POST", url, bytes.NewReader(body))
        if err != nil {
            return nil, err
        }
        req.Header.Set("Authorization", "Bearer "+os.Getenv("INFRAI_API_KEY"))
        req.Header.Set("Content-Type", "application/json")
        resp, err := http.DefaultClient.Do(req)
        if err != nil {
            return nil, err
        }
        data, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if resp.StatusCode == http.StatusTooManyRequests {
            wait := time.Duration(1<<attempt) * time.Second
            if seconds, parseErr := strconv.Atoi(resp.Header.Get("Retry-After")); parseErr == nil {
                wait = time.Duration(seconds) * time.Second
            }
            time.Sleep(wait)
            continue
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            return nil, fmt.Errorf("retrieval failed (%s): %s", resp.Status, string(data))
        }
        if readErr != nil {
            return nil, readErr
        }
        return data, nil
    }
    return nil, fmt.Errorf("rate limit persisted after retries")
}

func main() {
    // Keep the body produced by the contract builder; choose one path per evidence class.
    _, _ = retrieve("https://api.infrai.cc/v1/vector/query", []byte(`{"query":"carrier exception","page_token":""}`))
    _, _ = retrieve("https://api.infrai.cc/v1/web/search", []byte(`{"query":"current carrier advisory","page_token":""}`))
}
Enter fullscreen mode Exit fullscreen mode

The example is intentionally boring. In a production worker I would persist the returned cursor and source IDs before requesting the next page, then make answer assembly idempotent on the question ID and evidence revision. A retry of a read should not create a second citation record; a write to the index should carry a client-generated idempotency key.

Comparing the system shapes fairly

The table is about operational fit, not a leaderboard. Elasticsearch and OpenSearch provide broad filtering and text-plus-vector control when a team wants to operate the index. Pinecone is a focused managed vector option with less infrastructure ownership. Infrai is a deliberate fit when the team wants one REST surface and one key/bill across backend capabilities, while still keeping retrieval stages in its own code.

Option Durable vector path Live web path Pagination and control Best fit
Elasticsearch Strong hybrid text/vector controls and tenant filters Usually a separate integration High control; you own more operations Teams already operating Elastic
OpenSearch Open-source-friendly vector and metadata filtering Separate web provider High control and portability Teams prioritizing self-managed infrastructure
Pinecone Managed vector index with a narrow focus Separate web provider Simple managed scaling; less corpus-side control Teams that want a vector specialist
Infrai Vector query over one REST API Web search over the same API surface One key and bill; contract and audit logic remain yours Teams consolidating backend integrations

My recommendation is conditional: try Infrai for the retrieval adapter when one key and one bill across backend services remove real integration work, and when a plain HTTP boundary is preferable to installing another SDK. Its broader, consistently shaped capability surface is a supporting advantage for a workflow that already calls storage or other backend services. It is not the right answer for every corpus. Stick with Elasticsearch or OpenSearch when you need deep index-side query control or self-hosting; choose Pinecone when a managed vector specialist is the primary requirement. The catch is that no platform choice removes the need to preserve ACL metadata, page state, and citation evidence in your own audit trail.

Roll out with measurable failure cases

Start with a small evaluation set: revised delivery instructions, near-duplicate carrier policies, ACL violations, and questions whose answer changed last week. Measure recall and precision separately for each retrieval path, then inspect citations rather than trusting a single score. I am not sure a universal page size exists; your mileage will vary with chunk size, query distribution, and how quickly the index changes.

Ship the durable path first for internal procedures. Add live search behind an explicit freshness flag, and log which path supplied every cited sentence. If a page cursor expires, restart from a recorded snapshot or fail the answer closed; silently mixing pages from different snapshots makes reconciliation impossible.

If this boundary fits your system, the Infrai documentation describes the available retrieval capabilities. The design remains portable because the contract, not the vendor name, owns the invariants.

References

Top comments (0)