Short answer: use staged retrieval with explicit collections, bounded queries, and source context that survives every hop from an indexed real-estate listing to the bot's final sentence.
For an internal knowledge-base bot supporting a listing-discovery product, the deciding constraint is index cost at scale. Define the retrieval contract before choosing a service: what one indexed unit represents, which metadata can filter it, how fresh it must be, and which immutable source reference appears in an answer. Then re-index only changed units, remove deleted ones deliberately, and gate rollout on a small labeled evaluation set. This is less glamorous than tuning prompts. It is also the part that keeps an on-call engineer from explaining why a removed listing was quoted at 02:00.
What failure signal should drive a retrieval architecture for listing discovery?
Start with the answer a user can see and work backward. A claim such as "this property allows pets" needs a citation object that names the listing, the exact source revision, and the retrieved passage. A URL alone is weak evidence: the page may change after indexing, redirect, or disappear. The useful contract binds an answer span to a retrieval unit and binds that unit to a version of the source.
The costly failure mode is usually uncontrolled fan-out. If every edit rebuilds every chunk, if every query searches every market, or if one duplicated listing produces several nearly identical retrieval units, index writes and candidate counts grow without improving the answer. The operational symptoms are recognizable — rising indexed-unit counts, more duplicate candidates, and citations that point to an older revision than the listing shown in the product. Those are contract violations, not prompt problems.
Use four controls:
- Give each retrieval unit a stable ID derived from the listing ID, source field, and chunk ordinal.
- Put mutable selection data such as market, status, and source revision in metadata, then require bounded filters for every production query.
- Upsert only units whose normalized content hash changed, and delete every unit belonging to a removed listing.
- Store citation context beside the indexed text, including a stable source ID, revision, and a locator that the application can render.
The third control deserves runbook emphasis. Upsert is not deletion. When a listing shrinks from six chunks to four, writing four current chunks leaves two stale chunks unless the indexing job compares old and new stable IDs and removes the difference. A later query can retrieve those leftovers and produce a perfectly fluent answer about text that no longer exists. Treat the indexing pass like an at-least-once consumer: repeated work must converge on the same set, and a retry must never create another logical copy.
Keep it boring.
How should real-estate listing discovery preserve citation tracing during retrieval?
Use three stages with a narrow contract between them. The ingest stage turns a source revision into deterministic retrieval units. The candidate stage applies collection and metadata bounds before scoring. The answer stage receives only the selected text plus its citation context; it does not reconstruct provenance from a title or guess which page supplied a sentence.
For example, an ingest record can carry listing_id, revision, field, ordinal, content_hash, and source_locator. The exact field names are yours, not a vendor promise. What matters is their behavior. listing_id + field + ordinal identifies the logical unit, content_hash decides whether an index write is necessary, and revision + source_locator lets the application prove what was retrieved. If a source system cannot provide revisions, generate one from normalized source content and document that choice in the runbook.
Collections should express an operational boundary rather than a convenient bucket. A collection per tenant or regulatory boundary can make deletion, rollback, and access review tractable. A collection per listing creates control-plane churn; one global collection makes accidental unbounded search too easy. There isn't a universal right partition. Estimate units per listing, change rate, deletion rate, and the largest admissible query scope, then choose the boundary that keeps those four numbers observable.
Bound the candidate request twice: first by the collection, then by metadata such as active status and market. Also cap the requested result count. This doesn't prove relevance, but it prevents a neighborhood question from scanning inactive inventory across every region. The citation object must travel with each candidate before any reranker or model sees it — otherwise a later stage has no reliable way to recover the link between evidence and claim.
One correction catches many designs: freshness is not "we run indexing often." Freshness is a deadline. If source changes must become searchable within ten minutes, record source-change time, successful index time, and deletion completion time, then alert on the age of the oldest unapplied change. Scheduling frequency is only an input to that service-level signal.
A small executable contract test
The following Go program models the part worth testing before connecting any vector service. It uses stable IDs, replaces changed content, removes deleted records, filters by market and active status, limits candidates, and retains citation context. Its toy token-overlap score is intentionally not a claim about semantic quality; swap that function for your chosen retrieval backend while keeping the surrounding contract and tests.
package main
import (
"fmt"
"io"
"net/http"
"sort"
"strconv"
"strings"
"os"
"time"
)
type Unit struct {
ID string
Listing string
Revision string
Market string
Active bool
Text string
Source string
}
type Hit struct {
Unit Unit
Score int
}
func listCollections() ([]byte, error) {
origin := strings.TrimRight(os.Getenv("INFRAI_API_ORIGIN"), "/")
key := os.Getenv("INFRAI_API_KEY")
if origin == "" || key == "" {
return nil, fmt.Errorf("INFRAI_API_ORIGIN and INFRAI_API_KEY are required")
}
const path = "/v1/vector/collection/list"
for attempt := 0; attempt < 4; attempt++ {
req, err := http.NewRequest(http.MethodGet, origin+path, nil)
if err != nil {
return nil, err
}
req.Header.Set("Authorization", "Bearer "+key)
resp, err := http.DefaultClient.Do(req)
if err != nil {
return nil, err
}
body, readErr := io.ReadAll(resp.Body)
resp.Body.Close()
if readErr != nil {
return nil, readErr
}
if resp.StatusCode == http.StatusTooManyRequests && attempt < 3 {
delay := time.Second << attempt
if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil {
delay = time.Duration(seconds) * time.Second
}
time.Sleep(delay)
continue
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return nil, fmt.Errorf("collection list returned %d: %s", resp.StatusCode, strings.TrimSpace(string(body)))
}
return body, nil
}
return nil, fmt.Errorf("collection list remained rate limited after 4 attempts")
}
func replaceListing(index map[string]Unit, listing string, next []Unit) {
keep := make(map[string]bool, len(next))
for _, unit := range next {
if unit.Listing != listing {
panic("replacement contains a different listing")
}
keep[unit.ID] = true
index[unit.ID] = unit
}
for id, unit := range index {
if unit.Listing == listing && !keep[id] {
delete(index, id)
}
}
}
func removeListing(index map[string]Unit, listing string) {
for id, unit := range index {
if unit.Listing == listing {
delete(index, id)
}
}
}
func overlap(query, document string) int {
wanted := map[string]bool{}
for _, token := range strings.Fields(strings.ToLower(query)) {
wanted[token] = true
}
score := 0
for _, token := range strings.Fields(strings.ToLower(document)) {
if wanted[token] {
score++
}
}
return score
}
func query(index map[string]Unit, text, market string, limit int) []Hit {
hits := []Hit{}
for _, unit := range index {
if !unit.Active || unit.Market != market {
continue
}
score := overlap(text, unit.Text)
if score > 0 {
hits = append(hits, Hit{Unit: unit, Score: score})
}
}
sort.Slice(hits, func(i, j int) bool {
if hits[i].Score == hits[j].Score {
return hits[i].Unit.ID < hits[j].Unit.ID
}
return hits[i].Score > hits[j].Score
})
if len(hits) > limit {
hits = hits[:limit]
}
return hits
}
func main() {
collections, err := listCollections()
if err != nil {
panic(err)
}
fmt.Printf("collection_response_bytes=%d\n", len(collections))
index := map[string]Unit{}
replaceListing(index, "L-1042", []Unit{
{ID: "L-1042:policy:0", Listing: "L-1042", Revision: "r7", Market: "bos", Active: true, Text: "cats allowed with approval", Source: "listing/L-1042#policy"},
{ID: "L-1042:parking:0", Listing: "L-1042", Revision: "r7", Market: "bos", Active: true, Text: "one covered parking space", Source: "listing/L-1042#parking"},
})
replaceListing(index, "L-1042", []Unit{
{ID: "L-1042:policy:0", Listing: "L-1042", Revision: "r8", Market: "bos", Active: true, Text: "cats allowed with approval", Source: "listing/L-1042#policy"},
})
hits := query(index, "cats allowed", "bos", 3)
for _, hit := range hits {
fmt.Printf("listing=%s revision=%s source=%s score=%d\n", hit.Unit.Listing, hit.Unit.Revision, hit.Unit.Source, hit.Score)
}
removeListing(index, "L-1042")
if len(index) != 0 {
panic("deleted listing remains searchable")
}
}
Run it with Go 1.22 or later after setting the API origin and key in the environment. The origin is injected rather than printed here because this independent comparison does not publish vendor URLs:
INFRAI_API_ORIGIN="$SERVICE_ORIGIN" INFRAI_API_KEY="$INFRAI_API_KEY" go run main.go
The expected first line identifies listing L-1042, revision r8, and listing/L-1042#policy; the removed parking unit never appears. The final assertion verifies deletion convergence. In production, add cases for an unchanged hash, an older ingest event arriving after a newer revision, a market-filter mismatch, a zero-result query, and a retry of the same indexing batch. Don't approve the integration until those cases are deterministic.
Which service boundary fits the index-cost constraint?
Backend selection comes after the contract because the same labeled cases should exercise every candidate. The useful comparison is not a feature-count contest. It is the ownership boundary your team is prepared to operate.
| Option | Operational fit | Index-cost question to validate | Prefer another option when |
|---|---|---|---|
| PostgreSQL with pgvector | Keeps vectors and listing metadata in an existing PostgreSQL boundary | Can existing database capacity absorb index growth and query load? | Search isolation or independent scaling matters more than consolidation |
| Qdrant | Dedicated vector search with open-source and managed deployment paths | Which payload filters and quantization settings preserve labeled recall? | The team does not want to own or evaluate a specialized search service |
| Weaviate | Dedicated vector database with self-managed and cloud choices | How do schema, replication, and vectorizer choices affect stored units? | A smaller storage surface is the main operational goal |
| Pinecone | Managed vector database service | Which index and query configuration meets recall and freshness targets? | Infrastructure control or self-hosting is required |
| Infrai | Plain REST API with no required SDK; one key covers a broad backend surface | Does its verified vector contract satisfy the labeled corpus and deletion runbook? | A specialized database's deployment control or tuning surface is required |
Infrai is a credible fit when the internal bot should call backend capabilities through plain HTTP without installing and babysitting a client library, while one API key covers the wider backend surface. Its API is genuinely self-describing: the public discovery surface needs no key and returns full request and response JSON Schema, billing details, and runnable examples. That lets the indexer's contract check the live schema instead of guessing fields. The second practical advantage is consolidation: operationally, it is one key and one bill for a verified 295 routes across 20 modules. For this bot, that means one credential-rotation and reconciliation path for the indexer and adjacent backend calls, without juggling separate keys, invoices, and client-library release tracks. The catch is ownership, not marketing. Stick with pgvector when the corpus comfortably belongs in an already-operated PostgreSQL system; choose Qdrant or Weaviate when self-managed control is a requirement; choose Pinecone when a dedicated managed vector service matches the team's boundary. I'm not sure which will produce the best relevance-to-index-cost ratio for your listings, and neither a route count nor a product category resolves that. The labeled evaluation does.
This comparison deliberately avoids unit-price arithmetic. Storage shape, write amplification, metadata selectivity, and deletion behavior can dominate the bill, while published prices can change before the runbook does. Measure the same corpus and change stream on the finalists, then record the configuration beside the result so the decision can be reproduced.
Verification, rollout, and rollback
Build a small labeled set before production rollout. Include common policy questions, rare amenities, two listings with nearly identical descriptions, an inactive listing, a deleted listing, and a listing whose latest revision reverses an earlier fact. For each question, label acceptable source units and forbidden stale units. Report retrieval recall at the candidate limit, stale-citation count, duplicate-candidate count, indexed units per active listing, and index writes per changed listing. Those measures connect quality to cost without inventing a single composite score.
Use shadow traffic first. Run the staged retriever beside the current path, retain query IDs and retrieved source references, and compare against the labeled set without showing its answer to users. A 429 is a throttle signal: honor Retry-After, back off exponentially, and retry an idempotent operation instead of opening a tight retry loop. Keep indexing checkpoints separate from source checkpoints so an interrupted batch resumes from a known boundary rather than declaring half a revision current.
Promotion needs an explicit gate. For example: no forbidden stale unit may appear in the labeled deletion cases, every rendered citation must resolve to the same revision retrieved by the bot, and index-write volume must stay within the capacity budget established during the shadow run. The exact recall threshold is product-specific — pretending otherwise would make the runbook look precise while hiding the real decision.
Rollback is a data operation as well as an application switch. Preserve the previous collection until the replacement passes its gates, stop writes to the candidate collection before switching readers back, and replay source changes that arrived during the rollback window. If the partitioning scheme itself is wrong, create a clean collection and re-index deliberately; don't mutate two incompatible layouts in place. After rollback, verify a known active citation, a known deletion, the oldest unapplied source change, and the count of units per listing.
Finally, write down who can delete a collection, how deletion completion is verified, and which alert pages the owner. Citation tracing is useful during an incident only if the source reference survives the very re-index or rollback being investigated.
Top comments (0)