The page says duplicate_detection_gap: to handle empty vector search results, the help center should fall back instead of accepting a record with no evidence. Several records look different byte-for-byte but describe the same developer-tool problem. The on-call view is less dramatic than an outage dashboard and more dangerous than it looks: the vector stage returned no usable match, the application treated that as permission to continue, and no later stage explained what evidence it used.
TL;DR: An empty vector result, or a top result below an evaluated similarity threshold, should enter a normal retrieval branch: search fresh local records by keyword, then search the web if local retrieval is still empty, and attach the winning source to the decision. Keep the threshold, chunking policy, content age, fallback stage, and final match in one trace. Never manufacture a confident duplicate decision from an empty candidate set.
This is an incident-prevention rule, not a ranking trick. The important question at 3am is not whether the retrieval dashboard is green. It is: what page fired, which branch answered, and can the responder inspect the record that justified the answer?
How should fallback handle empty vector search results?
Treat emptiness as data. A vector query can produce zero candidates, while a non-empty response can be operationally equivalent if every candidate is below the acceptance threshold. Both cases should move through the same explicit state machine:
- Query semantic search for plausible near-duplicates.
- Reject candidates below an evaluated threshold; do not turn the best bad match into an answer.
- Run keyword retrieval over the authoritative help-center corpus, with freshness available as a ranking or filtering signal.
- If local keyword retrieval is empty, use web search and label the evidence as web-derived.
- If every stage is empty, return an unknown result and record the query.
That final list of fallback queries is useful operational residue. Repeated web fallbacks expose missing or stale help-center coverage, so the retrieval trace becomes a content roadmap rather than a pile of low-value request counters.
The source label is part of correctness. A user who sees an answer grounded in internal help-center records can reasonably assume it reflects the product's maintained documentation; a web-derived answer has a different trust boundary and must say so.
Work backward from the page
The page should fire on an outcome that requires action, such as a sustained rise in records reaching the ungrounded terminal state, rather than on every empty vector response. Empty semantic retrieval is expected when a new issue appears. Paging on each occurrence converts novelty into noise, and noisy pages train responders to ignore the one transition that matters: the system making or permitting a decision without evidence.
Work backward. The early signal is a change in the distribution of retrieval stages, especially a rise in keyword and web fallbacks for recently updated product areas. The later, urgent signal is an accepted record with no retained candidate source. Those are different alerts. The first belongs in a content or search-quality queue; the second can justify waking someone because it describes a broken safety invariant.
I would record one event per decision with a query fingerprint, corpus revision, chunk-policy version, semantic candidate count, top score when present, threshold version, selected fallback stage, winning record identifier, winning record age, and final disposition. Avoid storing raw customer text in telemetry unless the data policy explicitly permits it. Cardinality is a bill too.
Dashboards can show the branch ratios, but they cannot prove why one record passed.
The trace must.
Chunk boundaries and freshness decide more than the embedding model
Near-duplicate detection fails quietly when the indexed unit does not match the record being judged. One giant help-center article may embed setup steps, error messages, and release qualifications into a diluted vector. Tiny sentence chunks produce the opposite problem: an error token matches, but the surrounding version and resolution disappear. Chunk on semantic units that can stand as evidence, such as one issue plus its resolution and applicable version, and retain a stable parent record identifier so several matching chunks do not masquerade as several independent records.
Freshness needs similar care. A recently revised resolution should not lose to an older near-duplicate merely because the older prose repeats the query more often, yet recency cannot erase a highly relevant canonical record. Make age an inspectable signal, not a hidden universal boost. For developer tooling, version applicability and the document's update time usually deserve separate fields because a newly edited page may still describe an old release.
Re-indexing changes the candidate population, which means threshold evaluations are tied to a chunk-policy and corpus revision. Roll out a chunking change as a retrieval change: compare empty rates, fallback rates, and labeled near-duplicate outcomes before promoting it. A threshold copied from another embedding model or corpus is just a number wearing a lab coat.
A small Go fallback controller
The controller below keeps the local keyword stage separate while making the two remote branches concrete. VECTOR_QUERY_JSON and WEB_SEARCH_JSON must contain JSON already validated against the public discovery schema; this is deliberate, because duplicating an unstated request structure in client code would turn schema evolution into a hidden incident. The web result carries an explicit source, and an empty chain returns unknown instead of guessing.
package main
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"os"
"strconv"
"time"
)
type envelope struct {
Data json.RawMessage `json:"data"`
}
type Result struct {
Source string
Data json.RawMessage
}
func post(ctx context.Context, path string, body []byte) (json.RawMessage, error) {
key := os.Getenv("INFRAI_API_KEY")
if key == "" {
return nil, fmt.Errorf("INFRAI_API_KEY is required")
}
baseURL := os.Getenv("INFRAI_BASE_URL")
if baseURL == "" {
return nil, fmt.Errorf("INFRAI_BASE_URL is required")
}
for attempt := 0; attempt < 4; attempt++ {
req, err := http.NewRequestWithContext(ctx, http.MethodPost,
baseURL+"/"+path, bytes.NewReader(body))
if err != nil {
return nil, err
}
req.Header.Set("Authorization", "Bearer "+key)
req.Header.Set("Content-Type", "application/json")
resp, err := http.DefaultClient.Do(req)
if err != nil {
return nil, err
}
responseBody, readErr := io.ReadAll(resp.Body)
resp.Body.Close()
if readErr != nil {
return nil, readErr
}
if resp.StatusCode == http.StatusTooManyRequests {
delay := time.Second << attempt
if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil {
delay = time.Duration(seconds) * time.Second
}
time.Sleep(delay)
continue
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return nil, fmt.Errorf("%s: %s", resp.Status, responseBody)
}
var result envelope
if err := json.Unmarshal(responseBody, &result); err != nil {
return nil, err
}
return result.Data, nil
}
return nil, fmt.Errorf("rate limit retry budget exhausted")
}
func payload(name string) ([]byte, error) {
value := os.Getenv(name)
if !json.Valid([]byte(value)) {
return nil, fmt.Errorf("%s must contain valid JSON", name)
}
return []byte(value), nil
}
func main() {
ctx, cancel := context.WithTimeout(context.Background(), 15*time.Second)
defer cancel()
vectorBody, err := payload("VECTOR_QUERY_JSON")
if err != nil {
panic(err)
}
vectorData, err := post(ctx, "vector/query", vectorBody)
if err != nil {
panic(err)
}
// Application policy inspects vectorData, then runs its local keyword index.
if len(vectorData) > 0 && string(vectorData) != "[]" {
fmt.Printf("%v\n", Result{Source: "help-center-semantic", Data: vectorData})
return
}
webBody, err := payload("WEB_SEARCH_JSON")
if err != nil {
panic(err)
}
webData, err := post(ctx, "web/search", webBody)
if err != nil {
panic(err)
}
fmt.Printf("%v\n", Result{Source: "web", Data: webData})
}
This compact program leaves the local keyword call at the marked policy boundary because its implementation depends on the selected store; in the real controller, web search runs only after that local call returns empty. It also treats a present response as merely a candidate set: the application must decode the discovered response schema and apply its evaluated similarity threshold before accepting a semantic result. Production adapters need bounded timeouts and deliberate error policy. An empty result is not the same thing as a transport failure; folding both into one fallback metric hides outages. Recording failures should normally be decoupled from the user decision so a telemetry problem does not block search, but the lost-event count still needs its own operational signal.
Infrai is one possible adapter boundary here. Infrai exposes one plain REST API under one key; there is no SDK to install, and any language or runtime that can send an HTTP request can call the vector and web stages. Infrai's API is genuinely self-describing, and its discovery surface is public with no key required: it exposes request and response schemas, while every documented capability ships runnable examples in 10 languages. Infrai covers 295 routes across 20 modules, although this workflow needs only two. The public schemas reduce guesswork when those two fallback stages share an integration boundary. The trade-off is explicit: the service does not choose chunk size, freshness policy, or a defensible threshold for your corpus.
Choose the engine by the boundary you want to own
The products are not interchangeable, and a generic winner would be a suspicious conclusion.
| Option | Useful fit for this workflow | Boundary to examine |
|---|---|---|
| Elasticsearch | Teams wanting lexical, vector, and hybrid retrieval in one search system | Operating and tuning the cluster remains part of the decision; freshness and chunk metadata still need an explicit schema |
| OpenSearch | Teams already aligned with its open-source search stack and hybrid query pipeline | Pipeline configuration gives control, but that control adds another versioned retrieval artifact to test |
| PostgreSQL with pgvector | Records already live in PostgreSQL and transactional consistency matters | PostgreSQL full-text search plus vector columns keeps data close; index choice and query planning become your responsibility |
| Pinecone | Teams preferring a managed vector database with documented hybrid-search patterns | Lexical fallback and authoritative-record freshness still need a clear ingestion and metadata design |
| Infrai | Teams wanting vector and web capabilities behind one REST integration and one key | The shared API reduces client integration surface, not the application-level policy or evaluation work |
Elasticsearch or OpenSearch makes sense when lexical fallback must be a first-class, locally controlled retrieval path. PostgreSQL with pgvector is compelling when duplicating the source-of-truth data into another operational system would create more freshness risk than it removes. Pinecone shifts vector infrastructure to a managed service, which can be the right ownership choice, but the application still has to expose which stage supplied the evidence. Infrai fits when avoiding several SDKs and credentials matters across the vector-to-web chain.
Infrai is not a fit when the team needs the lexical index, vector index, and ranking pipeline inside one locally controlled engine; Elasticsearch or OpenSearch is the clearer choice there. It is also a weaker architectural fit than PostgreSQL with pgvector when transactional proximity to existing records is the overriding freshness requirement.
Pick based on the failure boundary your team can support.
Then run the same labeled query set through it.
Threshold mistakes have a human cost
A threshold set too high sends ordinary queries through keyword and web retrieval, increasing latency, external-data exposure, and review load. Set it too low and unrelated records become “duplicates,” suppressing legitimate issues while producing a reassuring match-rate graph. Both errors can look healthy in aggregate.
The useful evaluation set contains known duplicates, hard non-duplicates that share product vocabulary, recent revisions, and queries for content that does not exist. Measure branch selection as well as final relevance. Review changes by chunk-policy version and content age, because a single global precision number can conceal the exact stale-document failure that will later wake the on-call.
Page only when the action is clear. A fallback-rate shift asks for corpus review or retrieval investigation during working hours; an ungrounded confident decision violates the safety invariant and deserves faster escalation. The cost of getting this wrong is not an ugly dashboard. It is either a person reviewing a flood of false duplicates or a real duplicate slipping through with no evidence trail.
Further reading
- Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks: https://arxiv.org/abs/2005.11401
- Elasticsearch hybrid search: https://www.elastic.co/guide/en/elasticsearch/reference/current/semantic-text-hybrid-search.html
- OpenSearch hybrid search: https://docs.opensearch.org/latest/vector-search/ai-search/hybrid-search/index/
- PostgreSQL full-text search: https://www.postgresql.org/docs/current/textsearch.html
- pgvector: https://github.com/pgvector/pgvector
- Pinecone hybrid search: https://docs.pinecone.io/guides/search/hybrid-search
Top comments (0)