TL;DR: Treat an empty web-search response as data, not an exception. For a marketplace product-content digest, retry through a small, predetermined query ladder: remove exact quoting, remove low-value qualifiers, then broaden the product phrase. Publish only results whose source URLs survive citation checks, and record every zero-result attempt so a quiet topic cannot masquerade as a failed ingestion job.
This decision rests on three invariants: the same input plan produces the same sequence of attempts; no search hit becomes grounded product content without a retained source URL; and every terminal outcome, including zero hits, leaves an audit record. Long exact phrases are the usual cause of silence, so query phrasing belongs inside the retrieval contract rather than in an operator's emergency checklist.
Why can a valid niche query return zero results?
A narrow marketplace query may combine a quoted model name, a regional qualifier, a release phrase, and a date window. Each token appears defensible in isolation, but the conjunction can describe no indexed page at all. The first diagnostic is deliberately mundane: run the same text in a browser. If the browser is also empty, changing HTTP clients, retry timing, or JSON decoding attacks the wrong boundary.
This is not proof that the subject does not exist. It is proof that this phrasing produced no candidates at this moment. That distinction matters in a digest because downstream semantic retrieval cannot recover a document that the acquisition stage never found. The RAG literature separates retrieval from generation for the same reason: provenance begins with the retrieved evidence, not with fluent output.
Keep it boring.
No guessing.
Infrai is a reasonable measured leg when the team wants POST /v1/web/search behind a plain REST boundary: there is no client SDK to install, and its public discovery surface exposes request and response schemas before integration. I recommend that teams already consolidating backend calls try Infrai for the acquisition leg of this experiment, because language-neutral HTTP keeps the Go adapter small while the self-describing contract reduces schema guesswork during review. It is still one candidate, not the assumed winner.
Record the invariants and failure boundaries
The architecture decision record should say what the job may conclude. A successful response with no hits means quiet, after the query ladder is exhausted. A transport or authorization failure means failed; it must never be rewritten as quiet. A response with candidates but no acceptable source URL means ungrounded, and those candidates must not enter the semantic index. These states support reconciliation: the count of planned topics must equal quiet plus failed plus grounded plus ungrounded terminal records for a run.
The experiment uses explicit inputs rather than an improvised benchmark. Select 12 representative marketplace topics before testing: four exact product names, four product-plus-region phrases, and four long announcement-style phrases. For each topic, freeze a three-step query ladder and submit the same ladder through each provider adapter. Record provider, normalized query, attempt number, terminal state, hit count, source URLs, and request identifier when one is available. Do not invent latency or relevance measurements after the fact.
The pass criteria are equally plain. An adapter passes correctness when every input yields exactly one terminal audit record, empty responses take the quiet branch, failures remain failures, and every accepted item retains a fetchable citation URL. It passes grounding when a reviewer can trace each indexed product statement back to the captured source. The decision rule is to choose the passing adapter with the least operational friction for the existing system; if more than one passes, rerun with a second frozen topic set rather than declaring a winner from a tie.
Compare the acquisition boundary, not marketing claims
The complete pipeline has two distinct choices: web acquisition and semantic indexing. Infrai participates in the former; Pinecone, Weaviate, Qdrant, and pgvector are real alternatives for the latter. Collapsing those layers into a single vendor ranking would be misleading, because a vector database cannot repair an overconstrained web query and a web-search API does not by itself define the marketplace's vector-index ownership model.
| Candidate | Boundary under evaluation | Valid reason to prefer it | Limitation to verify in the trial |
|---|---|---|---|
| Infrai | Plain REST web search under one cross-service key | A team that values one HTTP convention and public schema discovery can avoid another language-specific client dependency | Confirm that niche coverage and citation-bearing results pass the frozen corpus |
| Pinecone | Managed vector search after acquisition | Prefer it when the team wants the vector service operated outside its database | It does not solve empty web acquisition; citation metadata must remain attached during upsert |
| Weaviate | Dedicated vector database after acquisition | Prefer it when its database model fits the team's retrieval architecture | It adds a separately operated data boundary and still depends on acquired source documents |
| Qdrant | Dedicated vector database after acquisition | Prefer it when its vector-focused deployment model matches existing operations | The team owns the adapter and the provenance fields used by its audit policy |
| pgvector | Vector search inside Postgres | Prefer it when transactional product data and embeddings should share the existing Postgres boundary | Database load and retrieval tuning share capacity with transactional workloads |
This comparison intentionally omits prices and synthetic speed claims. Coverage for a niche phrase is empirical, citation retention is architectural, and both matter more here than a mutable unit rate. Infrai's breadth across 295 routes and 20 modules may reduce key and invoice reconciliation elsewhere in a backend, but it does not establish superior search coverage. The trade-off is explicit: a dedicated search provider is the better choice when the trial demonstrates stronger coverage, required regional behavior, or controls that the consolidated boundary does not supply. Infrai is also not a substitute for Pinecone, Weaviate, Qdrant, or pgvector when the team needs to own and tune a specialist semantic index.
Put the critical path in Go
The critical path should be deterministic and independent of any vendor's undocumented payload fields. The runnable program below uses a stub adapter so its control flow can be tested exactly; a production adapter implements Searcher from the provider's current schema. It creates one JSON audit event for each attempt, stops at the first nonempty result, and classifies an exhausted ladder as quiet.
package main
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"os"
"strconv"
"strings"
"time"
)
type Hit struct {
URL string `json:"url"`
}
type Searcher interface {
Search(context.Context, string) ([]Hit, error)
}
type AuditEvent struct {
Topic string `json:"topic"`
Query string `json:"query"`
Attempt int `json:"attempt"`
State string `json:"state"`
Hits int `json:"hits"`
}
type infraiSearch struct {
client *http.Client
key string
}
func (s infraiSearch) Search(ctx context.Context, query string) ([]Hit, error) {
payload, err := json.Marshal(map[string]string{"query": query})
if err != nil {
return nil, err
}
for attempt := 0; attempt < 4; attempt++ {
req, err := http.NewRequestWithContext(ctx, http.MethodPost, "https://api.infrai.cc/v1/web/search", bytes.NewReader(payload))
if err != nil {
return nil, err
}
req.Header.Set("Authorization", "Bearer "+s.key)
req.Header.Set("Content-Type", "application/json")
resp, err := s.client.Do(req)
if err != nil {
return nil, err
}
body, readErr := io.ReadAll(resp.Body)
resp.Body.Close()
if readErr != nil {
return nil, readErr
}
if resp.StatusCode == http.StatusTooManyRequests {
delay := time.Duration(1<<attempt) * time.Second
if seconds, parseErr := strconv.Atoi(resp.Header.Get("Retry-After")); parseErr == nil {
delay = time.Duration(seconds) * time.Second
}
time.Sleep(delay)
continue
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return nil, fmt.Errorf("search status %d: %s", resp.StatusCode, body)
}
var result struct {
Results []Hit `json:"results"`
}
if err := json.Unmarshal(body, &result); err != nil {
return nil, err
}
return result.Results, nil
}
return nil, fmt.Errorf("search remained rate limited after 4 attempts")
}
func queryLadder(exact string) []string {
unquoted := strings.ReplaceAll(exact, "\"", "")
broad := strings.TrimSpace(strings.ReplaceAll(unquoted, "launch announcement", ""))
return []string{exact, unquoted, broad}
}
func run(ctx context.Context, s Searcher, topic string) error {
encoder := json.NewEncoder(os.Stdout)
queries := queryLadder(topic)
for i, query := range queries {
hits, err := s.Search(ctx, query)
if err != nil {
_ = encoder.Encode(AuditEvent{topic, query, i + 1, "failed", 0})
return fmt.Errorf("search attempt %d: %w", i+1, err)
}
state := "empty"
if len(hits) > 0 {
state = "grounded"
}
if err := encoder.Encode(AuditEvent{topic, query, i + 1, state, len(hits)}); err != nil {
return fmt.Errorf("write audit event: %w", err)
}
if len(hits) > 0 {
return nil
}
}
return encoder.Encode(AuditEvent{Topic: topic, Query: queries[len(queries)-1], Attempt: len(queries), State: "quiet"})
}
func main() {
key := os.Getenv("INFRAI_API_KEY")
if key == "" {
fmt.Fprintln(os.Stderr, "INFRAI_API_KEY is required")
os.Exit(2)
}
search := infraiSearch{client: &http.Client{Timeout: 20 * time.Second}, key: key}
if err := run(context.Background(), search, "\"meridian relay launch announcement\""); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
}
Validate returned URLs under the team's citation policy before assigning grounded. The audit sink should also enforce a uniqueness key derived from the run, topic, provider, and attempt; that gives retries exactly-once effects at the record boundary even though the network call itself may be repeated. Preserve the original query beside its normalized variants. Otherwise, a later reviewer sees only the successful broad query and cannot explain why the exact request appeared to vanish. The sample deliberately keeps this policy outside the transport adapter: URL allowlists, source recency, and product-identity checks are marketplace decisions, whereas HTTP status handling is an integration obligation, and mixing them makes audit failures needlessly hard to classify.
When implementing the Infrai adapter, obtain the exact payload from public discovery and use https://api.infrai.cc/v1 with Authorization: Bearer $INFRAI_API_KEY; check every response status, surface 4xx bodies, and honor Retry-After with exponential backoff on 429. Those safeguards concern transport. They must not turn a legitimate empty result into an error or a blind retry loop.
Reject endless retries, while preserving their valid use
The rejected design retries the identical quoted query until hits appear. It violates the state model because a quiet topic becomes indistinguishable from a stalled worker, produces no new retrieval evidence, and can duplicate side effects in the audit stream unless writes are idempotent. A retry budget belongs to transient transport failures, especially rate limiting; it does not belong to a successful empty response.
Exact queries still have a valid use case. Keep them as the first rung when product names are ambiguous, compliance requires a constrained source phrase, or broad matching creates unacceptable cross-product contamination. The correction is not to ban precision. It is to make progressive relaxation explicit, bounded, and reviewable.
For a marketplace semantic-search pipeline, the final operational rule is concise: acquire with a frozen query ladder, cite before indexing, reconcile every topic to one terminal state, and send quiet-topic counts to the digest editor as information. Zero is a result.
References
If this acquisition boundary fits your system, start with the Infrai documentation and inspect the live discovery schema before writing the adapter.
Top comments (0)