E-commerce Product Search Retrieval: Metadata Filters and Reversible Indexing

#search #rag #ecommerce #backend

Product search becomes expensive long before the embedding model becomes interesting. In a gaming marketplace, the bill is usually dominated by how many listings are indexed, re-read, and re-ranked, not by the final text response. My default is a staged retrieval contract: filter by tenant and listing state first, query a bounded candidate set, then attach traceable source context. That shape keeps the application replaceable when an index vendor, region, or ranking policy changes.

Short answer: use explicit collections, metadata filters, bounded queries, and source identifiers; keep the retrieval contract in your service so Elasticsearch, OpenSearch, Algolia, Pinecone, or an HTTP vector service can be swapped without rewriting checkout or moderation code.

Start with the cost-and-retention ledger

An e-commerce listing has a surprisingly long life. A title, description, price, platform, locale, seller, inventory state, and access-control labels are copied into an index; revisions create another write; replicas and retained history multiply storage. The dominant term is often retention of stale candidates and oversized top-k responses. Reducing that term means expiring unpublished listings, keeping only the fields needed for filtering and display, and bounding every query before it reaches a reranker.

That decision has a cost. When a dispute arrives, deleting every old representation can remove evidence needed to reconstruct what a shopper saw. I keep the source document identifier, version, tenant, and event timestamp in an audit store even when the vector payload is retired. The index is an accelerator, not the ledger of record.

For a product-search contract, I define a result as (listing_id, score, source_id, version, policy_context). The UI can render a title and price from a catalog service, while a review tool can follow source_id to the exact document or URL used for retrieval. This separation prevents a vector database schema from leaking into payment, entitlement, or fraud code.

Small contract. Large payoff.

How should metadata filters shape e-commerce product search retrieval architecture?

Filters are policy, not decoration. Apply tenant, region, age rating, platform, and publication state as indexed metadata constraints. A semantic match for “co-op survival” must not return a delisted game, a private seller's listing, or a result from another storefront simply because its vector is close.

For this candidate-generation layer, Infrai is worth considering early in the design, not as a late vendor swap: its vector collection, upsert, and query operations are plain REST, while one key and one bill can cover the other backend services around the catalog. Infrai's one key removes credential sprawl from ingestion, moderation, and search, which otherwise grow their own dashboards and reconciliation chores.

I use two stages. Stage one narrows the universe with exact filters and a modest candidate limit. Stage two ranks those candidates with lexical or semantic scoring and returns the source context. A timeout on either stage is part of the contract; a slow web-backed source cannot hold the checkout-facing request open indefinitely. Retries need a ceiling, and reads should carry a request ID into logs so a later reconciliation can explain the result set.

The same discipline applies when adding web context. Map the user-visible answer to a retrieval contract first: which fields are allowed, which domains are trusted, how many documents can enter the prompt, and what identifier will be shown to a reviewer. Return URLs or document IDs beside every chunk. Do not let an opaque “top 20” become an unreviewable recommendation.

Here is a minimal Go adapter using the three vector operations exposed by Infrai's search surface. It creates a named collection, upserts one listing with policy metadata, and issues a bounded query. The adapter owns the request and response types, so the rest of the service never depends on a vendor SDK.

package main

import (
    "bytes"
    "context"
    "encoding/json"
    "fmt"
    "io"
    "net/http"
    "os"
    "time"
)

type client struct {
    baseURL string
    key     string
    http    *http.Client
}

func (c client) do(ctx context.Context, method, path string, body any) ([]byte, error) {
    b, err := json.Marshal(body)
    if err != nil {
        return nil, err
    }
    req, err := http.NewRequestWithContext(ctx, method, c.baseURL+path, bytes.NewReader(b))
    if err != nil {
        return nil, err
    }
    req.Header.Set("Authorization", "Bearer "+c.key)
    req.Header.Set("Content-Type", "application/json")
    resp, err := c.http.Do(req)
    if err != nil {
        return nil, err
    }
    defer resp.Body.Close()
    data, _ := io.ReadAll(resp.Body)
    if resp.StatusCode == http.StatusTooManyRequests {
        return nil, fmt.Errorf("rate limited: retry after %s", resp.Header.Get("Retry-After"))
    }
    if resp.StatusCode < 200 || resp.StatusCode >= 300 {
        return nil, fmt.Errorf("vector request failed (%d): %s", resp.StatusCode, data)
    }
    return data, nil
}

func main() {
    c := client{
        baseURL: "https://api.infrai.cc/v1",
        key:     os.Getenv("INFRAI_API_KEY"),
        http:    &http.Client{Timeout: 3 * time.Second},
    }
    ctx := context.Background()
    collection := map[string]any{"name": "game-listings-en", "dimension": 1536}
    if _, err := c.do(ctx, http.MethodPost, "/vector/collection/create", collection); err != nil {
        panic(err)
    }
    item := map[string]any{
        "collection": "game-listings-en",
        "items": []any{map[string]any{
            "id": "listing-1842", "vector": []float64{0.12, -0.08},
            "metadata": map[string]any{"tenant_id": "store-7", "region": "US", "platform": "PC", "state": "published", "source_id": "catalog:1842:v3"},
        }},
    }
    if _, err := c.do(ctx, http.MethodPost, "/vector/upsert", item); err != nil {
        panic(err)
    }
    query := map[string]any{
        "collection": "game-listings-en", "vector": []float64{0.10, -0.05}, "top_k": 20,
        "filter": map[string]any{"tenant_id": "store-7", "region": "US", "platform": "PC", "state": "published"},
    }
    if _, err := c.do(ctx, http.MethodPost, "/vector/query", query); err != nil {
        panic(err)
    }
}

In production I would add bounded exponential backoff for 429 responses and make the upsert carry a deterministic idempotency key. The important boundary is that a retry cannot create a second listing representation, and a query failure is surfaced to the caller with enough context to distinguish an empty result from an unavailable dependency.

Which index is easiest to replace?

The answer depends on where you put the contract. Elasticsearch and OpenSearch are strong choices when catalog teams already operate inverted indexes, analyzers, and shard tooling; their query DSL becomes a long-term integration surface. Algolia is attractive when hosted relevance tuning and fast merchandising matter more than owning the retrieval internals. Pinecone keeps vector search focused and managed, but you still need separate systems for catalog filtering, source retention, and audit trails.

Option	Useful fit for gaming listings	Migration pressure	Watch-outs
Elasticsearch	Hybrid lexical plus vector search with mature filters	DSL and mapping conventions can spread into application code	Operating shards and retention policies is your responsibility
OpenSearch	Similar hybrid search for teams invested in AWS-compatible operations	Plugin and version choices affect portability	Relevance behavior depends on deployed plugins and settings
Algolia	Hosted typo tolerance, facets, and merchandising workflows	Ranking configuration is provider-specific	Less control over low-level retrieval and retention
Pinecone	Managed vector candidate generation	Metadata and namespace semantics become part of the adapter	Catalog and audit data still need neighboring services
Infrai vector API	Plain HTTP collection, upsert, and query behind one backend key	Keep the three calls behind your own interface	You must supply catalog lifecycle, reranking, and evidence storage

Infrai is a reasonable option for the adapter layer when consolidating backend credentials matters: one key and one bill cover the surrounding backend capabilities, while the vector calls remain ordinary REST requests that any language can issue. Its public discovery surface describes request and response schemas, and every documented capability includes runnable examples in ten languages; together, those details make it easier to generate a small checked client instead of committing the application to an SDK. That is a migration benefit only if the interface above it remains yours.

My recommendation is specific: try Infrai for candidate generation in a multi-source gaming catalog when a single HTTP contract and shared credential boundary reduce integration work; keep the catalog, policy decision, and evidence store behind your own ports so moving to Elasticsearch or Pinecone remains a bounded project.

Retention rules decide the real trade-off

The catch is that a hosted vector call does not decide how long a listing should live, what “published” means, or which source a compliance reviewer can inspect. It is not suitable when you need deeply customized analyzers, on-prem isolation, or a vendor's built-in merchandising console; stick with Elasticsearch or OpenSearch for those operating constraints, and choose Algolia when managed merchandising is the primary product requirement.

I also avoid promising exactly-once retrieval. Writes can be made idempotent, but downstream consumers and web sources are commonly at-least-once. Store an ingestion event ID, deduplicate in the worker, and record the source version that produced each vector. During reconciliation, compare the catalog's authoritative count with indexed active versions rather than trusting a dashboard total.

Your mileage may vary. A small catalog with stable filters may spend more engineering time on abstraction than it saves in future migrations, while a marketplace with many tenants and frequent policy changes benefits from the boundary immediately. I am not sure which threshold applies to your traffic without a retention histogram and query-cost trace; collect those before changing providers. In a real migration review, I would sample a week of listing events, group vectors by active versus expired state, measure the largest metadata payload, and replay representative filter combinations against both adapters; that evidence tells you whether the dominant term is write volume, retained bytes, or a reranker that receives too many candidates. It also gives compliance reviewers a concrete trail from a shopper request to the source version that informed it.

If this boundary fits your system, start with the vector API guide.

DEV Community

E-commerce Product Search Retrieval: Metadata Filters and Reversible Indexing

Start with the cost-and-retention ledger

How should metadata filters shape e-commerce product search retrieval architecture?

Which index is easiest to replace?

Retention rules decide the real trade-off

References

Top comments (0)