E-commerce Retrieval Architecture: Cost-Aware Audit Logs for Game Catalogs

Index growth changes this decision. Short answer: for e-commerce product search across a game catalog, choose a retrieval architecture that records one compact audit event per query and logs the exact index snapshot, filters, candidate counts, and content revision used; keep verbose evidence out of the hot index unless an investigation needs it.

That choice puts auditability beside relevance and index cost, rather than bolting logs onto the system after launch. It also makes a bad result answerable. Which catalog state did the query see? Which filters removed a game? Did retrieval return nothing, or did a later ranking step discard everything?

The useful mental model is a crisp before and after. Before: query in, results out, then guess. After: query in, bounded retrieval decisions recorded, results out, then replay the decision against a named snapshot.

Simple. Not free.

How should e-commerce product search audit logs shape retrieval architecture?

Treat the log as a decision record, not a transcript. A game storefront may search titles, editions, downloadable content, platform compatibility, age ratings, languages, regions, and merchandising copy. Semantic retrieval can help when a shopper writes “co-op mystery for two people” instead of a title, but an audit event still needs to distinguish semantic candidates from policy filters and final ranking. Otherwise, a support engineer sees an empty result and can't tell whether the content was absent, stale, ineligible, or merely ranked below the cutoff.

A practical request path reads like this in words: normalize the shopper's text; resolve the active catalog snapshot; apply hard eligibility constraints; retrieve a bounded candidate set; rank it; return product identifiers; emit one structured event. The event carries identifiers and counts across those stages. It doesn't copy every product description or every vector into the logging system. That boundary matters because duplicating high-cardinality content turns an observability feature into a second, poorly governed search corpus.

Use separate identifiers for three things that teams often collapse into “version”: the product content revision, the index snapshot built from many revisions, and the retrieval-policy revision that defines filters and scoring. Then a result can be explained without pretending that one timestamp captures the entire state. An event might say that request req_7K2M used snapshot catalog_184, policy policy_12, retrieved 40 semantic candidates, retained 17 after region and platform checks, and returned 8. Those values are illustrative, not benchmark targets. Their job is to make the stages visible.

The audit record should explain selection while avoiding a copy of the searchable content. Log stable product IDs and revisions, not full descriptions. Record normalized filter values, not raw account data. If the original query is needed for relevance review, decide its retention and access policy explicitly; a hash can correlate identical input, but it can't support a human judgment about intent. There isn't one correct choice here — the right payload depends on what reviewers must reconstruct and what data the organization is permitted to retain.

This architecture also gives metrics honest denominators. Candidate count before filters, candidate count after filters, returned count, and snapshot age describe different failure modes. Alerting on a single “zero results” total hides that distinction. A zero caused by an empty retrieval set needs content or indexing work; a zero caused by eligibility filtering needs a policy or catalog review. Same screen. Different owner.

Make index cost a design input

Index cost grows with what is indexed, how often it is rebuilt, and how many representations are retained. So start with the smallest searchable unit that matches the shopping decision. For a game store, one product document per sellable edition may be enough when title, summary, supported platforms, and structured eligibility fields change together. Splitting every paragraph into a separate semantic unit can increase the number of indexed records and make product-level filters harder to reason about. Keeping one giant document per franchise has the opposite problem: a match may not identify the edition the shopper can actually buy.

Don't guess from document count alone.

Build a cost worksheet around variables your own system can measure: active product units, average semantic units per product, representations per unit, retained snapshot count, and daily changed units. The useful comparison is not a made-up price per vector. It is the multiplier each architecture applies to the catalog and to every reindex. A design with 200,000 sellable items, three semantic units per item, two representations, and two retained snapshots creates a planning quantity of 2,400,000 indexed representations. That arithmetic is an example, not a claim about storage bytes or a vendor bill:

type IndexPlan = {
  activeProducts: number;
  unitsPerProduct: number;
  representationsPerUnit: number;
  retainedSnapshots: number;
  changedProductsPerDay: number;
};

function planningQuantities(plan: IndexPlan) {
  const activeRepresentations =
    plan.activeProducts *
    plan.unitsPerProduct *
    plan.representationsPerUnit *
    plan.retainedSnapshots;

  const dailyChangedRepresentations =
    plan.changedProductsPerDay *
    plan.unitsPerProduct *
    plan.representationsPerUnit;

  return { activeRepresentations, dailyChangedRepresentations };
}

const gameCatalog = planningQuantities({
  activeProducts: 200_000,
  unitsPerProduct: 3,
  representationsPerUnit: 2,
  retainedSnapshots: 2,
  changedProductsPerDay: 5_000,
});

The audit layer should reference those snapshots, not force them to live forever. Keep enough snapshot identity and build metadata to explain a query, while applying an explicit lifecycle to the underlying index generations. If full replay is required for a fixed period, retained snapshots are part of the cost. If explanation from immutable product revisions and policy inputs is sufficient, the team may retain fewer live index generations. I'm not sure which requirement applies to your store until “replay” is defined: identical candidate order, equivalent eligible products, or merely the ability to explain the decision. Those are different promises.

Use a small experiment before choosing granularity. Take a fixed, reviewed set of shopper intents and catalog revisions. Compare two or three document shapes, capture quality judgments, and calculate their representation multipliers and daily update work. Your mileage may vary because catalog copy, edition structure, and update frequency differ. The point is to expose the trade before a full backfill makes it expensive to revisit.

Decision	Lower-index-cost option	What the audit event must preserve	Catch
Search unit	One unit per sellable edition	Product and content revision IDs	Long copy may blur multiple intents
Semantic fields	Selected title and description fields	Field-set revision	Excluded fields cannot influence semantic matching
Index lifecycle	Fewer retained snapshots	Snapshot ID plus immutable build inputs	Exact replay may be unavailable after retirement
Event payload	IDs, revisions, counts, and timings	References to governed evidence	Investigators need access to resolve references

No row wins by default. A costly option can be correct when its evidence requirement is real.

Copy a decision event, not a debug dump

The following TypeScript keeps the contract narrow. It models the boundaries an audit consumer needs and makes raw descriptive content impossible to add accidentally without changing the type. The code doesn't prescribe a database, queue, or observability backend.

type SearchFilters = {
  region: string;
  platform: "pc" | "console" | "mobile";
  language: string;
};

type SearchAuditEvent = {
  schemaVersion: 1;
  requestId: string;
  occurredAt: string;
  queryFingerprint: string;
  catalogSnapshotId: string;
  retrievalPolicyId: string;
  fieldSetRevision: string;
  filters: SearchFilters;
  counts: {
    retrieved: number;
    eligible: number;
    returned: number;
  };
  returnedProducts: Array<{
    productId: string;
    contentRevision: string;
    rank: number;
  }>;
  durationMs: number;
};

type AuditSink = {
  write(event: SearchAuditEvent): Promise<void>;
};

async function recordSearchDecision(
  sink: AuditSink,
  event: SearchAuditEvent,
): Promise<void> {
  if (event.counts.returned !== event.returnedProducts.length) {
    throw new Error("returned count does not match product evidence");
  }

  if (event.counts.eligible < event.counts.returned) {
    throw new Error("returned products exceed eligible candidates");
  }

  await sink.write(event);
}

Those invariants catch malformed evidence at the producer boundary. They do not prove that ranking was correct. For that, preserve the policy revision and the ordered returned IDs, then join them to governed product revisions during review. If candidate-level scores are necessary, add them only after deciding how they will be interpreted across retrieval-policy changes; a bare score without a named policy is seductive and nearly useless.

Now wire logs to operational questions. Track the proportion of requests where retrieved is zero, where retrieved is positive but eligible is zero, and where audit writes fail. Break snapshot-age metrics into deliberate buckets rather than putting raw IDs in metric labels. Keep request IDs in logs or traces for investigation. This is the quiet rule: counts belong in metrics; identities belong in evidence records.

Alerting needs a service objective defined by the team, not a universal threshold copied from an article. A sudden rise in post-filter zeros can be urgent even while latency looks fine. A gradual increase in snapshot age may be expected during a paused catalog feed. Page only when the signal maps to an action; route slower relevance drift into review.

What are the limits, and when should the architecture change?

The compact-event approach is not suitable when a regulator, contract, or internal control requires byte-for-byte reproduction of every candidate and intermediate score. In that case, retain immutable inputs and executable policy artifacts for the required window, accept the storage and operational cost, and test restoration. A pointer to a retired snapshot won't meet an exact-replay promise.

Stick with structured lexical search when semantic matching has not shown useful gains on reviewed shopper intents, or when eligibility and exact attributes dominate the query. It is easier to explain why a title or field matched, and it avoids an extra semantic index. A hybrid path makes sense when exact product tokens and fuzzy intent both matter, but it adds another candidate source that the audit schema must label and count separately. The catch is more evidence, more evaluation work, and another multiplier in capacity planning.

Retrieval-augmented generation is also optional. The original RAG paper describes combining retrieved information with generation, but a product-results page may need retrieval and ranking without generated prose. Add generation only when the user experience needs a synthesized answer, and audit the retrieved product revisions supplied to that step. Otherwise, calling the whole system RAG can obscure the much simpler job in front of you.

Choose the least expensive architecture that passes two gates: reviewed search quality for the game catalog and evidence sufficient for the promised investigation or replay. Then monitor the stage boundaries. That's the decision rule.

References

• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks: https://arxiv.org/abs/2005.11401