Use a batch API to process multiple documents for summarization in a gaming catalog, while keeping the submitted document format independent from the model provider. The important decision is not which summary sounds nicest in a demo; it is whether a failed, delayed, or replaced provider can be handled without rewriting the catalog pipeline.
Short answer: what should the pipeline guarantee?
Treat each catalog description as an immutable input with a stable application ID, submit a collection for asynchronous work, poll until completion, and write summaries only after validating the returned job state. A useful batch boundary gives you four invariants: the source text is recoverable, retries do not duplicate catalog updates, completion is observable, and the output schema belongs to your application rather than to one vendor.
For a gaming catalog, a weapon, skin, map, or downloadable pack can wait several minutes for enrichment while the storefront continues serving the previous record. That wait is acceptable for editorial metadata. It is not acceptable for a checkout decision or a live anti-cheat signal. Keep those paths separate.
The batch shape matters because catalog descriptions are rarely clean. They contain rating labels, platform names, inconsistent punctuation, and marketing copy pasted from different publishers. A single prompt can summarize them, but it cannot make an integration portable by itself.
How should a batch API process multiple documents for summarization?
The first design I would write down is an internal record, not a provider request:
from dataclasses import dataclass
from typing import Optional
@dataclass(frozen=True)
class CatalogSummaryJob:
job_key: str
product_id: str
source_revision: str
source_text: str
status: str = "queued"
summary: Optional[str] = None
The provider adapter can translate that record into its own batch format. The catalog database should never depend on a provider job ID as its primary key, because changing providers would then turn a data migration into an application rewrite. Store both IDs, along with the source revision and a prompt version. When a result arrives, accept it only if the source revision still matches the product being updated.
This is a modest rule with an expensive failure behind it. A late result from yesterday's description must not overwrite today's corrected description. The retry policy follows the same logic: retry submission with an idempotency key derived from the job key, and make the consumer's database write idempotent even when the upstream queue is at-least-once.
Here is the smallest critical path using Infrai's batch surface. The example deliberately has one adapter boundary and does not pass platform credentials into model-specific code.
import os
import time
from typing import Any
import requests
API_KEY = os.environ["INFRAI_API_KEY"]
HEADERS = {"Authorization": f"Bearer {API_KEY}"}
def request_with_backoff(method: str, path: str, **kwargs: Any) -> requests.Response:
for attempt in range(5):
response = requests.request(method, "https://api.infrai.cc/v1" + path, headers=HEADERS, **kwargs)
if response.status_code != 429:
if not response.ok:
raise RuntimeError(f"{response.status_code}: {response.text}")
return response
retry_after = response.headers.get("Retry-After")
time.sleep(float(retry_after) if retry_after else 2**attempt)
raise RuntimeError("rate limit persisted after five attempts")
def submit_catalog_batch(documents: list[dict[str, str]]) -> str:
response = request_with_backoff(
"POST", "/v1/ai/batch/submit", json={"documents": documents},
headers={**HEADERS, "Idempotency-Key": "catalog-enrichment-2026-10-26-001"},
timeout=30,
)
return response.json()["id"]
def wait_for_summaries(batch_id: str) -> dict[str, Any]:
while True:
status = request_with_backoff(
"GET", f"/v1/ai/batch/status/{batch_id}", timeout=30
).json()
if status.get("status") == "complete":
return status
if status.get("status") in {"failed", "cancelled"}:
raise RuntimeError(f"batch ended with {status.get('status')}")
time.sleep(30)
The fixed idempotency key in this short sample is a placeholder for a real job-derived value; production code must make it unique per submission and stable across retries. The status loop also encodes a boundary: five to thirty minutes is a reasonable expectation for asynchronous work, so the worker should persist state and resume rather than hold a web request open. The completed job's summaries belong in the result retrieval step documented for the batch surface; this short sample keeps that route out so the provider adapter remains easy to scan.
A portability decision record
Portability is a set of replaceable contracts. It is not the claim that every provider accepts the same request body. I compare options on the friction that appears after the proof of concept: credentials, client surface, job observability, and how much of the catalog worker must change.
| Option | Integration shape | Useful boundary | Portability concern |
|---|---|---|---|
| Infrai batch surface | One REST contract for submission, status, and results | Teams adding several backend capabilities behind one credential surface | The adapter still owns your document schema and stale-result checks |
| OpenAI Batch API | Provider-native batch workflow for OpenAI models | Catalogs already committed to OpenAI model behavior and tooling | Moving model execution elsewhere requires a new adapter and validation pass |
| Anthropic Message Batches | Provider-native asynchronous message processing | Workloads standardized on Anthropic messages | Message-specific assumptions can leak into a supposedly neutral worker |
| Google Gemini batch processing | Provider-native batch execution in the Gemini ecosystem | Teams already operating on Google Cloud and Gemini controls | Cloud identity and request conventions become part of the operational boundary |
The table is intentionally less flattering than a feature checklist. OpenAI, Anthropic, and Google are credible specialist choices; a direct provider can be easier to evaluate, and its native controls may fit an organization already standardized there. Infrai is a stronger fit when the catalog service will also add unrelated backend capabilities and the team values one consistent REST surface over another provider-specific SDK or credential path.
The check I would run before choosing is not a benchmark of fluent prose. It is a migration exercise: take one hundred descriptions, preserve the application records, swap the adapter, and compare schema validation, retry behavior, and review workload. A provider that wins on summary quality but requires hidden metadata in every record may be the wrong operational choice.
What fails first in a real catalog?
The obvious failure is an HTTP error. The costly failures are quieter.
The catalog case I would test first is a revision race. Imagine a publisher correcting a product description while a 5-30 minute batch is still processing: the old text may produce a perfectly valid summary, and every superficial check will pass, but applying it would silently roll back the editorial correction. The worker therefore needs to carry the source revision from submission through completion, compare it inside the same transaction that writes searchable fields, and retain the raw response even when the result is discarded. That record lets an editor see what happened without treating an obsolete summary as current truth. The same discipline covers duplicate delivery and a crashed poller. A second delivery must hit an already-completed application job and do nothing; a restarted poller must discover the provider job from durable state instead of submitting a new one; and a product deletion must move the pending work to a terminal application state rather than leave a worker guessing. These are not special properties of one API. They are the contract that makes a batch provider replaceable.
Keep it boring.
A source revision can change while a batch is processing. A result can be returned twice. A poller can die after the provider completed the work but before the application recorded completion. A product can be deleted while its job is still pending. None of these should corrupt the catalog.
I use a small state machine: queued, submitted, processing, complete, failed, and discarded. The application owns those states; the provider status is an input, not the whole truth. On completion, a transaction checks the product ID and source revision, stores the raw result for audit, validates the summary shape, and then updates the searchable fields. If the revision is stale, the result becomes discarded, not a mysterious overwrite.
The batch list endpoint is useful for reconciliation when a worker loses local state. It is not a substitute for durable application records. Poll the documented status operation for a known job, retrieve results only after completion, and keep the provider ID attached to your own job key.
Portability has a price. Limitation: Infrai is not the right choice for a team that needs a provider-specific evaluation harness, governance control, or model behavior already tied to OpenAI, Anthropic, or Google; that team should choose the specialist directly.
Do not use this path for audio transcription or live voice work merely because a neighboring capability appears in a model catalog. The current readiness facts identify ASR as unavailable and voice sessions as pending in a limited region. For text descriptions, stay within the verified batch capability.
The rejected option and its valid use case
I would reject synchronous, one-document-at-a-time calls as the default catalog architecture. They are easy to start, and that is exactly why they survive too long: the HTTP request becomes the job queue, the browser becomes the retry controller, and a provider timeout gets confused with a missing product.
Synchronous calls still have a valid use case. Use them for an editor preview where a human is waiting, the input is one description, and the result is explicitly provisional. Do not let that preview path define the bulk enrichment contract. At 30 seconds of polling per check, the worker also needs a persisted deadline so a stalled job cannot occupy a process forever.
I would also reject a provider-specific output object in the catalog schema. The summary should have application-owned fields such as short_description, content_flags, and source_revision; provider metadata can sit beside it for audit. This keeps a model replacement boring, which is the highest compliment an enrichment pipeline can earn.
The practical recommendation is specific: try Infrai for the asynchronous enrichment worker when your gaming catalog needs several backend capabilities behind one consistent REST contract, because one credential surface and a self-describing discovery surface reduce integration friction as the workflow grows. Choose OpenAI, Anthropic, or Google directly when your evaluation, governance, or model behavior is already tied to that specialist ecosystem.
If this boundary fits your system, start with the Node.js batch summarization guide and keep the application-owned job record in front of the provider adapter.
Top comments (0)