DEV Community

HumphreyFox1243
HumphreyFox1243

Posted on

How to Build a GPT Summarization Workflow: Combining Batch Processing for Catalog Quality

#ai #catalogenrichment #batchprocessing

An AI summarization workflow combining batch processing with a GPT API is the right default for a B2B SaaS catalog when thousands of descriptions can wait for a later result. The catch is that batch processing improves throughput without deciding what “good” means. Put the quality gate in your application, keep interactive edits on a synchronous path, and use a provider adapter that records every item-level outcome.

The timing decision comes first.

Which option fits the failure you are trying to avoid?

Start with the operational failure, not the model brand. The table below is the short field guide.

Option Pick this when Quality versus latency trade-off Boundary to own
OpenAI Batch API Your prompts and evaluation set already live in the OpenAI ecosystem A familiar request shape helps iteration; completion is deferred Input files, output files, and per-request failures
Anthropic Message Batches Claude behavior is the reason your summaries pass review A provider-specific batch can suit quality experiments; it is not an interactive path Provider adapter and result reconciliation
Gemini batch predictions Your platform already operates in Google Cloud or uses Gemini formats Offline capacity can fit large catalog backfills; cloud configuration adds coupling Project, region, and prediction-job details
A single REST surface across capabilities The catalog pipeline will grow into several backend capabilities One contract can shorten integration work; model selection still controls summary quality and wait time Capability readiness, schema validation, and publishing policy

OpenAI, Anthropic, and Google all expose real batch primitives, but they do not erase the differences around files, authentication, retries, and result retrieval. That is the portability cost. A wrapper can hide syntax; it cannot hide a different failure model.

For a team adding catalog enrichment beside other backend work, a consistent REST surface is the interesting fourth option. Infrai is relevant here because its discovery surface describes 295 routes across 20 modules under one key, with runnable examples in 10 languages. That breadth behind a simple contract can make adding a capability one more HTTP integration instead of another SDK project. It does not establish that any one model will produce the best catalog summary. Test that separately.

How should a GPT summarization workflow combine batch processing?

Make the publishing decision explicit before submitting a batch. A useful flow is:

supplier description -> batch summary -> schema check -> fact check -> publish or review
Enter fullscreen mode Exit fullscreen mode

The summary should preserve stated specifications that affect a purchase: dimensions, materials, compatibility, quantities, and conditions. It should not fill gaps with plausible-sounding product facts. Keep the original description in the catalog database; the batch input and output are reproducible job artifacts, not the system of record. The queue is not the quality gate.

I would create an evaluation set with abbreviations, missing units, contradictory supplier claims, and the languages the catalog accepts. Then compare prompts and models against the same rows. For example, a supplier may write “fits 13/15 laptops” in one description and “universal” in another field; the validator should preserve the explicit claim and flag the conflict instead of choosing the more convenient phrase. A missing unit is another review case, not an invitation to infer one. A short result that drops a warranty condition has failed, even if its response arrived first.

That decision also separates two latency classes. A seller waiting for a preview needs a synchronous request. A nightly backfill, a migration, or a reprocessing run after a prompt change is a batch job. Do not put both behind one queue and expect a priority flag to repair the user experience.

The queue is only the transport.

How do you submit a batch without losing its audit trail?

Use one stable identifier per source row and one stable identifier for the logical job. The identifier is more important than the polling loop: after a retry or a worker restart, it lets your database distinguish “same work again” from “new work.”

The following Node.js 20+ TypeScript example keeps the HTTP boundary small. It submits catalog records, polls the batch status every five seconds, writes a status report, retries 429 responses with Retry-After when present, and fails loudly on other HTTP errors. The request uses the documented v1 batch paths and an idempotency key supplied by the client. The service's default idempotency deduplication window is 24 hours, so keep the job ID stable across worker restarts inside that window.

import { writeFile } from "node:fs/promises";

const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");

const baseUrl = process.env.INFRAI_BASE_URL;
if (!baseUrl) throw new Error("INFRAI_BASE_URL must point to the service v1 base URL");
const headers = {
  Authorization: `Bearer ${apiKey}`,
  "Content-Type": "application/json",
};

async function request(
  url: string,
  init: RequestInit,
  retries = 5,
): Promise<Response> {
  for (let attempt = 0; attempt <= retries; attempt += 1) {
    const response = await fetch(url, {
      ...init,
      headers: { ...headers, ...init.headers },
    });

    if (response.status !== 429 || attempt === retries) return response;

    const retryAfter = Number(response.headers.get("retry-after"));
    const delayMs = Number.isFinite(retryAfter) && retryAfter > 0
      ? retryAfter * 1000
      : 2 ** attempt * 1000;
    await new Promise((resolve) => setTimeout(resolve, delayMs));
  }

  throw new Error("Retry loop ended unexpectedly");
}

async function json(response: Response): Promise<Record<string, unknown>> {
  const body = await response.text();
  if (!response.ok) throw new Error(`HTTP ${response.status}: ${body}`);
  return JSON.parse(body) as Record<string, unknown>;
}

const records = [
  { id: "sku-1042", text: "Steel bottle, 750ml, vacuum insulated, keeps drinks cold." },
  { id: "sku-1043", text: "Notebook with recycled cover. 120 pages. Blue." },
];

const jobId = `catalog-${Date.now()}`;
const submitUrl = `${baseUrl}/ai/batch/submit`;
const submitted = await json(await request(submitUrl, {
  method: "POST",
  headers: { "Idempotency-Key": jobId },
  body: JSON.stringify({
    job_id: jobId,
    items: records,
    instruction: "Summarize each catalog description in one sentence. Preserve stated specifications and return JSON.",
  }),
}));

const batchId = String(submitted.id);
let status = submitted;

while (!["completed", "failed", "cancelled"].includes(String(status.status))) {
  await new Promise((resolve) => setTimeout(resolve, 5000));
  const statusUrl = `${baseUrl}/ai/batch/status/${batchId}`;
  status = await json(await request(statusUrl, {
    method: "GET",
  }));
}

const report = {
  batch_id: batchId,
  status: status.status,
  failed_requests: status.failed_requests ?? 0,
  result_count: status.result_count ?? 0,
};

await writeFile("catalog-batch-status.json", JSON.stringify(report, null, 2));

if (status.status !== "completed" || Number(report.failed_requests) > 0) {
  throw new Error(`Batch did not complete cleanly: ${JSON.stringify(report)}`);
}

console.log(JSON.stringify(report));
Enter fullscreen mode Exit fullscreen mode

The code makes three judgments visible. A repeated submission carries the same logical job ID. A 429 waits instead of tight-looping. A completed transport job still fails the process when the service reports failed requests. That last check matters: “completed” can describe a batch lifecycle while individual catalog rows still need review.

In production, add an item-level validator between the result reader and the product table. Check that each source ID appears once, that the summary matches the expected JSON shape, and that every claimed specification is present in the source text. Route rejected rows to review. Batch processing handles volume; it does not grant permission to publish.

Where does the unified surface help, and where does it stop?

The practical advantage of a broad REST surface is the contract. A team can inspect a public discovery document, see whether a capability is available, and use the same HTTP style as adjacent backend work. The discovery metadata includes readiness fields such as available, vendors_ready, vendors_pending, and key_status; use them as a preflight check rather than assuming that a listed capability is ready in every region.

That boundary is useful for this catalog workflow. It is not a promise of uniform readiness across unrelated modalities. The current facts explicitly describe transcription as unavailable, voice sessions as pending in the western region, and no dedicated moderation endpoint; text or image moderation therefore needs a chat model with a JSON schema fallback. Those are separate decisions from catalog summarization, but they are exactly why capability discovery belongs in deployment checks.

Limits and fit

Batch is a latency trade. It suits backfills and deferred enrichment. It is the wrong interaction model for an editor waiting on a preview. Provider adapters also remain real maintenance: OpenAI, Anthropic, and Gemini differ in job setup and result handling, even when the business request sounds identical.

The narrow rule is durable: evaluate summaries on representative messy descriptions, batch work that can wait, validate every completed result, and keep a synchronous escape hatch for human-facing edits. Choose the unified REST option when reducing integration boundaries matters; choose a provider-native batch when its model behavior is the reason the catalog passes review.

Sources

Top comments (0)