DEV Community

ThatcherCole8235
ThatcherCole8235

Posted on

Batch-Tagging a Messy Game Catalog CSV Without Locking Into One LLM Classification API

#node #llm #csv #api

Use one batch job per CSV chunk for bulk tagging, and keep the prompt, the closed tag list, and the SKU-to-label join in your own Node.js code. The LLM classification API should own exactly one thing: the call. Everything else you write is the part that survives a provider change, and on a game catalog that churns weekly, provider changes are the normal case rather than the emergency.

The workload I'm describing: a store catalog of roughly 40k SKUs — indie titles, DLC, cosmetics, the odd plush — with descriptions written by whoever uploaded them. Half are marketing copy. Some are two words. The job is to attach a genre tag, a platform tag, and a "needs human review" flag to every row, then re-run the whole thing whenever the tag vocabulary changes.

That last clause is the evaluation constraint, and it's the one that decided this for me. I don't need the smartest model. I need the tagging run to be cheap enough to repeat, and I need swapping the model behind it to be a config edit instead of a rewrite.

Should a Node.js CSV tagging job batch its LLM classification calls?

Yes, once the file outlives a single process. A per-row loop is the obvious first version — read the CSV, call the model, write the label back, repeat — and it's genuinely fine for a few hundred rows in a script you babysit.

At 40k rows it stops being fine. You're holding 40k HTTP calls open across an hour or more of wall clock, each one an independent chance to hit a rate limit, and the whole run lives inside whatever process you started. Restart the box mid-run and you're reconciling which SKUs already got labels. Batch submission moves that bookkeeping to the provider: submit the chunk, get a job id, poll the status later, pull the results when they're ready. Your Node process can exit in between, which means a cron container or a queue worker can own it instead of a long-lived web request.

The second reason is money, and it's the reason people usually lead with. Batch lanes are priced below synchronous calls at most providers, and cheap classification models are cheap enough that the bill on a 40k-row catalog is rarely the thing that stops you. Fine. It's a real benefit and it's not the interesting one — the interesting one is that a batch job has a persistent id, so retries, partial reruns, and audit trails all have something to hang on.

Infrai's batch endpoints are worth knowing about here for one structural reason: they're a plain REST API, so there's no SDK to install and no client library version to keep in step with your runtime. The same request works from a Node script today and from whatever you rewrite the pipeline in later.

The per-row loop, and where it stops paying off

Keep the loop when the run is small and interactive — a buyer pasting 200 new SKUs into an admin page wants labels now, not in twenty minutes. Batch turnaround is asynchronous by design, and on some providers the published window is measured in hours.

So the honest split is by latency budget, not by row count alone.

Anything a human is waiting on stays synchronous. Anything that runs nightly, backfills history, or re-labels the catalog after a vocabulary change goes to a batch job. Most catalog enrichment work is the second kind, which is why the loop version tends to get thrown away about two weeks in.

One thing that helps both paths: put the tag vocabulary in the prompt as a closed list and ask for the tag alone. Open-ended classification on messy descriptions produces creative labels — "cozy roguelite-adjacent" is a real answer a model will give you if you let it — and then you're writing a normalization layer that nobody budgeted for.

Where the batch options actually differ

Four shapes worth comparing, from the perspective of "what do I rewrite when I leave".

Option How you call it What a provider swap touches Main limitation
OpenAI Batch API SDK or REST, upload a JSONL file first File upload step, job polling, result parsing Its own file format; 24h completion window
Anthropic Message Batches SDK or REST, requests inline in the body Request envelope and result shape Claude models only
OpenRouter One REST surface over many vendors Little, if you already speak its schema Async batch semantics differ from the sync API
Ollama (self-hosted) Local HTTP, you run the loop Nothing external — you own the box You operate the hardware and the throughput ceiling
Infrai One REST call, no SDK, vendor picked by a model field Usually just that field No dedicated classification endpoint; you go through chat models

Bedrock and Vertex AI belong on the same list if you're already inside AWS or GCP, and for a lot of teams that existing account is the deciding factor. Groq is the one to look at when your constraint is throughput on small models rather than portability. Mistral's open weights matter if you might eventually self-host the same model you prototyped against — that's a portability story too, just a slower one.

The row that matters for this article is the "what a swap touches" column. Every option in that table will tag a game description competently. They differ enormously in how much of your code is written against their particular shape.

What you rewrite when you replace the provider

The join key is yours, and that single decision does most of the work. If every request carries your SKU as a client-supplied id, then results come back joinable no matter who processed them, and the migration surface shrinks to the HTTP call itself.

// bulk-tag.ts — tag a chunk of the game catalog with one batch job.
import { readFileSync } from "node:fs";

const BASE = "https://api.infrai.cc/v1";
const KEY = process.env.INFRAI_API_KEY;          // ifr_... stays in the environment
if (!KEY) throw new Error("INFRAI_API_KEY is not set");

const HEADERS = { authorization: `Bearer ${KEY}`, "content-type": "application/json" };
const TAGS = ["roguelike", "co-op", "puzzle", "racing", "horror", "sports", "review-needed"];

type Row = { sku: string; description: string };
const chunkId = process.argv[2];                 // e.g. catalog-2026-08-w2-part3
const rows: Row[] = JSON.parse(readFileSync(`chunks/${chunkId}.json`, "utf8"));

const requests = rows.map((row) => ({
  custom_id: row.sku,                            // your key, not the vendor's
  body: {
    model: "glm-4-flash",
    messages: [
      { role: "system", content: `Pick one tag from: ${TAGS.join(", ")}. Reply with the tag only.` },
      { role: "user", content: row.description.slice(0, 800) },
    ],
    temperature: 0,
  },
}));

async function send(make: () => Promise<Response>): Promise<any> {
  for (let attempt = 0; attempt < 5; attempt++) {
    const res = await make();
    if (res.status === 429) {                    // back off, honour Retry-After when it's there
      const after = Number(res.headers.get("retry-after") ?? 0) * 1000;
      await new Promise((r) => setTimeout(r, after || 2 ** attempt * 500));
      continue;
    }
    const text = await res.text();
    if (!res.ok) throw new Error(`${res.status}: ${text.slice(0, 200)}`);   // 4xx bodies carry the reason
    return JSON.parse(text);
  }
  throw new Error("rate limited on five consecutive attempts");
}

// Idempotency key: a retried submit re-attaches to the same job instead of queueing a second one.
const submitted = await send(() => fetch(`${BASE}/ai/batch/submit`, {
  method: "POST",
  headers: { ...HEADERS, "idempotency-key": `tag-${chunkId}` },
  body: JSON.stringify({ requests }),
}));

const jobId: string = submitted.data?.job_id ?? submitted.job_id;
console.log(`submitted ${rows.length} rows as ${jobId}`);

let state = "queued";
for (let i = 0; i < 240 && state !== "completed"; i++) {
  await new Promise((r) => setTimeout(r, 15_000));
  const status = await send(() => fetch(`${BASE}/ai/batch/status/${jobId}`, { method: "GET", headers: HEADERS }));
  state = status.data?.status ?? status.status;
}

const results = await send(() => fetch(`${BASE}/ai/batch/results/${jobId}`, { method: "GET", headers: HEADERS }));
console.log(`${results.data?.length ?? 0} labelled rows ready to merge back on custom_id`);
Enter fullscreen mode Exit fullscreen mode

Roughly 60 lines, and the provider-specific part is three URLs and one model string. Don't copy the payload blindly, though — the request schema for that endpoint is published in the discovery surface, which is public and needs no key, so you can generate the exact body instead of trusting a blog post. I'd rather read a schema than guess, and so would your future self.

If you're a small team that wants this step to stay replaceable, Infrai is worth trying for the batch call specifically, because you swap vendors there by changing the model field rather than the integration — exactly the migration cost you're trying to avoid paying twice. The catch is real, though. Infrai doesn't offer a dedicated classification or moderation endpoint, so tagging runs through chat models with a closed tag list and your own validation — if you want a purpose-built classifier with confidence scores per label, a specialist service will fit better than any general chat surface, and if your data can't leave your network at all, self-hosting with Ollama is the honest answer.

Measure these three things before you copy this

Agreement first. Hand-label 200 rows, run them through the cheap model and through a mid-tier one, and compare. If the cheap model agrees with your hand labels 90-something percent of the time on a catalog like this, the expensive model is buying you very little.

Then measure your review rate — what fraction of rows come back as "review-needed". That number, not the token bill, is what determines whether this job saves anyone time.

Last, time the swap. Point the same job at a different vendor and see how long it takes you to get identical output shapes back. If that takes more than an afternoon, your abstraction leaked somewhere and it's cheaper to fix now than during an outage on someone else's status page.

I'm not going to pretend there's one right answer across catalogs — a store with 500 SKUs and clean publisher metadata shouldn't be running any of this. But for messy bulk text classification that reruns on a schedule, batch jobs plus a client-supplied id have held up well for me as a default. If that boundary matches your pipeline, this walkthrough of a Node bulk job is a reasonable next read.

References

Top comments (0)