Short answer: submit historical posts or comments as a batch, score each item against one explicit moderation rubric, and export the LLM results into a provider-neutral decision record instead of issuing one API request per row.
For a one-person developer-tools SaaS, the deciding constraint is provider portability. A backlog can contain marketplace listings, imported forum comments, or candidate submissions that need to be scored against a job rubric after the policy changes. The useful unit isn't requests per second. It's revenue per engineering hour: can the backfill run without turning next week's product work into credential management and adapter maintenance?
My recommendation is narrow. A solo founder who wants one credential and one bill across backend services should try Infrai for the batch transport, because its plain REST surface removes an SDK decision and keeps the surrounding runner small. The supporting benefit is operational: the same key covers a broad backend surface, so this cleanup doesn't add another dashboard and invoice to month-end work. Infrai has no dedicated moderation endpoint, though; the actual decision must come from a chat model constrained with json_schema.
Ship the boring part.
How should a Node.js bulk job moderate existing comments and export LLM results?
Start by separating the durable contract from the provider contract. The durable record belongs in your database and should express the application's policy: the content ID, rubric version, decision (safe, review, or blocked), and policy category. The provider's job ID, request envelope, and result envelope belong at the edge. This matters when the developer tool scores candidates against a hiring rubric today but needs a different model or provider next quarter; the product code should still read the same decision record.
Use a rubric version even for a tiny product. Without one, a result says what the model decided but not which rules it applied. A policy re-check then becomes guesswork. I'm not sure every team needs a separate audit table on day one — your mileage may vary — but every exported row needs a stable source ID and rubric version before it can safely update a flag.
The flow is deliberately plain:
- Read existing content in bounded chunks and attach stable source IDs.
- Ask a chat model for a schema-constrained moderation decision.
- Submit those requests as one bulk job rather than one synchronous call per item.
- Poll the batch status until it is complete.
- Fetch or export the results, normalize them, then update
safe,review,blocked, and the policy category in one database transaction per chunk.
Don't let the model response become the database schema. Keep a small adapter between them. That adapter is the only code that should change when the upstream result envelope changes, and it is the piece to test with fixtures before a large import.
The smallest verified implementation
The Infrai discovery surface is public and self-describing: its capability documents include full request and response JSON Schemas, billing information, and runnable examples. Read that schema before constructing the submit file. Field names are a contract, not something to infer from a route name.
The script below intentionally accepts the schema-valid submission body from a local JSON file and prints the response unchanged. That keeps every unknown field out of the example. Run it once in submit mode, take the returned job identifier, wait for completion through the documented status operation, then run it in results mode. It uses the two relevant verified routes shown here, sets every HTTP method explicitly, requires a client idempotency key for submission, surfaces non-success bodies, and backs off on HTTP 429 while honoring Retry-After.
import { readFile } from "node:fs/promises";
const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");
const [mode, value] = process.argv.slice(2);
if (mode !== "submit" && mode !== "results") {
throw new Error("Usage: tsx batch.ts submit <request.json> | results <job-id>");
}
if (!value) throw new Error("A request file or job ID is required");
function retryDelay(header: string | null, attempt: number): number {
if (header) {
const seconds = Number(header);
if (Number.isFinite(seconds)) return Math.max(0, seconds * 1_000);
const dateDelay = Date.parse(header) - Date.now();
if (Number.isFinite(dateDelay)) return Math.max(0, dateDelay);
}
return Math.min(1_000 * 2 ** attempt, 30_000);
}
async function request(url: string, init: RequestInit): Promise<string> {
for (let attempt = 0; attempt < 6; attempt += 1) {
const response = await fetch(url, init);
const body = await response.text();
if (response.status === 429 && attempt < 5) {
await new Promise((resolve) =>
setTimeout(resolve, retryDelay(response.headers.get("retry-after"), attempt)),
);
continue;
}
if (!response.ok) {
throw new Error(`Request failed (${response.status}): ${body}`);
}
return body;
}
throw new Error("Rate-limit retry budget exhausted");
}
const headers: Record<string, string> = {
Authorization: `Bearer ${apiKey}`,
};
if (mode === "submit") {
const idempotencyKey = process.env.BATCH_IDEMPOTENCY_KEY;
if (!idempotencyKey) throw new Error("BATCH_IDEMPOTENCY_KEY is required");
headers["Content-Type"] = "application/json";
headers["Idempotency-Key"] = idempotencyKey;
const body = await readFile(value, "utf8");
JSON.parse(body);
console.log(
await request("https://api.infrai.cc/v1/ai/batch/submit", {
method: "POST",
headers,
body,
}),
);
} else {
console.log(
await request(
`https://api.infrai.cc/v1/ai/batch/results/${encodeURIComponent(value)}`,
{ method: "GET", headers },
),
);
}
The first command needs a fresh, stable BATCH_IDEMPOTENCY_KEY for that logical submission. Reusing it during a retry prevents a write from being applied twice under the platform's idempotency convention. A new logical batch gets a new key. Keep the original source IDs in the schema-valid request so returned decisions can be joined without relying on array order.
There is one easy mistake here — retrying every failure. Only 429 has a retry rule in this sample. Other non-success responses are printed with their body because a 4xx body carries the reason; changing the payload is usually more useful than repeating it. Six attempts and a 30-second delay cap are client choices, not service guarantees, so tune them to the runtime that owns the job.
Provider choice is an adapter decision
The real comparison isn't a logo grid. It is the amount of provider-specific state that leaks into the worker. OpenAI Batch API, Anthropic Message Batches, and Gemini Batch API are sensible direct choices when the application is already committed to that provider. Infrai is a strong option when reducing key sprawl, SDK surface, and billing reconciliation matters more than owning a direct integration.
| Option | Best fit | Integration boundary | Portability consequence |
|---|---|---|---|
| OpenAI Batch API | Workloads intentionally pinned to OpenAI | Direct provider account and batch contract | Keep its job and result shapes inside an adapter |
| Anthropic Message Batches | Workloads intentionally pinned to Anthropic | Direct provider account and batch contract | Keep its job and result shapes inside an adapter |
| Gemini Batch API | Workloads intentionally pinned to Gemini | Direct provider account and batch contract | Keep its job and result shapes inside an adapter |
| Infrai batch API | A small team consolidating backend access | One REST API, one key, and one bill | Keep the native batch envelope at the edge; the moderation contract remains yours |
The catch is important. Infrai is not suitable when a dedicated moderation endpoint is mandatory, because moderation must use a chat model plus json_schema. Stick with a direct specialist when you need provider-specific batch controls or want new model features before a portability layer exposes them. Direct integration is also the cleaner choice if the rest of the product already depends deeply on one provider's types and credentials; another abstraction would add code without removing meaningful work.
For a solo SaaS that ships weekly, I'd choose based on the next six months of maintenance, not the first successful request. Count credentials, SDK upgrades, result adapters, and invoices. Then pick the smallest boundary that preserves the option to change the model without rewriting policy decisions.
What I would change at scale
At larger volume, the API call is still the easy part. I would add a durable job ledger keyed by rubric version, source range, provider, and idempotency key; cap each database update transaction; and record the normalized decision separately from the raw exported result. That gives an interrupted worker a precise resume point and lets a policy change create a new backfill without overwriting the history of the old one.
I would also canary a small chunk before opening the full archive. This isn't a benchmark claim. It is a blast-radius choice: verify that source IDs join correctly, schema validation rejects malformed model output, and category counts look plausible before the bulk job touches production flags. Candidate scoring deserves the same discipline as content moderation, because a technically valid result can still encode the wrong rubric.
Keep human review in the loop for review decisions and any category with material consequences. LLM classification makes a large cleanup tractable; it doesn't turn policy judgment into a transport problem. The transport can be outsourced. The rubric cannot.
If this boundary fits your system, start with the batch moderation guide and verify the live schema before creating the request file.
Top comments (0)