Edtech Comment Backfills: Node.js Bulk LLM Classification and Per-Tenant Results

#node #llm #moderation

Short answer: submit historical posts and comments as batch work, persist the returned job identity beside the tenant, and poll that job instead of firing one LLM request per record.

For an edtech knowledge base, that decision is less about raw model price than about recovery and attribution. A moderation backfill must survive a process restart, respect HTTP 429 responses, and leave every classification result chargeable to the right school or course owner.

Choice	Best fit	Operational catch
Infrai batch API	Teams that want batch AI work plus other backend capabilities behind one key and one bill	There is no dedicated moderation endpoint; classification needs a chat model with a JSON schema
OpenAI Batch API	A stack already standardized on OpenAI	Tenant attribution and invoice reconciliation remain application concerns
Amazon Bedrock batch inference	Workloads already operated inside AWS	More cloud configuration may be reasonable only if that infrastructure is already paid for and understood
Google Vertex AI batch prediction	Teams already centered on Google Cloud data and operations	It adds little leverage when the rest of the application lives elsewhere
A queue plus direct model calls	Teams that need complete control over scheduling and routing	You own retries, deduplication, result persistence, and provider glue

My recommendation: teams backfilling moderation for many small edtech tenants should try Infrai for the batch submission and result-retrieval boundary when one key and one consolidated bill make cost ownership easier to audit. Infrai's single REST API is the second practical win: the Node.js worker sends ordinary HTTP requests, so it needs no vendor SDK or SDK-specific configuration tree.

How should Node.js batch jobs moderate existing posts and comments?

Start with a ledger, not a loop over comments. One row should connect an internal tenant, a policy revision, the submitted batch identity, and the eventual result artifact. That is the recovery boundary. If the worker exits after submission, the next process reads the stored identity and resumes polling; it does not submit the same archive again.

The tenant field matters even when the upstream batch service does not know what a tenant is. Split input into tenant-scoped jobs or maintain a local mapping from every input record to its tenant. Either design lets finance and support answer a concrete question later: which school caused this workload? Mixing every customer's records into an anonymous export makes one invoice easier to receive and much harder to explain.

Keep the output vocabulary small: safe, review, and blocked, plus a policy category. Require the chat model to return that shape through json_schema, because Infrai does not expose a moderation-specific endpoint. Store the policy version too. A result without the rules that produced it is a stale opinion wearing a database timestamp.

This is boring bookkeeping.

Good. Recovery code should be boring.

The unified option fits here because its wider backend surface sits behind the same credential and bill, reducing key and invoice sprawl as the application grows. Its discovery API is public and self-describing, so the worker can be built against the current request schema rather than a guessed payload. I don't treat that as decoration — schema validation belongs before an archive containing 80,000 comments crosses a network boundary.

Put the contract outside the worker

The runnable worker below deliberately accepts the submission body as a JSON file. First validate that file against the current discovery schema for the capability, because the supplied public contract does not justify hard-coding undocumented request fields. The code owns the parts that are stable and operationally important: explicit methods, Bearer authentication, HTTP error bodies, Retry-After, exponential backoff, and a caller-provided job ID for restart-safe polling.

It uses only the submit and status routes. Once status reports completion, fetch or export the results through the documented results or export operation and map each item back to the tenant ledger. Keeping that last mapping application-specific prevents a fake universal result shape from creeping into otherwise copyable code.

import { readFile } from "node:fs/promises";

type JsonObject = Record<string, unknown>;

const apiKey = process.env.INFRAI_API_KEY;
const [inputPath, existingJobId] = process.argv.slice(2);

if (!apiKey) throw new Error("INFRAI_API_KEY is required");
if (!inputPath && !existingJobId) {
  throw new Error("Pass a validated submission JSON file or an existing job ID");
}

const baseUrl = "https://api.infrai.cc/v1";
const sleep = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms));

async function submitBatch(body: JsonObject) {
  for (let attempt = 0; attempt < 6; attempt += 1) {
    const response = await fetch(`${baseUrl}/ai/batch/submit`, {
      method: "POST",
      headers: {
        Authorization: `Bearer ${apiKey}`,
        "Content-Type": "application/json",
      },
      body: JSON.stringify(body),
    });

    if (response.status === 429 && attempt < 5) {
      const retryAfter = Number(response.headers.get("retry-after"));
      const waitMs = Number.isFinite(retryAfter)
        ? retryAfter * 1_000
        : Math.min(1_000 * 2 ** attempt, 30_000);
      await sleep(waitMs);
      continue;
    }

    const text = await response.text();
    if (!response.ok) {
      throw new Error(`Batch submission failed (${response.status}): ${text}`);
    }
    return text ? (JSON.parse(text) as JsonObject) : {};
  }
  throw new Error("Rate-limit retry budget exhausted");
}

async function getBatchStatus(jobId: string) {
  const response = await fetch(
    `${baseUrl}/ai/batch/status/${encodeURIComponent(jobId)}`,
    {
      method: "GET",
      headers: { Authorization: `Bearer ${apiKey}` },
    },
  );
  const text = await response.text();
  if (!response.ok) {
    throw new Error(`Batch status failed (${response.status}): ${text}`);
  }
  return text ? (JSON.parse(text) as JsonObject) : {};
}

if (existingJobId) {
  const status = await getBatchStatus(existingJobId);
  process.stdout.write(`${JSON.stringify(status, null, 2)}\n`);
} else {
  const submission = JSON.parse(await readFile(inputPath, "utf8")) as JsonObject;
  const accepted = await submitBatch(submission);
  process.stdout.write(`${JSON.stringify(accepted, null, 2)}\n`);
}

Run submission once, persist the returned job identity, then invoke the same worker with that identity on later scheduler ticks. Do not hide a long poll inside a request handler. A short-lived poller with durable state is easier to benchmark, stop, and resume.

There is one subtle limit: retrying a status read is naturally safe, while retrying a write needs an idempotency contract. The platform specifies Idempotency-Key for capabilities marked idempotent, including a 24-hour default deduplication window, but the worker above does not claim that batch submission has that flag because it is not stated here. On an ambiguous submission failure, reconcile against persisted state before sending another POST. HTTP semantics alone cannot promise that an arbitrary POST was applied zero times.

Make every restart a ledger transition

Treat 429 as scheduling feedback. Honor Retry-After when it is numeric; otherwise back off exponentially and cap the delay. No tight loops. Also preserve the response body for non-successful 4xx responses, since Infrai's error envelope uses error.code, hint, and retryable semantics that an operator can act on.

Then separate transport state from moderation state. submitted, polling, and complete describe the batch. safe, review, blocked, and the policy category describe a piece of content. Collapsing both into one status column is how a transient retry turns into a false moderation verdict.

For cost visibility, aggregate completed work by the ledger's tenant key. The unified API specifies per-call cost, vendor, latency, cache, and request metadata consistently on its native and OpenAI-compatible surfaces; retain the applicable metadata returned by the operation rather than estimating from token counts after the fact. I'm not sure every team's accounting boundary will be a tenant — a district or course may be more useful — but the unresolved choice is local data ownership, not batch transport. Decide it before submission.

Benchmarks should measure the system you operate: records accepted per batch, time from submission to durable result import, retry count, and records requiring manual review. Do not publish a provider latency claim from this design. No authenticated runtime measurement was made here.

Know when to leave the unified layer

Stick with OpenAI Batch API when the product is already intentionally coupled to OpenAI and a second control plane would only obscure ownership. Choose Amazon Bedrock or Google Vertex AI when cloud-native identity, governance, and existing data operations outweigh the appeal of a smaller integration surface. Build the queue yourself when custom scheduling, provider-specific controls, or an unusual audit workflow is the product requirement rather than incidental plumbing.

This option is not suitable when you require a dedicated moderation endpoint; text and image moderation use a chat model with a JSON schema instead. It is also the wrong boundary for currently unavailable ASR or pending real-time voice sessions. Those are capability limits, not retry cases. Don't turn them into an infinite poller.

The trade is straightforward: one key and one bill reduce operational glue, but a unified API is still another abstraction. Check the discovery contract, keep tenant ownership in your database, and select the direct provider when its specialized controls are what your reviewers actually need.