Moderation Intake Accounting: Bulk LLM Text Classification API With Tenant Chargeback

Short answer: For cheap bulk CSV tagging, use an asynchronous LLM text classification API, estimate each tenant batch before it runs, and attach the eventual export to the same tenant ledger instead of sending one request per row.

For a one-person B2B SaaS, the useful comparison is not a model leaderboard. It is the amount of accounting and integration work left in the product after classification finishes.

Option	Choose it when	Tenant-cost consequence	Catch
Infrai	You want a self-describing REST API whose public discovery supplies request and response schemas plus runnable examples	One key and one bill make the external side of reconciliation smaller	Moderation uses chat classification with JSON Schema because there is no dedicated moderation endpoint
OpenAI direct	Your product has already standardized on OpenAI	Keep tenant attribution in your own job ledger	A direct contract does not remove application-level CSV reconciliation
Anthropic direct	Your model decision is already Anthropic-specific	Use the same internal ledger pattern	You own the provider-specific adapter and export mapping
Google Gemini direct	Your model decision is already Gemini-specific	Use the same internal ledger pattern	You own the provider-specific adapter and export mapping

Recommendation: use asynchronous chat classification with a closed label set, but treat the tenant ledger as the primary artifact and the provider batch as an execution detail. That keeps a nightly backfill away from the request path and makes every charge explainable before a human moderator sees the result.

The model matters. The accounting boundary matters more.

Start with the allocation unit, not the provider

A moderation upload arrives as a CSV, but a CSV is a transport format, not a billing unit. The billing unit should be an immutable application job owned by one tenant. Give that job an internal ID, record the source-file identity, preserve the row identifiers, and bind the approved label vocabulary to it. Then estimate the job before submission. A non-expert operator can approve the whole file or sample it first without having to understand token pricing.

This changes the product conversation. Instead of asking, "What did AI cost this month?" you can answer, "Which moderation imports created the spend, for which tenant, under which prompt version?" That is the level needed for support, plan limits, and margin review. It also fits a revenue-per-hour test: tenant accounting differentiates the SaaS; building another generic batch executor does not. Outsource the undifferentiated part and ship the moderation workflow weekly.

Use a closed list such as spam, harassment, self_harm, fraud, and other, then require JSON matching that enum. Otherwise near-synonyms become separate report buckets. A reviewer may understand that abuse and harassment overlap, but an export and a usage ledger won't infer that safely.

There is no magic sample size. I'm not sure a universal threshold would even be honest, because report length and prompt size change the estimate. Put the estimate in front of the operator and let the product's own plan policy decide when approval is required.

How should a bulk CSV LLM classification batch allocate tenant costs?

Allocate at the job boundary first, then reconcile at the row boundary. Before submission, the job owns the estimate. After results arrive, the same job owns actual cost metadata when supplied, while each result is matched back to an expected source row. Never try to reconstruct ownership later from a provider invoice or from whichever user happened to start a worker.

The sequence is deliberately plain:

1. Parse and validate the uploaded CSV inside the application.
2. Create one tenant-owned job and a stable ID for every accepted report row.
3. Build a batch request using the current discovered schema and the fixed moderation labels.
4. Show the estimate, then submit only after the product's approval rule passes.
5. Store the returned batch identifier beside the internal job.
6. Check state from a worker, fetch results when ready, and reconcile every returned row before export.

That separation handles an awkward but common case. Imagine tenant A uploads 18,000 reports, tenant B uploads 240, and the larger job is still running when the smaller one completes. A global "AI usage" counter can tell you that work happened; it cannot explain ownership, partial completion, rejected source rows, or which customer export is ready. Two tenant-owned records can. They also let the UI report progress without keeping the upload request open, and they keep retries from quietly moving work between billing periods or customer accounts.

Do not allocate by successful label count. Invalid input, model output that fails schema validation, and a report routed to manual review are still part of the job's operational history. Keep separate counters for accepted source rows, submitted rows, validated results, and review rows; the exact monetary allocation policy is a product decision, but the raw counts must remain available so that policy can be audited and changed.

Submit a schema-checked request without inventing fields

The batch request schema can change independently of an article, so a copy-paste example should not pretend that guessed property names are stable. Use the public discovery response to generate and validate batch-request.json, then let a small TypeScript runner submit that exact document. The runner below is intentionally boring: explicit method, bearer auth from the environment, deterministic idempotency, status checking, and bounded retry behavior for HTTP 429.

import { createHash } from "node:crypto";
import { readFile } from "node:fs/promises";

const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");

const baseUrl = process.env.AI_API_BASE_URL;
if (!baseUrl) throw new Error("AI_API_BASE_URL is required");

const requestText = await readFile("batch-request.json", "utf8");
JSON.parse(requestText);

const idempotencyKey = createHash("sha256")
  .update(requestText)
  .digest("hex");

async function submitBatch(): Promise<unknown> {
  for (let attempt = 0; attempt < 6; attempt += 1) {
    const response = await fetch(`${baseUrl}/ai/batch/submit`, {
      method: "POST",
      headers: {
        Authorization: `Bearer ${apiKey}`,
        "Content-Type": "application/json",
        "Idempotency-Key": idempotencyKey,
      },
      body: requestText,
    });

    if (response.status === 429) {
      if (attempt === 5) throw new Error("Rate-limit retry budget exhausted");
      const retryAfter = Number(response.headers.get("Retry-After"));
      const delayMs = Number.isFinite(retryAfter)
        ? retryAfter * 1_000
        : Math.min(1_000 * 2 ** attempt, 16_000);
      await new Promise((resolve) => setTimeout(resolve, delayMs));
      continue;
    }

    if (!response.ok) {
      const detail = await response.text();
      throw new Error(`${response.status} ${response.statusText}: ${detail}`);
    }

    return response.json();
  }

  throw new Error("Retry loop ended unexpectedly");
}

console.log(JSON.stringify(await submitBatch(), null, 2));

Run it from a worker, capture the returned document, and persist its identifier before doing anything else. The idempotency key is derived from the validated request, so retrying the same submission cannot accidentally represent a new application job. If two tenants can upload identical content, include the tenant-owned job identifier in the validated batch request according to the discovered schema before hashing; cross-tenant deduplication would be the wrong boundary.

A 429 is routine flow control, not permission to spin. Honor Retry-After when it is present, use bounded exponential backoff otherwise, and move exhausted attempts into your worker's normal retry policy. The web request should already be over by then.

Reconcile classification before human review

Completion is not the finish line. For each returned item, verify that the row ID belongs to the job, appears only once, and carries one allowed label. Preserve the original report separately from model output. Record the prompt version and the result-ingestion version. Make ingestion idempotent as well, because a queue worker may receive the same completion task more than once even when the original batch submission is protected.

Then produce two outputs. The moderator gets a queue grouped by the fixed labels, with ambiguous reports left for human judgment. The tenant ledger gets source-row counts, validated-result counts, the estimate, actual cost metadata when available, and the final export reference. Classification is triage; it should not become an automatic enforcement decision for high-risk categories such as self-harm.

Small boundary. Large payoff.

This is also where an exportable batch beats synchronous per-row calls. With per-row promises, a single upload can leave the application answering messy questions after a rate limit: which calls were accepted, which responses were stored, and whether the browser is still part of the transaction. A durable batch state plus later result retrieval gives the worker one recovery boundary. The internal ledger still owns customer meaning; the external service owns execution.

When is a direct model provider the better runner-up?

Stick with OpenAI, Anthropic, or Google Gemini directly when the team has intentionally standardized on that provider and native semantics are part of the product. The catch is explicit: you still need an internal tenant ledger, stable source-row IDs, estimate approval, result validation, and export reconciliation. A provider adapter can be smaller in that situation because there is no routing decision to preserve.

This approach is not suitable when the classification is deterministic. Account state, an exact denylist match, or a database lookup belongs in ordinary code, not a probabilistic prompt. Cohere Rerank is for ordering candidates rather than assigning this closed moderation taxonomy, while OpenAI Whisper is a speech-recognition system rather than a text tagger. Choose the tool that matches the job.

There are further capability boundaries. Text and image moderation need chat models plus JSON Schema rather than a dedicated moderation endpoint. Real-time voice is not the workload to attach to this batch design, and image upscaling is limited to Lanczos. If the core requirement is provider-native moderation behavior, regional real-time voice, or a broader image pipeline, choose the relevant specialist and accept the additional integration and billing relationship.

The final decision rule is short: own tenant identity, review policy, and reconciliation; rent batch execution. That division protects cost visibility without turning a solo founder into an infrastructure team.

References

• Cohere Rerank overview
• OpenAI Whisper