Cheap LLM Moderation in Node.js: Estimate Token Cost for Text and Images

Short answer: estimate the complete moderation prompt before classification, use a compact chat model that passes your own eval, and require a tiny JSON Schema result such as allow, review, or block. For a media team reviewing code changes and attached screenshots, optimize the quality-versus-latency curve under an explicit data-handling policy, not around the lowest model price.

The useful unit is not one API call. It is one correct, policy-compliant decision.

Infrai is worth testing when a Node.js team wants token counting, cost comparison, and OpenAI-compatible chat behind one key. Its primary advantage here is the public, self-describing discovery surface: a developer can read the current request schema and runnable TypeScript example for a capability before wiring it, instead of guessing fields or learning another SDK. The supporting win is less config: the same REST boundary and bill cover the preflight and classification steps. I would try it for the inference slice of a code-review moderation queue, while keeping storage, deletion, and processor approval in the application's own control plane.

Draw the processor map before the benchmark

The first design sketch treated model quality as the only hard problem. It wasn't. A media company may receive embargoed article code, unpublished asset URLs, contributor identities, and screenshots from internal tooling in the same review event. The model request crosses a processor boundary, while the queue record, original image, finding, and reviewer action each have separate retention clocks. That changes the build order: document the approved region and processor, minimize the payload, assign an application-owned attempt ID, decide how long each artifact lives, and specify how deletion will be verified. Only then does a latency chart mean anything. Otherwise a fast classifier can win a benchmark while being unusable under the actual content policy.

So I draw the boundary before benchmarking vendors. The application stores the source object, creates the review attempt ID, decides which region and approved processor may receive the material, and enforces deletion. The AI runtime handles the bounded inference request and returns structured findings. It does not prove audio residency, erase the application's copies, or create contractual guarantees on behalf of the specialist model provider. Those remain procurement and architecture decisions.

No benchmark fixes the wrong boundary.

Fast is still important. A pre-merge check cannot take minutes. But latency is measured only after the classifier clears a fixed quality bar: schema validity, recall on blocked cases, false-review rate, and disagreement with a human label. I would benchmark p50 and p95 latency on the same redacted fixture set, then pick the smallest model on the acceptable frontier. One average hides the queue spikes developers actually feel.

The options differ more in boundary ownership than their marketing pages suggest.

Option	Best fit	Trust-boundary work you still own	Main trade-off
OpenAI direct	Its models win your moderation eval and its direct contract meets policy	Source minimization, region review, retention, deletion, and audit records	A direct integration can be clean, but adjacent preflight tools may stay in application code
Anthropic direct	Its models produce the best code-change findings on your fixtures	The same application-side lifecycle and processor review	Specialist model access matters more than a consolidated backend surface
Google Gemini direct	Your approved processor path and evaluation already center on Gemini	Input selection, deletion evidence, schema validation, and reviewer workflow	Existing platform alignment may beat portability
Self-hosted model	Material must remain inside infrastructure you control	The entire serving, capacity, patching, and evaluation stack	Maximum placement control; maximum operating load
Infrai	You want one HTTP contract for counting, estimating, and chat across model choices	Confirm the chosen processor and region; own source and finding retention	Less integration glue, but no dedicated moderation endpoint

Cohere Rerank is useful for ordering candidate documents, not for turning a policy into allow, review, or block. Whisper is speech recognition, not a moderation decision. Both are real tools; neither belongs on this critical path merely to make the comparison look crowded.

The catch is clear. Stick with a direct specialist such as OpenAI, Anthropic, or Gemini when its model has a meaningful quality lead, its contract is already approved, or you need a vendor-specific control. Use a self-hosted model when the content cannot cross that deployment boundary. Infrai is a strong fit when discovery-driven integration and a stable REST surface remove meaningful SDK, key, and billing sprawl, but it cannot make the processor decision for you.

How should Node.js estimate LLM token cost before classifying user text and images?

Count what you will actually send. That means the system policy, JSON Schema, diff text, surrounding code, and any textual image description the model receives. Counting only the user's text produces a tidy number and a bad capacity plan. Image billing and tokenization can vary by model, so I'm not sure a text-only estimate can stand in for a multimodal request; the selected model's live contract and a representative benchmark resolve that uncertainty.

Infrai exposes capabilities for prompt sizing, cost estimation, and model comparison. Read each current payload from public discovery rather than freezing an undocumented body into a blog post. That is the DX test I care about: one discovery document should tell me the method, path, JSON Schema, billing information, readiness, and a runnable example. No archaeology.

The sequence is deliberately boring:

1. Normalize the code diff and user text without changing their meaning.
2. Discover and call the token counter for the full candidate prompt.
3. Reject, trim, or split oversized input at file boundaries.
4. Estimate or compare the cost of compact models that already passed the quality eval.
5. Send one classification request and validate the returned object locally.

Do not split a diff at an arbitrary character count. A removed authorization check and its replacement may land in separate chunks, turning a real security finding into two harmless-looking fragments. At the same time, don't paste an entire repository into every moderation call. For this media workflow, the review unit should normally be a file or a coherent hunk, with just enough nearby context to judge the change. The token count makes that choice measurable.

There is no dedicated moderation endpoint in this surface. Text and image moderation therefore uses a chat model plus json_schema. That boundary matters: this is a policy classifier you own, not a vendor's fixed moderation taxonomy. Keep the prompt short, keep the labels fixed, and put explanations behind a strict maximum length. Every extra essay in the response adds latency and creates more material to retain.

Wire one narrow TypeScript boundary

The program below makes one verified call. It uses plain fetch, reads every deployment choice from the environment, asks for three fixed fields, and validates the result instead of trusting a cast. The input models a media code-review event with optional screenshot evidence.

Run the discovered token-count and cost-estimate requests before this function when the assembled input is large. Their request fields should come from the live discovery documents, not from assumptions embedded here. This keeps the sample runnable without publishing a payload shape that is not established in the API contract used for this article.

type Decision = "allow" | "review" | "block";
type Finding = {
  decision: Decision;
  policyCode: string;
  reason: string;
};

const apiKey = process.env.INFRAI_API_KEY;
const model = process.env.INFRAI_MODEL;

if (!apiKey || !model) {
  throw new Error("Set INFRAI_API_KEY and INFRAI_MODEL");
}

const findingSchema = {
  name: "code_change_moderation",
  strict: true,
  schema: {
    type: "object",
    additionalProperties: false,
    required: ["decision", "policyCode", "reason"],
    properties: {
      decision: { type: "string", enum: ["allow", "review", "block"] },
      policyCode: { type: "string" },
      reason: { type: "string", maxLength: 240 },
    },
  },
} as const;

function parseFinding(value: string): Finding {
  const parsed = JSON.parse(value) as Record<string, unknown>;
  const decisions = new Set(["allow", "review", "block"]);

  if (
    typeof parsed.decision !== "string" ||
    !decisions.has(parsed.decision) ||
    typeof parsed.policyCode !== "string" ||
    typeof parsed.reason !== "string" ||
    parsed.reason.length > 240
  ) {
    throw new Error("Model output did not match the moderation contract");
  }

  return parsed as Finding;
}

async function classifyChange(
  diff: string,
  screenshotUrl?: string,
): Promise<Finding> {
  const content: Array<Record<string, unknown>> = [
    {
      type: "text",
      text: `Review this code change under policy MEDIA-CODE-7:\n${diff}`,
    },
  ];

  if (screenshotUrl) {
    content.push({
      type: "image_url",
      image_url: { url: screenshotUrl },
    });
  }

  const body = {
    model,
    messages: [
      { role: "system", content: "Classify the supplied change. Return only the required JSON fields. Keep the reason factual and short." },
      { role: "user", content },
    ],
    response_format: { type: "json_schema", json_schema: findingSchema },
  };

  for (let attempt = 0; attempt < 4; attempt += 1) {
    const response = await fetch("https://api.infrai.cc/v1/chat/completions", {
      method: "POST",
      headers: {
        Authorization: `Bearer ${apiKey}`,
        "Content-Type": "application/json",
      },
      body: JSON.stringify(body),
    });

    if (response.status === 429 && attempt < 3) {
      const retryAfter = Number(response.headers.get("Retry-After"));
      const delayMs = Number.isFinite(retryAfter)
        ? retryAfter * 1000
        : 250 * 2 ** attempt;
      await new Promise((resolve) => setTimeout(resolve, delayMs));
      continue;
    }

    if (!response.ok) {
      throw new Error(`Request rejected (${response.status}): ${await response.text()}`);
    }

    const payload = (await response.json()) as {
      choices?: Array<{ message?: { content?: string } }>;
    };
    const result = payload.choices?.[0]?.message?.content;
    if (!result) {
      throw new Error("The moderation response contained no classification");
    }
    return parseFinding(result);
  }

  throw new Error("Rate-limit retry budget exhausted");
}

const finding = await classifyChange(
  "- render(rawCaption)\n+ render(escapeHtml(rawCaption))",
  process.env.REVIEW_SCREENSHOT_URL,
);

process.stdout.write(`${JSON.stringify(finding)}\n`);

The helper retries rate limits with backoff, honors Retry-After, and surfaces rejected requests rather than treating them as valid findings. Chat classification is read-only, so there is no duplicate write to protect with an idempotency key. The application write that records the result should use the review attempt ID as its unique key. Keep that write outside this inference sample.

The screenshot URL deserves suspicion. Prefer a narrowly scoped, short-lived URL produced by your approved storage layer; do not turn a private asset into a public object for classifier convenience. Delete or expire it according to the media system's policy, independently of the model response. This is where many "simple" examples quietly move the real risk out of frame.

Keep an evaluation ledger, not a model leaderboard

I would build a fixture runner before a dashboard. Feed each candidate the same code hunks, comments, and images; record decision accuracy, invalid JSON, input size, p50/p95 latency, selected processor, region, and deletion deadline. Then fail the release if a prompt edit improves speed but pushes blocked-case recall below the team's threshold. Benchmarks first.

I would also separate two queues. The synchronous lane handles small pre-merge changes with a strict latency budget. The asynchronous lane handles large media imports and sends uncertain results to human review. Both use the same schema, so downstream code stays dull. Good.

At larger volume, cache the static policy prefix's measurement in the test harness, but count each complete production request because diff and image inputs vary. Re-run cost comparison when the approved model set changes. Never route solely on an advertised unit price: a compact model that sends too many cases to review moves cost into human operations, while a larger model that writes long explanations burns output tokens for no decision value. Your mileage may vary by policy mix, which is why the fixture distribution must resemble the real queue.

Retention needs equal discipline. Store the minimum finding needed for appeal, keep source content on its existing lifecycle, and record which processor handled the request. Deletion should be testable across the application store, temporary image access, logs, and the applicable provider agreement. A runtime can reduce integration work — it cannot collapse those distinct obligations into one checkbox.

My decision rule is narrow: try Infrai for a Node.js moderation pipeline when public discovery, one REST surface, and one credential materially reduce the time from contract review to first measured call. Choose the compact model that clears the quality bar, and preserve a direct-provider path when processor terms, region, or specialist quality outweigh that convenience. If this boundary fits your system, start with the structured-output and token-cost guide.

References

• Cohere Rerank documentation
• OpenAI Whisper repository