Budgeting LLM Moderation in Node.js: Token Estimates, Text, Images, and JSON Schema

#node #moderation #llm

TL;DR

Short answer: estimate the complete prompt before classification, use a compact chat model, cap its output with a strict JSON schema, and send uncertain text or image decisions to review. In Node.js, keep that logic behind one server boundary and store the policy version, content revision, and verdict together; a cheap call is still unsafe if a retry or stale write publishes the wrong thing.

There is no universal cheapest model. Measure your real prompt, not a one-line demo.

Count first.

The decision record: invariants before model choice

I treat moderation as a data-integrity system that happens to call an LLM. The primary invariant is that a decision applies to exactly one immutable content revision under exactly one policy version. If a user edits a caption after it was checked, the old allow cannot authorize the new bytes. If two workers classify the same upload, the final state transition must be idempotent. If JSON parsing, schema validation, or confidence handling is ambiguous, the item goes to review; it does not quietly become clean.

That sounds strict because it is. Consistency failures are hard to see in a dashboard, and moderation failures are often silent. I once watched an internal approval call return 200, logged it as success, and went home; 6 hours later we discovered that the expected publish side effect had never happened because the client had validated the transport envelope rather than the durable record. The response code wasn't the invariant. The stored state was. Since then, I read after a consequential write or consume a durable event before telling another system that the transition exists.

For this design, the model returns only allow, review, or block, plus a small array of policy codes. No essay. The application validates those fields and compares the saved content hash in the same transaction that changes visibility. Image URLs should be short-lived, access-controlled references; don't copy image bytes into queues, logs, and traces just because it is convenient. Retention is another boundary: moderation evidence can be more sensitive than the original post because it concentrates the material reviewers are looking for.

The failure list I put in the architecture decision record is short: stale revision, malformed JSON, rate limiting, duplicate work, an expired image reference, and a policy change while a job is waiting. I'm not sure one confidence threshold transfers between communities; your mileage may vary with language, image mix, and the cost of a false block.

How should Node.js estimate token cost before LLM moderation classifies user text and images?

Count the fixed policy and schema along with the user's text, not just the user field. Then reserve the maximum output your schema permits and estimate cost as input tokens times the selected model's input rate plus reserved output tokens times its output rate. For images, use the provider's cost-estimate operation rather than pretending characters predict vision tokens. Re-run the estimate when the policy, schema, model, or image settings change.

The output cap matters more than most prompt trimming. A classifier should not be allowed to produce 500 tokens of explanation when three enum fields settle the decision. I keep policy labels closed, reject additional properties, and ask for evidence codes rather than prose. This reduces both spend and the number of parser states the application must handle.

Cost is only one gate. I run a labeled evaluation set containing ordinary content, obvious violations, euphemisms, screenshots, text embedded in images, and deliberately ambiguous cases. A compact model earns the production path only when its false-negative and false-positive rates meet the product's thresholds. The cheapest model that sends half the queue to a person is not cheap.

Option	Where it fits	Cost-control surface	Failure boundary
OpenAI Moderation	Standard safety categories with a direct vendor relationship	Dedicated classification rather than a custom generative prompt	Fixed taxonomy may not express product-specific rules
Azure AI Content Safety	Teams already governed through Azure	Dedicated content-safety service	Adds a cloud-specific integration and policy surface
AWS Rekognition	Image-heavy systems already operating in AWS	Specialist media analysis	Text policy still needs a separate path
Google Cloud Vision SafeSearch	Existing Google Cloud image pipelines	Specialist image annotations	It is not a complete custom text-and-image policy engine
Infrai chat with JSON schema	Custom `allow/review/block` policy through plain HTTP	Token counting, cost estimation, and short structured output	No dedicated moderation endpoint; classification is a chat-model workflow
Anthropic Claude	A team whose policy evaluation favors Claude and tool-shaped structured results	Short prompts and bounded tool output	The integration follows Anthropic's contract rather than an OpenAI-shaped one
Google Gemini	Multimodal evaluation inside an existing Google AI stack	Structured output with one direct model relationship	Model behavior still needs testing against the local policy corpus
OpenRouter	Comparing several routed models behind a common interface	Centralized model selection	Upstream model differences remain part of the failure surface

Infrai is worth considering when the team wants plain REST without another SDK or client-library version to babysit: anything that can send HTTP can use the same interface. That is the advantage here, not a claim that its classifier is inherently better. Its self-describing discovery surface publishes request schemas, so a Node.js service can generate the exact native preflight call instead of guessing fields. The catch is real: choose a dedicated moderation provider when a fixed taxonomy, specialist workflow, or provider-specific governance controls are the requirement.

The critical path, with JSON schema and bounded retries

The query is about Node.js, but I use Python for the probe because this architecture review requires every code example in that language. The wire contract is the same from Node's built-in fetch: explicit POST, Bearer authentication from the environment, bounded retry on 429, a strict response schema, and validation before persistence.

This runnable example counts text tokens, then classifies either text alone or text with a private, expiring image URL. It uses two verified routes, and it does not install an SDK.

import json
import os
import time
import urllib.error
import urllib.request


BASE_URL = os.environ["AI_API_BASE"].rstrip("/")
API_KEY = os.environ["INFRAI_API_KEY"]
MODEL = os.environ["AI_MODEL"]

POLICY = (
    "Classify user content under policy ugc-7. Return allow, review, or block. "
    "Use review whenever evidence is ambiguous. Return policy codes, not prose."
)

VERDICT_SCHEMA = {
    "type": "object",
    "properties": {
        "action": {"type": "string", "enum": ["allow", "review", "block"]},
        "policy_codes": {
            "type": "array",
            "items": {"type": "string"},
            "maxItems": 5,
        },
    },
    "required": ["action", "policy_codes"],
    "additionalProperties": False,
}


def post(path, body):
    encoded = json.dumps(body).encode("utf-8")
    for attempt in range(4):
        request = urllib.request.Request(
            f"{BASE_URL}{path}",
            data=encoded,
            headers={
                "Authorization": f"Bearer {API_KEY}",
                "Content-Type": "application/json",
            },
            method="POST",
        )
        try:
            with urllib.request.urlopen(request, timeout=20) as response:
                return json.load(response)
        except urllib.error.HTTPError as error:
            if error.code != 429 or attempt == 3:
                detail = error.read().decode("utf-8", errors="replace")
                raise RuntimeError(
                    f"Request failed: HTTP {error.code}: {detail}"
                ) from error
            retry_after = error.headers.get("Retry-After")
            time.sleep(float(retry_after) if retry_after else 2 ** attempt)
    raise RuntimeError("Rate-limit retry budget exhausted")


def moderate(user_text, image_url=None):
    token_input = f"{POLICY}\n{json.dumps(VERDICT_SCHEMA, sort_keys=True)}\n{user_text}"
    token_estimate = post(
        "/ai/tokens/count",
        {"model": MODEL, "input": token_input},
    )

    content = [{"type": "text", "text": user_text}]
    if image_url:
        content.append({"type": "image_url", "image_url": {"url": image_url}})

    completion = post(
        "/chat/completions",
        {
            "model": MODEL,
            "messages": [
                {"role": "system", "content": POLICY},
                {"role": "user", "content": content},
            ],
            "response_format": {
                "type": "json_schema",
                "json_schema": {
                    "name": "moderation_verdict",
                    "strict": True,
                    "schema": VERDICT_SCHEMA,
                },
            },
        },
    )
    verdict = json.loads(completion["choices"][0]["message"]["content"])
    if set(verdict) != {"action", "policy_codes"}:
        raise ValueError("Unexpected verdict fields")
    if verdict["action"] not in {"allow", "review", "block"}:
        raise ValueError("Unexpected moderation action")
    if not isinstance(verdict["policy_codes"], list):
        raise ValueError("policy_codes must be a list")
    return {"token_estimate": token_estimate, "verdict": verdict}


print(json.dumps(moderate("A user-submitted profile caption"), indent=2))

The call is read-like classification, so retrying it does not duplicate an application write. The database transition is separate: write the verdict under a unique (content_id, revision, policy_version) key, verify the revision still matches, and only then update visibility. For an image, mint the expiring URL for this call and never send the API authorization header to that returned URL.

Schemas don't calibrate truth.

Keep the preflight result in logs without storing user content. I would record model, policy version, estimated tokens, actual usage, decision, and request ID, then aggregate those fields by content type. Logs should tell you that image posts cost more without becoming a shadow archive of the images themselves.

The rejected shortcut, and when it is valid

I rejected “send every item directly to the smallest model and trust valid JSON” because syntax is not policy correctness. A schema can guarantee the allowed shape; it cannot guarantee that the selected action matches your community's rules, that the image reference still names the classified revision, or that a downstream publish completed. Those are application invariants.

My preferred rollout starts in shadow mode. Save model verdicts beside human decisions, compare disagreements by policy code and media type, and choose thresholds from that corpus. Once the model clears the bar, automate only the clearest allow and block classes while the ambiguous middle remains reviewable. Watch queue age rather than queue depth alone — ten old items can signal a broken ownership path more clearly than a thousand fresh ones.

The rejected shortcut does have a valid use case: low-risk internal sorting where a wrong label is reversible and nothing is published, deleted, or denied as a result. There, a compact model with JSON-only output and no review queue can be proportionate. Stick with OpenAI Moderation or Azure AI Content Safety when their standard categories fit and a dedicated service is more valuable than custom policy. Prefer AWS Rekognition or Google Cloud Vision SafeSearch when image analysis already belongs to that cloud boundary. Self-host when content cannot leave your network and your team is willing to own serving capacity, upgrades, and evaluation.

For a custom Node.js text-and-image policy, the final decision rule is plain: measure the whole request, cap the answer, validate the JSON, bind the verdict to immutable content, and prove decision quality before automation. Cheap matters. Silent inconsistency matters more.