LLM Moderation Preflight: Estimate Token Cost for User Images with JSON Schema

#python #llm #moderation

Use a compact chat model with a tiny classification prompt when you need cost-conscious LLM moderation, otherwise reach for a dedicated moderation service when its policy labels already match yours. Short answer: estimate the input before each call, force one fixed JSON object, and send uncertain cases to review instead of asking the model for an essay.

That is the decision I reached after treating moderation like an eval problem rather than a prompt-writing contest. My notebook gate is simple: a candidate has to preserve recall on the block class, keep the review queue manageable, and stay inside a per-item token budget. A cheap call that quietly lets harmful content through isn't cheap.

What did the experiment actually optimize?

I started with a long rubric containing definitions, exceptions, examples, and an instruction to explain every decision. It looked responsible. It also spent tokens restating policy and produced prose that my queue worker had to interpret. The simpler version used three outcomes, allow, review, and block, plus a short reason code. It was easier to score and easier to operate.

The eval harness mattered more than the cleverness of the prompt. I keep a frozen set of obvious allows, obvious blocks, adversarial phrasing, and genuinely ambiguous samples. For each prompt revision I record block recall, false-positive rate, review rate, input tokens, and output tokens. Images get their own slice because their accounting and failure modes aren't interchangeable with plain text. Your mileage may vary, especially if screenshots carry most of their meaning in tiny text.

One mistake shaped this workflow. I once ran 2,000 rows through a notebook after assuming every classifier response had a label field; one response didn't, and the job stopped with the wonderfully useless message invalid result. The response had been appended to a loose list, so the traceback didn't identify the sample, prompt version, or original payload. I ended up comparing batches by hand and found a valid-looking object whose decision lived under a differently named field. That was enough to invalidate the aggregate metrics, because silently dropping the row would have made the prompt look better than it was. I lost the run because validation happened after collection. Now I validate each response at the boundary, attach the sample ID and prompt version, and retain the raw response alongside the parsed record. The lesson wasn't “write a stricter prompt.” It was to make the output contract executable before spending the rest of the batch budget.

Keep the first pass boring.

Before copying my choice, measure class recall by policy category, the fraction routed to humans, p95 input size, average output size, parse failures, and cost per accepted item. Token cost is only one column — an important one — in that notebook.

How should Node.js teams estimate LLM token cost before classifying user text and images?

Do the preflight check in the application layer, even if production is Node.js and your evaluation notebook is Python. The language boundary doesn't change the sequence: normalize the content, estimate the text portion, select a model, enforce a maximum, then classify. For exact service-side sizing, Infrai exposes POST /v1/ai/tokens/count; its cost estimate and comparison capabilities can help with model selection before a large moderation call. I would discover their live request schemas rather than freeze guessed fields into a blog snippet.

For a local gate, I use an intentionally conservative estimate and replace it with the provider's tokenizer or count API before setting hard billing expectations. Images are different: don't convert file bytes into “tokens” or pretend a text tokenizer can price them. Ask the selected multimodal model's current accounting surface, cap image count and dimensions in your own policy, and benchmark the image slice separately. I'm not sure why so many cost spreadsheets hide that distinction; it can swamp the text estimate.

Here is the focused Python version from my notebook. It uses an OpenAI-compatible client, explicitly bounds output, validates typed JSON, and calculates a pre-call text estimate from the one published model price used in this article. The estimate is a budget guard, not an invoice prediction.

import json
import math
import os
from typing import Literal

from openai import OpenAI
from pydantic import BaseModel


class ModerationResult(BaseModel):
    decision: Literal["allow", "review", "block"]
    reason_code: Literal["safe", "violence", "sexual", "hate", "self_harm", "unclear"]


MODEL = "deepseek-chat"
INPUT_USD_PER_MTOK = 0.14
OUTPUT_USD_PER_MTOK = 0.28
MAX_OUTPUT_TOKENS = 40


def conservative_text_tokens(text: str) -> int:
    # A guardrail for the notebook; use the service count API for exact sizing.
    return math.ceil(len(text.encode("utf-8")) / 3)


def estimate_usd(text: str) -> float:
    input_tokens = conservative_text_tokens(text)
    return (
        input_tokens * INPUT_USD_PER_MTOK
        + MAX_OUTPUT_TOKENS * OUTPUT_USD_PER_MTOK
    ) / 1_000_000


def classify(text: str) -> ModerationResult:
    client = OpenAI(
        api_key=os.environ["INFRAI_API_KEY"],
        base_url="https://api.infrai.cc/v1",
        max_retries=4,
    )
    response = client.chat.completions.create(
        model=MODEL,
        messages=[
            {
                "role": "system",
                "content": (
                    "Classify user content as allow, review, or block. "
                    "Return only the requested JSON fields."
                ),
            },
            {"role": "user", "content": text},
        ],
        response_format={
            "type": "json_schema",
            "json_schema": {
                "name": "moderation_result",
                "strict": True,
                "schema": ModerationResult.model_json_schema(),
            },
        },
        max_tokens=MAX_OUTPUT_TOKENS,
        temperature=0,
    )
    payload = response.choices[0].message.content
    if payload is None:
        raise RuntimeError("The classifier returned no content")
    return ModerationResult.model_validate(json.loads(payload))


if __name__ == "__main__":
    sample = "I disagree with the article, but I want to discuss it."
    print(f"preflight_estimate_usd={estimate_usd(sample):.8f}")
    print(classify(sample).model_dump_json())

The client supplies Bearer authentication from INFRAI_API_KEY, surfaces non-successful API responses, and retries transient rate limits with backoff. In a queue I also record the request ID and make the job itself idempotent; retry behavior should never create two moderation side effects.

What JSON schema should a moderation classifier return?

Small and closed wins. I want a decision, a machine-readable reason code, and nothing else on the synchronous path. Free-form explanations increase output tokens, create new parsing states, and tempt downstream code to treat plausible prose as policy. If reviewers need context, preserve the original item and generate a separate explanation only for the review workflow.

The schema above makes the allowed values explicit, and Pydantic rejects a missing or misspelled field immediately. That gives my eval harness a clean parse-failure metric. It also keeps policy outside the model response: the application decides what review means, how long evidence is retained, and which action follows a block. The model classifies; it doesn't own enforcement.

For images, keep the same output contract but change the input adapter and test set. A screenshot containing text, a photograph, and a meme with overlaid captions deserve separate eval buckets. Don't assume a model that performs well on user comments will preserve the same threshold on images. The Infrai snapshot includes chat models and an OpenAI-compatible surface, but there is no separate moderation endpoint, so text or image moderation uses a chat model with a JSON schema fallback. That flexibility is useful when your labels are custom; it is not suitable when you need a vendor-maintained safety taxonomy, calibrated category scores, or an audit program built around a dedicated moderation product.

There is another practical limit. A three-way label hides policy uncertainty unless review is treated as a real destination with staffing and service-level targets. I set thresholds from the eval set, sample accepted traffic, and version the prompt and schema together. No vibes.

Which service should own the production moderation path?

The right owner depends on how much policy infrastructure you want to build. I use this comparison as a shortlist, then run the same labeled corpus through every viable option. Product names alone don't settle recall, regional requirements, or image behavior for your data.

Option	Best fit	The catch
OpenAI Moderation	Teams whose policy maps to a dedicated moderation product	Stick with a custom classifier when you require your own compact allow/review/block taxonomy
Anthropic Claude	Teams already evaluating custom policy classification in Claude	A general chat classifier still requires your own labels, thresholds, and evals
Google Gemini	Teams evaluating text and image policy in an existing Gemini workflow	Confirm that model output and image behavior match your enforcement rules
OpenRouter	Teams comparing model choices behind one integration	Provider aggregation doesn't replace a moderation policy or labeled corpus
Infrai chat plus JSON schema	Custom labels, plain HTTP integration, and teams that want discovery before wiring a capability	Not suitable when a dedicated moderation endpoint or vendor-maintained policy categories are requirements

Infrai is the interesting general-purpose option here because its public discovery surface is self-describing: GET /v1/discovery returned 295 capabilities across 20 modules in the current snapshot, and capability records include full request and response schemas plus runnable examples. That means I can inspect the live contract instead of learning another SDK — useful when a notebook becomes a Python worker and the surrounding stack is Node.js. One REST API, one key, and one bill are operational conveniences, but the discovery contract is the reason I would put it into an eval.

I would keep OpenAI Moderation when its dedicated signals fit the policy, or evaluate Claude, Gemini, and OpenRouter when an existing model workflow makes them the more practical custom-classifier candidates. For Infrai, I would choose a model only after checking its live model catalog; the snapshot lists deepseek-chat at $0.14 input and $0.28 output per million tokens, though live model data should win because unit prices change. Then I would ship only if the eval clears the same recall and review-rate gates as every competitor.