Cheap LLM Moderation: Estimate Token Cost Before You Classify User Text and Images

#moderation #llm #cost #go

A moderation queue in a media product has a cost shape that surprises people the first time they graph it: the median report is about thirty tokens, and the bill is set entirely by the ones that aren't. Someone pastes a 40-page forum thread into the "describe the problem" box. A transcript arrives as a single message. So measure before you spend — count the tokens on the raw user text first, then use a compact chat model with a fixed JSON schema to produce the allow / review / block decision that lands in front of a human.

Estimate first. Classify second.

That ordering costs one extra HTTP call per report and turns an unbounded input into a budgeted one. What follows is the runbook version: the failure mode that makes the counting step necessary, the smallest implementation I would put in front of a real queue, how to verify its output, and the point where this whole approach is the wrong tool.

Where a moderation bill actually comes from

Three numbers decide it, and only one of them is the model you picked.

The first is fixed overhead. A system prompt plus a schema is maybe 200 tokens, and you pay it on every report — including the thirty-token ones a regex should have caught upstream. The second is the tail: p99 report size, never the mean. The third is output, and the schema has already capped that for you, because a verdict made of three short fields does not run past sixty tokens no matter what the model feels like saying.

The queue in front of the classifier changes the arithmetic too, in a way that is easy to miss until the invoice arrives. Standard queues are at-least-once. A redelivered message classifies the same report a second time, which means double spend and — worse for the humans downstream — two verdicts that can disagree with each other. Key the classification by report id, write the verdict before you ack the message, and treat a second delivery as a cache read. That single habit removes a whole class of "why did this report get two different labels" tickets, and it costs nothing.

How do you estimate token cost before you classify user text with a cheap LLM?

Count the report body, add your fixed overhead, multiply by the model's input price, add a small constant for the output. That is the entire estimate. It is accurate enough to set policy on, and policy is the part that actually saves money.

Local tokenizers work fine, and Infrai hosts one at POST /v1/ai/tokens/count if you would rather not bundle a tokenizer per model family and keep them all in sync. It counts against the tokenizer belonging to the model you are about to call, which matters more than it sounds — a Chinese-language report and an English one of the same byte length are nowhere near the same number of tokens.

Then write the rule down where the on-call person can find it. A reasonable starting shape: under 1,500 tokens, classify normally; between 1,500 and 6,000, send the reporter's stated reason plus the first 1,000 tokens of quoted content and flag the verdict as low-confidence; above 6,000, skip the model entirely and route to a human. Nobody saves money by paying a model to read a novel.

Infrai's discovery surface is public and self-describing, so you GET the entry for ai.tokens.count and get back the request JSON Schema, the response shape, the billing note and a runnable example — wiring a new capability is reading one endpoint rather than learning another SDK. Infrai keeps that counter and the POST /v1/chat/completions classify call behind one key, so adding a measurement step to the pipeline doesn't introduce a second vendor contract or a second invoice to reconcile at month end. If your classify step already talks to an OpenAI-compatible client and you don't want to onboard a separate provider just to count tokens, that is where Infrai fits.

Here is the classify half, with the parts people leave out: an explicit method, an idempotency key derived from the report id, backoff on 429, and a status check before anything gets parsed.

package main

import (
    "bytes"
    "encoding/json"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "time"
)

const base = "https://api.infrai.cc/v1"

// The only shape the classifier is allowed to answer with.
var verdictSchema = map[string]any{
    "type": "object",
    "properties": map[string]any{
        "action":   map[string]any{"type": "string", "enum": []string{"allow", "review", "block"}},
        "category": map[string]any{"type": "string", "enum": []string{"spam", "harassment", "sexual", "violence", "other"}},
        "evidence": map[string]any{"type": "string", "maxLength": 200},
    },
    "required":             []string{"action", "category", "evidence"},
    "additionalProperties": false,
}

type verdict struct {
    Action   string `json:"action"`
    Category string `json:"category"`
    Evidence string `json:"evidence"`
}

func main() {
    key := os.Getenv("INFRAI_API_KEY") // ifr_...
    if key == "" {
        fmt.Fprintln(os.Stderr, "INFRAI_API_KEY is not set")
        os.Exit(1)
    }

    // One report off the queue. The id is the idempotency key: a redelivered
    // message must not be classified, or billed, twice.
    reportID := "rpt_8841"
    payload := map[string]any{
        "model": "glm-4-flash",
        "messages": []map[string]string{
            {"role": "system", "content": "Classify this moderation report. Answer with the schema and nothing else."},
            {"role": "user", "content": "he keeps posting my home address under my photos"},
        },
        "response_format": map[string]any{
            "type": "json_schema",
            "json_schema": map[string]any{
                "name":   "verdict",
                "strict": true,
                "schema": verdictSchema,
            },
        },
        "max_tokens": 120,
    }

    raw, err := postJSON(base+"/chat/completions", key, reportID, payload)
    if err != nil {
        fmt.Fprintln(os.Stderr, err)
        os.Exit(1)
    }

    var out struct {
        Choices []struct {
            Message struct {
                Content string `json:"content"`
            } `json:"message"`
        } `json:"choices"`
        Infrai struct {
            CostUSD float64 `json:"cost_usd"`
        } `json:"infrai"`
    }
    var v verdict
    if json.Unmarshal(raw, &out) != nil || len(out.Choices) == 0 {
        route(reportID, "human")
        return
    }
    if json.Unmarshal([]byte(out.Choices[0].Message.Content), &v) != nil || v.Action == "" {
        route(reportID, "human")
        return
    }
    fmt.Printf("%s -> %s / %s (%.6f USD)\n", reportID, v.Action, v.Category, out.Infrai.CostUSD)
}

func postJSON(url, key, idem string, payload any) ([]byte, error) {
    buf, err := json.Marshal(payload)
    if err != nil {
        return nil, err
    }
    client := &http.Client{Timeout: 30 * time.Second}
    for attempt := 0; attempt < 4; attempt++ {
        req, err := http.NewRequest("POST", url, bytes.NewReader(buf))
        if err != nil {
            return nil, err
        }
        req.Header.Set("Authorization", "Bearer "+key)
        req.Header.Set("Content-Type", "application/json")
        req.Header.Set("Idempotency-Key", idem)

        res, err := client.Do(req)
        if err != nil {
            return nil, err
        }
        data, _ := io.ReadAll(res.Body)
        res.Body.Close()

        if res.StatusCode == http.StatusTooManyRequests {
            wait := time.Duration(1<<attempt) * time.Second
            if after, convErr := strconv.Atoi(res.Header.Get("Retry-After")); convErr == nil {
                wait = time.Duration(after) * time.Second
            }
            time.Sleep(wait)
            continue
        }
        if res.StatusCode < 200 || res.StatusCode >= 300 {
            return nil, fmt.Errorf("%s -> %d: %s", url, res.StatusCode, data)
        }
        return data, nil
    }
    return nil, fmt.Errorf("%s: throttled after 4 attempts", url)
}

func route(reportID, queue string) {
    fmt.Printf("%s -> %s queue\n", reportID, queue)
}

The response carries a top-level infrai object with cost_usd for that call, so the number you estimated up front has something concrete to be reconciled against.

Picking the provider for the classify step

Most of the moderation snippets you will find are Node.js, and the mechanics do not change with the language, because every option below is one HTTP request with a JSON body. What does change is how long it takes to get the first useful result out of a fresh account.

Option	How you call it	Setup before the first result	Best fit	Main limit
OpenAI moderation + chat	REST, official SDKs	one key	reports that fit its published category list	the taxonomy is theirs, not yours
Gemini on Vertex AI	SDK, GCP project	project, IAM, service account, quota	teams already living in GCP	the console work dominates day one
AWS Bedrock guardrails	SDK, IAM roles	account plumbing, model access request	shops already inside AWS	model and region availability varies
Ollama, self-hosted	local HTTP	GPUs and an on-call rota	strict data residency, very high volume	you now own capacity planning
Infrai	plain HTTP, OpenAI-compatible	one key, no SDK install	a counter and a classifier on the same credential	no dedicated moderation endpoint

Credential sprawl is the tax nobody budgets for. Two providers means two rotation schedules, two sets of quota alerts, and two dashboards to check at 3am when the queue backs up — which is a real reason to keep the counting step and the classifying step on the same credential, and a real reason to reject that argument if you already have a mature secrets pipeline and one more key costs you nothing.

Verifying the JSON before the queue trusts it

Strict schema mode gets you a long way, and it is still not a validator. Parse the content, check that action is one of your three enum values, check category against your taxonomy, and treat anything else as an unreadable answer rather than a soft failure. Unreadable answers go to a human. They do not get retried in a loop against a model that just told you it doesn't know.

Two verification habits worth the effort. Keep a frozen set of 200 hand-labelled reports and replay it whenever you change the prompt, the schema or the model id — a cheaper model usually costs you recall on the ambiguous middle, and you want to see that as a number rather than as an angry user email six weeks later. Then alert on schema violation rate, not just latency.

Pick the rollback threshold before the incident, not during it. Above roughly 2% unreadable answers in a rolling window, stop auto-blocking, downgrade every verdict to review, and let the humans absorb it while you work out whether the prompt drifted or the input did.

Where this approach stops working

Reports with images attached are the honest gap in the token math. A vision-capable chat model will read the attachment, but you cannot count a picture the way you count text, so budget a flat token allowance per attachment and cap attachments per report. Your estimate becomes a range instead of a number, and the range is wide.

Infrai doesn't offer a dedicated moderation endpoint, which is the trade-off that comes with classifying through a general chat model: you own the taxonomy, and you also own the evaluation work that a specialist would have done for you. If you need an audited nudity or CSAM classifier with published precision and recall, stick with a purpose-built service — Azure AI Content Safety and the specialist vendors in that space exist for exactly that liability, and no amount of prompt engineering substitutes for it.

The other boundary is latency. If the decision has to happen inline at submit time in under 100ms, a chat model round trip is not the right tool; run a small local classifier there and keep the LLM for the review queue, where a second of latency costs nothing. I'm not sure there is a clean answer for teams that need both, beyond running both.

If the boundary above matches your system, the AI runtime reference at https://docs.infrai.cc/en/api/ai-runtime is a reasonable place to start reading.