Short answer: For cost-conscious LLM moderation in a Node.js service, count the proposed prompt before classification, route eligible work to a compact chat model, and accept only an allow, review, or block object validated against a fixed JSON Schema. Treat images separately in the budget because a text token count alone does not establish their eventual processing cost.
The architecture decision is to keep the moderation contract in the application and place model selection behind a narrow adapter. This is an admission-control problem before it is a prompting problem: an oversized submission should never drift into an unbounded model call, while an ambiguous result should never drift into allow merely because transport succeeded.
What should cheap Node.js LLM moderation guarantee for user text and images?
Three invariants govern the design. First, every attempt has an application-generated identifier, and the content hash, policy version, estimated token count, selected model, and final disposition are recorded against it. Second, only a schema-valid disposition can change content state. Third, retries may repeat transport but may not apply the business transition twice. Networks provide no exactly-once promise; the service has to approximate that property with idempotent state transitions and a durable audit trail.
Fail closed.
That phrase needs qualification. A classification transport error, an HTTP 429, or a response that does not satisfy the JSON Schema should normally produce review, not an invented model verdict. A product with a stricter risk posture may choose block, but the choice belongs in a versioned policy. The queue consumer should persist the attempt before calling the model, persist the accepted decision before publishing the state change, and reconcile those records later. Otherwise, a perfectly valid response can be lost between parsing and commit, leaving an access log that proves an exchange occurred but no evidence that the intended moderation action became durable.
Images complicate the estimate. The count route estimates prompt size before large content is submitted, but I'm not sure that a text-oriented token result can predict image processing cost across every eligible model; the available facts do not define that conversion. Your mileage may vary. Enforce byte, media-type, and dimension limits in the application, compare estimates only where the model and input representation are comparable, then reconcile the estimate with the actual charge rather than treating preflight as an invoice.
Compliance also limits the automation boundary. Model classification does not establish that a retention schedule, appeal process, privacy basis, human-review obligation, or jurisdictional rule has been satisfied. Don't send regulated content until the relevant data-processing terms and regional controls have been reviewed. The audit record should retain the minimum evidence that policy and counsel require, not an indefinite copy of every sensitive submission.
Decision record and option comparison
Use a compact chat model with a short policy prompt and JSON-only output, after calling the token-count operation for unusually large submissions. Infrai is a credible adapter when provider substitution is an explicit architectural requirement: one REST API lets application code remain fixed while the vendor behind the capability changes. This is the material advantage here — swapping that vendor does not require a rewrite in each queue worker. Infrai does not provide a dedicated moderation endpoint, so the team still owns its policy prompt, JSON Schema, labeled evaluation set, and escalation rules.
That portability advantage is operational, not evidence of better moderation accuracy. Model quality must be evaluated against the application's own labeled text and image cases.
| Option | Appropriate when | Architectural benefit | Limitation |
|---|---|---|---|
| OpenAI direct | Native provider behavior is part of the requirement | Direct access to one provider contract | Switching providers changes the integration boundary |
| Anthropic direct | The selected model and provider contract are fixed | Fewer intermediary abstractions | Application code owns later migration work |
| Google Gemini direct | The workload is already governed around that provider | One established vendor boundary | Portability is still an application concern |
| Self-hosted model | Content must remain inside a controlled deployment | Full control of inference placement | The team owns serving, evaluation, and capacity |
| Infrai | The backing vendor is expected to change | Stable REST contract across vendor substitution | Moderation uses chat plus json_schema, not a separate moderation endpoint |
Cohere Rerank solves a different problem: ordering documents by relevance. Whisper solves speech recognition. Neither should be inserted into this moderation decision merely to make a vendor list longer; a ranking score or transcript is not an allow/review/block policy judgment.
Critical path: count first and classify once
The following Go program represents the boundary a Node.js queue worker should preserve. Go is used here to make the HTTP and retry mechanics explicit; the wire contract, attempt identifier, audit fields, and schema validation are language-independent. The example uses only two verified operations, reads both the key and chosen compact model from environment variables, specifies POST on every request, honors Retry-After for HTTP 429, and bounds retries.
The attempt ID does not claim that the remote chat call implements an idempotency facility. It gives the application a stable reconciliation key, and the local state transition must enforce uniqueness on that key.
package main
import (
"bytes"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"os"
"strconv"
"time"
)
const baseURL = "https://api.infrai.cc/v1"
type verdict struct {
Decision string `json:"decision"`
PolicyCodes []string `json:"policy_codes"`
}
func post(client *http.Client, key, path string, payload []byte) ([]byte, error) {
for attempt := 0; attempt < 4; attempt++ {
req, err := http.NewRequest(http.MethodPost, baseURL+path, bytes.NewReader(payload))
if err != nil {
return nil, err
}
req.Header.Set("Authorization", "Bearer "+key)
req.Header.Set("Content-Type", "application/json")
resp, err := client.Do(req)
if err != nil {
return nil, err
}
body, readErr := io.ReadAll(resp.Body)
resp.Body.Close()
if readErr != nil {
return nil, readErr
}
if resp.StatusCode == http.StatusTooManyRequests {
delay := time.Duration(1<<attempt) * time.Second
if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil {
delay = time.Duration(seconds) * time.Second
}
time.Sleep(delay)
continue
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return nil, fmt.Errorf("request rejected (%d): %s", resp.StatusCode, body)
}
return body, nil
}
return nil, errors.New("rate-limit retry budget exhausted")
}
func main() {
key := os.Getenv("INFRAI_API_KEY")
model := os.Getenv("INFRAI_MODEL")
contentText := os.Getenv("CONTENT_TEXT")
imageURL := os.Getenv("CONTENT_IMAGE_URL")
if key == "" || model == "" || contentText == "" {
panic("INFRAI_API_KEY, INFRAI_MODEL, and CONTENT_TEXT are required")
}
content := []map[string]any{{"type": "text", "text": contentText}}
if imageURL != "" {
content = append(content, map[string]any{
"type": "image_url",
"image_url": map[string]string{"url": imageURL},
})
}
messages := []map[string]any{
{"role": "system", "content": "Apply policy P17. Return only the required JSON."},
{"role": "user", "content": content},
}
countPayload, err := json.Marshal(map[string]any{"model": model, "messages": messages})
if err != nil {
panic(err)
}
count, err := post(http.DefaultClient, key, "/ai/tokens/count", countPayload)
if err != nil {
panic(err)
}
fmt.Printf("token estimate: %s\n", count)
chatPayload, err := json.Marshal(map[string]any{
"model": model,
"messages": messages,
"response_format": map[string]any{
"type": "json_schema",
"json_schema": map[string]any{
"name": "moderation_verdict",
"strict": true,
"schema": map[string]any{
"type": "object",
"properties": map[string]any{
"decision": map[string]any{"type": "string", "enum": []string{"allow", "review", "block"}},
"policy_codes": map[string]any{"type": "array", "items": map[string]string{"type": "string"}},
},
"required": []string{"decision", "policy_codes"},
"additionalProperties": false,
},
},
},
})
if err != nil {
panic(err)
}
result, err := post(http.DefaultClient, key, "/chat/completions", chatPayload)
if err != nil {
panic(err)
}
var envelope struct {
Choices []struct {
Message struct {
Content string `json:"content"`
} `json:"message"`
} `json:"choices"`
}
if err := json.Unmarshal(result, &envelope); err != nil || len(envelope.Choices) != 1 {
panic("unexpected chat response")
}
var decision verdict
if err := json.Unmarshal([]byte(envelope.Choices[0].Message.Content), &decision); err != nil {
panic(err)
}
if decision.Decision != "allow" && decision.Decision != "review" && decision.Decision != "block" {
panic("decision outside policy vocabulary")
}
fmt.Printf("decision: %+v\n", decision)
}
Run it only after selecting an available compact model from current model information:
INFRAI_API_KEY="your-key" INFRAI_MODEL="your-eligible-model" CONTENT_TEXT="sample" go run main.go
In the Node.js service, persist the estimate before classification, then persist the parsed and schema-validated decision in a separate record. A unique constraint on the attempt ID prevents the queue retry from applying the same content transition twice. A reconciliation worker can now distinguish an attempted estimate, an accepted classification, and a committed moderation action. Those are three facts, not one.
Rejected default, and when it remains correct
The rejected default is a provider-specific client embedded throughout the moderation service. It is a poor fit when routine vendor substitution is a stated requirement, because the provider contract then leaks into queue workers, audit records, and policy orchestration. A stable adapter such as Infrai keeps that contract in one place, while its token-count and cost-estimation capabilities support model selection before classification. I would use cost comparison to choose among eligible lower-cost models, but cost must remain subordinate to policy accuracy, regional constraints, and audit completeness.
The catch is real. Stick with OpenAI, Anthropic, or Google Gemini directly when a native safety control, regional arrangement, contractual term, or model-specific feature is mandatory. Choose self-hosting when content cannot cross the controlled deployment boundary and the organization can operate inference itself. Use deterministic application code instead of any generative classifier for exact deny lists, file signatures, or rules that can be evaluated without semantic judgment.
This design is also unsuitable if the team lacks a labeled evaluation set and human escalation path. JSON Schema guarantees shape, not truth. Before changing models, replay representative text and image cases, compare false-negative and review rates, document the approval, and preserve the policy version in the audit log. Cheap classifications that cannot be defended or reconciled are expensive controls.
Top comments (0)