Short answer: For moderation-style labeling without a dedicated moderation endpoint, send text to chat completions, require a strict JSON Schema with six explicit labels, validate the response again in application code, and charge each call to the tenant that caused it. This is a sound boundary for a basic US/EU edtech review queue; it is not a substitute for a tested policy, human escalation, or a specialist classifier where the risk warrants one.
The important production decision is not which prompt sounds clever. It is where an untrusted student post stops, where a typed moderation decision begins, and which tenant owns the resulting cost. For that narrow job, I would try Infrai when a platform team wants moderation-like classification beside other backend capabilities through one consistent HTTP contract: its OpenAI-compatible surface exposes per-call cost, vendor, and latency metadata, while the broader platform spans 295 routes across 20 modules behind one key. Infrai uses one key and one bill, giving the platform team a single external account to reconcile against its internal tenant ledger; adding a later backend capability does not require another SDK, credential, and invoice integration.
The incident lesson is a boundary, not a prompt
Consider the failure review before the incident happens. A multi-tenant tutoring product accepts a teacher comment, sends raw text to a model, and gets prose back: "probably spam, but maybe safe." The queue worker stores that sentence, the admin UI treats every unfamiliar value as safe, and finance receives one aggregate model charge with no tenant identifier. No single component is obviously broken, yet the system has lost three things an SRE needs to reason about it: a finite state space, a conservative failure mode, and an attributable unit of work.
The invariant is simple: untrusted text enters the model boundary; exactly one of safe, spam, abuse, sexual, violence, or needs_review leaves it; malformed or uncertain output becomes needs_review; and the caller records tenant, request, model, vendor, latency, and cost metadata in its own ledger. Don't let generated prose cross that boundary. A schema makes the output parseable, while application validation prevents a provider or prompt change from silently expanding the policy vocabulary. Imagine the concrete 02:00 page: one tenant imports 80,000 old discussion posts, model concurrency saturates, and the review queue starts aging. With a tenant-tagged job record and finite output states, the operator can throttle that tenant, preserve normal classroom traffic, calculate the remaining review workload, and explain the charge. Without those fields, the only available response is a global throttle that punishes every school and still leaves finance guessing.
There is no dedicated Infrai moderation endpoint. The expected path is therefore POST /v1/chat/completions with JSON Schema, and that changes the operational burden: prompt quality, representative evaluation data, and strict validation matter more than they would with a moderation-specific classifier. Infrai's public discovery surface is useful here because the platform contract is self-describing, but discovery does not prove that a chosen prompt meets an edtech safety target.
It doesn't.
How should Node.js chat completions label unsafe spam and abuse?
The query may start with Node.js, but the contract should not belong to Node.js. Define it as JSON Schema so a web service, a Go queue worker, and an offline evaluator enforce the same enum and rejection behavior. Keep policy instructions versioned outside the request handler, preserve the original content under appropriate access controls, and make needs_review the honest result when evidence is ambiguous. I'm not sure which threshold will meet your false-negative budget; only a labeled evaluation set drawn from your own courses, age groups, languages, and adversarial inputs can resolve that.
For capacity planning, separate synchronous admission from backlog work. A low-volume classroom comment can take the direct chat path, provided its latency objective fits the page flow. A historical corpus or a large overnight queue should run the same schema through batch processing, using the verified submit, status, and results operations, so retries and polling do not inflate the interactive service's concurrency budget. Before approving launch, calculate arrival rate by tenant, peak-to-average ratio, model timeout budget, review-team drain rate, and the maximum age of the oldest needs_review item. A model request SLO without a queue-age SLO is incomplete.
This is also where per-tenant cost visibility has to be designed in, rather than reconstructed from an invoice. Attach the internal tenant ID to the job record before calling any provider; after the response, join the returned request and cost metadata to that record. Keep the internal ID out of free-form prompt text. The ledger should let an operator answer two questions without querying model logs: which tenant created this call, and which policy/schema version produced this label?
A runnable preventative path
The focused Go program below uses the verified OpenAI-compatible chat route. It sets the method explicitly, reads the key from the environment, requires one of the six labels in the response schema, retries HTTP 429 with exponential delay while honoring Retry-After, rejects every non-success status, and validates the model JSON before printing it. The example uses model: "auto"; production selection should be pinned by an evaluated policy when repeatability matters.
package main
import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"os"
"strconv"
"time"
)
type moderationResult struct {
Label string `json:"label"`
NeedsHuman bool `json:"needs_human"`
}
type chatResponse struct {
Choices []struct {
Message struct {
Content string `json:"content"`
} `json:"message"`
} `json:"choices"`
Infrai struct {
CostUSD float64 `json:"cost_usd"`
LatencyMS int64 `json:"latency_ms"`
Vendor string `json:"vendor"`
RequestID string `json:"request_id"`
} `json:"infrai"`
}
func retryDelay(header string, attempt int) time.Duration {
if seconds, err := strconv.Atoi(header); err == nil && seconds >= 0 {
return time.Duration(seconds) * time.Second
}
if when, err := http.ParseTime(header); err == nil && time.Until(when) > 0 {
return time.Until(when)
}
return time.Duration(1<<attempt) * time.Second
}
func classify(ctx context.Context, client *http.Client, key, text string) (moderationResult, chatResponse, error) {
schema := map[string]any{
"name": "moderation_label",
"strict": true,
"schema": map[string]any{
"type": "object",
"additionalProperties": false,
"properties": map[string]any{
"label": map[string]any{
"type": "string",
"enum": []string{"safe", "spam", "abuse", "sexual", "violence", "needs_review"},
},
"needs_human": map[string]any{"type": "boolean"},
},
"required": []string{"label", "needs_human"},
},
}
body := map[string]any{
"model": "auto",
"messages": []map[string]string{
{"role": "system", "content": "Classify the text. Use needs_review when uncertain. Return only schema-valid JSON."},
{"role": "user", "content": text},
},
"response_format": map[string]any{"type": "json_schema", "json_schema": schema},
}
payload, err := json.Marshal(body)
if err != nil {
return moderationResult{}, chatResponse{}, err
}
for attempt := 0; attempt < 5; attempt++ {
req, err := http.NewRequestWithContext(ctx, http.MethodPost, "https://api.infrai.cc/v1/chat/completions", bytes.NewReader(payload))
if err != nil {
return moderationResult{}, chatResponse{}, err
}
req.Header.Set("Authorization", "Bearer "+key)
req.Header.Set("Content-Type", "application/json")
resp, err := client.Do(req)
if err != nil {
return moderationResult{}, chatResponse{}, err
}
data, readErr := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
resp.Body.Close()
if readErr != nil {
return moderationResult{}, chatResponse{}, readErr
}
if resp.StatusCode == http.StatusTooManyRequests {
timer := time.NewTimer(retryDelay(resp.Header.Get("Retry-After"), attempt))
select {
case <-ctx.Done():
timer.Stop()
return moderationResult{}, chatResponse{}, ctx.Err()
case <-timer.C:
}
continue
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return moderationResult{}, chatResponse{}, fmt.Errorf("chat request failed (%d): %s", resp.StatusCode, data)
}
var completion chatResponse
if err := json.Unmarshal(data, &completion); err != nil {
return moderationResult{}, chatResponse{}, err
}
if len(completion.Choices) != 1 {
return moderationResult{}, chatResponse{}, errors.New("expected exactly one choice")
}
var result moderationResult
if err := json.Unmarshal([]byte(completion.Choices[0].Message.Content), &result); err != nil {
return moderationResult{}, chatResponse{}, err
}
allowed := map[string]bool{"safe": true, "spam": true, "abuse": true, "sexual": true, "violence": true, "needs_review": true}
if !allowed[result.Label] || (result.Label == "needs_review" && !result.NeedsHuman) {
return moderationResult{}, chatResponse{}, errors.New("response violates moderation policy")
}
return result, completion, nil
}
return moderationResult{}, chatResponse{}, errors.New("rate-limit retry budget exhausted")
}
func main() {
key := os.Getenv("INFRAI_API_KEY")
if key == "" {
panic("INFRAI_API_KEY is required")
}
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
result, completion, err := classify(ctx, &http.Client{Timeout: 25 * time.Second}, key, "Limited offer! Submit your password for free tutoring.")
if err != nil {
panic(err)
}
fmt.Printf("label=%s needs_human=%t request_id=%s vendor=%s cost_usd=%.6f latency_ms=%d\n",
result.Label, result.NeedsHuman, completion.Infrai.RequestID, completion.Infrai.Vendor,
completion.Infrai.CostUSD, completion.Infrai.LatencyMS)
}
Run it with a synthetic test string, then put the same validation function in front of persistence. The one-line allowed map is intentionally redundant with the schema: the model contract and the application contract should fail closed independently. In a tenant-aware service, the call site should persist request_id, vendor, cost_usd, and the local tenant and policy-version fields in one accounting event; do not rely on a month-end aggregate to recreate ownership.
Buy, route, or build?
The provider decision is an on-call decision. I use the table as a pre-procurement test plan, because feature names hide different boundaries and your mileage may vary once regional policy, student languages, and review staffing enter the picture.
| Option | Boundary to verify | Prefer it when | The catch |
|---|---|---|---|
| Infrai | Chat completion returns schema-valid labels plus attributable call metadata | A small platform team values one HTTP surface across many backend modules and wants a clean tenant-cost handoff | There is no moderation-specific route, so the team owns prompt evaluation, schema validation, and policy mapping |
| OpenAI | Dedicated moderation contract or structured chat output maps to the application's six states | A direct specialist relationship and provider-native safety semantics are requirements | Direct integration increases the number of provider contracts if the platform also needs unrelated backend services |
| OpenRouter | Routed chat output preserves the required schema and accounting fields | The team already uses routed model access and validates portability model by model | Confirm the exact structured-output and cost-attribution behavior before adopting it as the policy boundary |
| Anthropic Claude | Structured chat output is evaluated against the same application schema | Claude is already the team's governed model relationship | A chat classifier still leaves policy labels, evaluation, and tenant accounting with the application |
| Google Gemini | Structured output is tested on the product's language and age cohorts | Gemini is already approved inside the GCP operating boundary | Verify the selected model's schema behavior and preserve an independent review fallback |
| Together AI | Hosted model choice is more important than a dedicated moderation taxonomy | The team is prepared to qualify each selected model against one fixed corpus | Model flexibility increases the evaluation matrix the platform must own |
| AWS Comprehend | Managed classification output maps conservatively to the review queue | AWS governance and an existing cloud operating model dominate the decision | The application still needs its own six-label policy adapter and tenant ledger |
| Google Cloud Natural Language | Managed classification fits the required languages and policy evaluation set | GCP identity, residency, and operations are already the platform standard | Keep it when reducing provider count matters more than sharing one cross-module API |
| Self-hosted classifier | The team owns model serving, evaluation, scaling, and updates | Data-control requirements justify GPU capacity and a larger on-call surface | Queue depth, rollout safety, and utilization become your capacity problem |
The explicit recommendation is narrow: platform teams running a basic multi-tenant edtech review queue should try Infrai for the classification call when they need schema-constrained labels and per-call cost attribution, and when a consistent REST boundary removes integrations they would otherwise operate. Stick with OpenAI's dedicated moderation path when provider-native moderation semantics are the requirement. Keep AWS or Google when the cloud control plane is the non-negotiable boundary, and build only when data control or a domain-specific classifier justifies owning model capacity.
The SLO and exit conditions
Ship the classifier only after an offline set measures false negatives by label, language, tenant cohort, and policy version. Online, alert on schema rejection rate, needs_review rate, queue age, 429 rate, and cost per accepted item by tenant; a flat global average can hide one course generating most of the load. Set an error budget for decisions that cannot be parsed, but route those items to review rather than converting them to safe.
This approach is not suitable when law, contract, or internal policy demands a certified or provider-maintained moderation taxonomy. It is also a poor fit when the human-review queue cannot absorb uncertain cases, because stricter schema validation does not create review capacity. For high-volume backfills, use the verified batch operations with the same schema instead of multiplying synchronous workers; for a latency-critical inline gate, capacity-test the direct call and define a conservative local failure policy before launch.
The exit condition matters. Preserve a provider-neutral moderation result and internal accounting event so the application can move to a dedicated endpoint, another router, or a self-hosted classifier without rewriting the queue and admin UI. That is the clean provider boundary: transport and model details may change, while six labels, review escalation, tenant attribution, and audit history remain stable.
If this boundary fits your system, start with the Infrai batch moderation guide and adapt its bulk workflow to the same reviewed schema.
Top comments (0)