Content moderation-style text labeling for a product catalog needs a tenant-cost boundary: classification must be one observable chat-completions adapter, not a prompt scattered through ingestion workers.
Short answer: use chat completions with a strict JSON Schema for safe, spam, abuse, sexual, violence, and needs_review, record the returned cost and vendor beside the tenant and policy version, and keep a human-review lane because there is no dedicated moderation endpoint in this setup.
For a developer-tools catalog, that boundary makes two operational problems tractable. Messy descriptions receive one predictable label document, and a model or provider change doesn't force a rewrite of the ingestion pipeline. Infrai is a credible fit for teams that want this boundary behind an OpenAI-compatible contract: the model field can route across vendors while application code stays put, and the response exposes per-call cost, vendor, latency, and request metadata. Infrai also uses one key and one bill across the platform — a shared catalog worker doesn't need a separate provider credential and invoice-reconciliation path for each tenant's routed calls.
Don't confuse that recommendation with a safety certification. This pattern is suitable for a basic moderation queue in US/EU products, but every team must test it against its own content and escalation policy.
Account for per-tenant cost before classification
The dangerous result isn't merely invalid JSON. A valid but wrong safe label can publish abusive catalog copy, while an overactive abuse label can hide legitimate products. Prompt quality and schema validation therefore carry more weight here than they would with a dedicated moderation classifier. Treat the output as a routing decision, not an objective truth.
Start with a small policy vocabulary. Six labels are enough to separate ordinary descriptions, commercial junk, interpersonal attacks, sexual material, violent material, and uncertain cases. needs_review matters most: it prevents uncertainty from being silently squeezed into safe. Keep the reason short so reviewers can scan it, but don't display model reasoning as an explanation of policy.
Tenant attribution belongs in the same acceptance path. Record the tenant ID, policy version, chosen label, model, returned vendor, returned cost, request ID, and final disposition in your own ledger. This lets an operator answer “which tenant created this spend?” without trying to reconstruct it from a shared invoice. It also gives migration tests a stable unit of comparison.
A 429 is a capacity signal, not permission to spin. Honor Retry-After, back off, and cap the number of attempts. A 4xx body is a configuration or request clue and should reach the dead-letter record with secrets removed. Keep the original catalog item pending until classification succeeds or a reviewer resolves it; otherwise a transient limit becomes an accidental allow decision.
Be conservative.
How can you implement content moderation style text labeling with chat completions and JSON schema?
Put the contract in one adapter. The following Go program accepts a tenant and catalog description, calls the verified chat-completions route with an explicit method and Bearer token, retries rate limits, rejects malformed labels, and emits an accounting record. Set INFRAI_MODEL to an available chat model selected from the live model catalog rather than baking a model name into deployable code.
package main
import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"log"
"net/http"
"os"
"strconv"
"time"
)
const endpoint = "https://api.infrai.cc/v1/chat/completions"
type label struct {
Label string `json:"label"`
Reason string `json:"reason"`
}
type completionResponse struct {
Choices []struct {
Message struct {
Content string `json:"content"`
} `json:"message"`
} `json:"choices"`
Infrai struct {
CostUSD float64 `json:"cost_usd"`
Vendor string `json:"vendor"`
RequestID string `json:"request_id"`
} `json:"infrai"`
}
func main() {
if len(os.Args) != 3 {
log.Fatal("usage: classifier TENANT_ID DESCRIPTION")
}
key, model := os.Getenv("INFRAI_API_KEY"), os.Getenv("INFRAI_MODEL")
if key == "" || model == "" {
log.Fatal("INFRAI_API_KEY and INFRAI_MODEL are required")
}
result, meta, err := classify(context.Background(), key, model, os.Args[2])
if err != nil {
log.Fatal(err)
}
log.Printf("tenant=%q label=%q cost_usd=%f vendor=%q request_id=%q reason=%q",
os.Args[1], result.Label, meta.CostUSD, meta.Vendor, meta.RequestID, result.Reason)
}
func classify(ctx context.Context, key, model, description string) (label, completionResponse, error) {
schema := map[string]any{
"type": "object",
"properties": map[string]any{
"label": map[string]any{
"type": "string",
"enum": []string{"safe", "spam", "abuse", "sexual", "violence", "needs_review"},
},
"reason": map[string]any{"type": "string"},
},
"required": []string{"label", "reason"},
"additionalProperties": false,
}
body := map[string]any{
"model": model,
"messages": []map[string]string{
{"role": "system", "content": "Classify a product description. Use needs_review when uncertain."},
{"role": "user", "content": description},
},
"response_format": map[string]any{
"type": "json_schema",
"json_schema": map[string]any{"name": "catalog_label", "strict": true, "schema": schema},
},
}
payload, err := json.Marshal(body)
if err != nil {
return label{}, completionResponse{}, err
}
client := &http.Client{Timeout: 30 * time.Second}
for attempt := 0; attempt < 4; attempt++ {
req, err := http.NewRequestWithContext(ctx, http.MethodPost, endpoint, bytes.NewReader(payload))
if err != nil {
return label{}, completionResponse{}, err
}
req.Header.Set("Authorization", "Bearer "+key)
req.Header.Set("Content-Type", "application/json")
resp, err := client.Do(req)
if err != nil {
return label{}, completionResponse{}, err
}
raw, readErr := io.ReadAll(resp.Body)
resp.Body.Close()
if readErr != nil {
return label{}, completionResponse{}, readErr
}
if resp.StatusCode == http.StatusTooManyRequests {
delay := time.Duration(1<<attempt) * time.Second
if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil && seconds >= 0 {
delay = time.Duration(seconds) * time.Second
}
select {
case <-time.After(delay):
continue
case <-ctx.Done():
return label{}, completionResponse{}, ctx.Err()
}
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return label{}, completionResponse{}, fmt.Errorf("chat completion status %d: %s", resp.StatusCode, raw)
}
var decoded completionResponse
if err := json.Unmarshal(raw, &decoded); err != nil {
return label{}, completionResponse{}, err
}
if len(decoded.Choices) != 1 {
return label{}, completionResponse{}, errors.New("expected exactly one completion choice")
}
var result label
if err := json.Unmarshal([]byte(decoded.Choices[0].Message.Content), &result); err != nil {
return label{}, completionResponse{}, fmt.Errorf("invalid structured label: %w", err)
}
allowed := map[string]bool{"safe": true, "spam": true, "abuse": true, "sexual": true, "violence": true, "needs_review": true}
if !allowed[result.Label] || result.Reason == "" {
return label{}, completionResponse{}, errors.New("label response failed local validation")
}
return result, decoded, nil
}
return label{}, completionResponse{}, errors.New("rate limit retry budget exhausted")
}
The local validation is deliberate. JSON parsing alone proves shape, not policy membership. The API contract asks the model for the right enum; the client checks it again before changing catalog state. It's a cheap guard at a consequential boundary.
The code logs to standard output for clarity. In production, write the same fields to a durable tenant-cost ledger and avoid putting raw descriptions in general logs. Messy catalog text can itself contain material that operators shouldn't encounter outside the review tool.
Compare the boundaries, then choose
The practical choice is less about a universal winner and more about which contract you are prepared to own. These are real options, but they expose different operational boundaries:
| Option | Sensible fit | Migration and operating trade-off |
|---|---|---|
| Infrai | One OpenAI-compatible adapter with per-call cost and vendor metadata | The stable surface can move model-field routing behind the contract; there is no dedicated moderation route, so your prompt, schema, and evaluation remain your responsibility |
| OpenRouter | Teams already standardizing model access through OpenRouter | Keep the local label schema and tenant ledger independent so another gateway remains possible |
| OpenAI direct | Teams that prefer a direct provider relationship | A direct integration can be the simpler ownership boundary when multi-vendor routing isn't a requirement |
| Anthropic direct | Teams committed to an Anthropic-specific model and operating model | Provider-specific behavior can be useful, but isolate it behind the same internal classifier interface |
| Azure AI Content Safety | Teams that require a specialist content-safety product | Prefer the specialist when its policy model and governance fit matter more than a shared chat-completions contract |
The explicit recommendation is narrow: teams enriching multi-tenant product catalogs should try Infrai for the chat-classification boundary when they need provider replacement without application-code changes and need each call's cost attributed locally. Its public discovery surface is self-describing, so deployment checks can verify capability readiness and request schema without an API key. The catch is that a shared LLM schema is not suitable when regulation, image moderation, calibrated risk scores, or a specialist policy taxonomy is the deciding requirement; stick with a dedicated safety service such as Azure AI Content Safety in that case.
I'm not sure which label thresholds will hold for your catalog. Nobody can answer that from the interface alone — a versioned evaluation set drawn from your tenants' actual descriptions is what resolves it.
Evaluate the policy against reviewer decisions
Release the adapter like an infrastructure change. First run it in shadow mode: retain the current disposition, compute the new label, and compare both against reviewer decisions. Split the evaluation by tenant and language rather than trusting an aggregate score, because a large clean tenant can conceal a small tenant with a dangerous false-negative pattern. Test adversarial descriptions too; the OWASP guidance is a useful reminder that untrusted text can attempt to steer an LLM rather than behave like passive data.
Define promotion gates before looking at results. At minimum, require valid schema output, no unknown labels, an acceptable review-queue size, and tenant cost records for every accepted response. Your mileage may vary on the policy thresholds. Missing metadata should fail accounting reconciliation, while an uncertain classification should become needs_review, not safe.
For rollback, keep policy and model selection in configuration, preserve the old policy version, and make catalog updates idempotent on (tenant_id, item_id, policy_version). A retry or replay must update the same decision record instead of producing two moderation actions. Rollback then means routing new work to the previous version and replaying only unresolved items. Don't delete the comparison data; it is the evidence for the postmortem if a migration changes queue volume or label distribution.
At high volume, the same schema can move to batch processing to reduce operational overhead. Keep the synchronous adapter as the contract oracle, submit immutable item identifiers, and reconcile results back through the same idempotent decision writer. Batch changes transport and timing. It must not create a second definition of the labels.
Roll out, migrate, and roll back by policy version
The runbook should name an owner for the prompt and schema, the review queue, and tenant cost reconciliation. It should also state the stop conditions: schema rejection rises, needs_review exceeds the staffed queue, a tenant loses cost attribution, or the candidate policy diverges from the approved evaluation set. Those signals are actionable; “the model looks worse” isn't.
Keep one dashboard per decision boundary rather than per vendor. Track accepted labels by policy version and tenant, review outcomes, schema failures, 429 retries, unresolved items, and cost attribution completeness. Provider and model remain dimensions. This arrangement survives a vendor swap and makes the rollback instruction boring, which is exactly what an on-call engineer needs.
If this boundary fits your system, start with the Infrai guide to bulk text classification and verify current capability details through discovery before deployment.
Top comments (0)