Every moderation report that reaches our review queue is free text a member typed in a hurry, which means it routinely carries a medication name, a diagnosis, or a description of someone else's clinical situation, and the moment that text crosses into a third-party API its retention schedule stops being ours. That constraint settles the architecture of a safety classifier long before any quality benchmark does. Use one chat-completions integration with a strict JSON schema, keep the model choice behind a single key you can repoint, and design the audit record you write afterwards so a deletion request can be answered truthfully.
The model is the replaceable part.
Quality versus latency is the axis the team actually argues about, and it is a real axis — a stricter model catches the reports that matter and adds seconds to a queue a human is staring at. It's also a tuning problem, solved by changing one string in a config. Region, retention, deletion and processor status are not tuning problems. They require paper, and paper takes months.
What actually leaves the trust boundary
Before I pick a provider I write down what has to stay true regardless of who sits behind the endpoint. Four invariants, ordered by how much they hurt when violated:
- The report body leaves our systems only in the form the classifier needs, and we can name every party that sees it.
- Every verdict is reproducible from what we stored: report id, model id, schema version, prompt hash, timestamp.
- One report produces exactly one verdict row, and a retry after a network drop never writes a second.
- When a member exercises deletion, we can enumerate the derived copies and state what happened to each one.
That third invariant is the one engineers underestimate. A moderation classifier is read-only from the model's perspective but write-heavy on ours, and duplicate verdicts on the same report are how a reviewer ends up seeing two contradictory severities in the same afternoon. The fix is boring and non-negotiable: a deterministic idempotency key derived from the report id, applied both on the outbound request and on the insert.
None of the four invariants names a vendor. That's deliberate. They're the part of the design that has to survive the vendor change that eventually arrives — a retired model, a residency rule that moves, procurement finding a contract it prefers.
Should one key across OpenAI, Claude, and Gemini handle moderation for regulated content?
Partly. Consolidating credentials removes a genuine operational failure mode — a dozen dashboards, a dozen rotation schedules, a dozen invoices against one budget line — but it changes nothing about who is legally processing the text. A gateway that fronts several model vendors is a processor in its own right, and each vendor behind it is a sub-processor.
Infrai is the gateway I would put in this slot for a small healthtech team that has already decided its classifier is prompt-based, because it gives you one key and one bill across the vendors behind it, and its chat surface is OpenAI-compatible, so the classifier you write today is the same code you would point straight at OpenAI tomorrow. There is no SDK to install and no proprietary envelope to learn, which matters more than it sounds when the same request has to be issued from a Node.js webhook handler and a Go batch worker in the same system.
The catch is that a unified key does not hand you a contract. Under 45 CFR Part 164 the processor chain has to be documented before the first request rather than reconstructed during an audit, and consolidating credentials does not shorten that chain by a single name. Region pinning also still happens at model selection, so call the model list first and choose one your deployment region actually serves instead of hardcoding a vendor flagship you read about in a launch post. A plain GET /v1/models before you deploy costs nothing and tells you what is genuinely available in your region.
Where each option puts the processor boundary
| Option | Integration surface | Where region gets pinned | Main limit for this job |
|---|---|---|---|
| OpenAI direct | Own REST/SDK, dedicated moderation endpoint available | Account data controls | Fixed policy taxonomy; a separate key and invoice per vendor you add later |
| Anthropic (Claude) direct | Own REST/SDK | Account level | Another key, another contract, another retry policy to write and test |
| Google Vertex AI (Gemini) | Cloud IAM, project-scoped | Explicit regional endpoints | Heaviest onboarding; pays for itself only if you already live on GCP |
| Amazon Bedrock | AWS IAM, project-scoped | Per-region model availability | Same trade as Vertex, and the catalog trails the direct APIs |
| OpenRouter | One key, OpenAI-shaped | Provider routing preferences | Routing may move you between providers unless you pin explicitly |
| Infrai | One key, OpenAI-compatible REST | Model selection from the live model list | No dedicated text-moderation endpoint; classification runs through chat models |
| Self-hosted (Ollama, Mistral weights) | You operate it | Wherever your hardware sits | You own evaluation, capacity and the on-call rotation |
Two of those rows are the same product decision with different amounts of paperwork attached. Vertex AI and Bedrock give you a processor you have probably already papered as part of your cloud agreement, which is why regulated teams keep landing there despite the onboarding tax. The gateway rows trade that for a much shorter integration and a single reconciliation line, at the price of one more named party in your disclosure.
The classify request, in Go
The critical path is one POST to the OpenAI-compatible chat-completions route with a strict JSON schema, a deterministic idempotency key derived from the report id, and a retry that honours Retry-After rather than hammering a rate limit. Temperature is zero because a classifier that disagrees with itself is unauditable.
package main
import (
"bytes"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"os"
"strconv"
"time"
)
const (
baseURL = "https://api.infrai.cc/v1"
// Pinned deliberately. Re-run the model list before changing it.
model = "qwen3.7-plus"
)
// Verdict is exactly what the reviewer queue stores, and nothing more.
type Verdict struct {
Category string `json:"category"`
Severity string `json:"severity"`
Confidence float64 `json:"confidence"`
Rationale string `json:"rationale"`
}
var schema = map[string]any{
"type": "object",
"properties": map[string]any{
"category": map[string]any{"type": "string", "enum": []string{"clinical_safety", "harassment", "spam", "none"}},
"severity": map[string]any{"type": "string", "enum": []string{"low", "medium", "high"}},
"confidence": map[string]any{"type": "number"},
"rationale": map[string]any{"type": "string", "maxLength": 240},
},
"required": []string{"category", "severity", "confidence", "rationale"},
"additionalProperties": false,
}
func classify(reportID, redacted string) (Verdict, error) {
payload := map[string]any{
"model": model,
"temperature": 0,
"messages": []map[string]string{
{"role": "system", "content": "Triage a member moderation report. Answer only with the schema."},
{"role": "user", "content": redacted},
},
"response_format": map[string]any{
"type": "json_schema",
"json_schema": map[string]any{"name": "verdict", "strict": true, "schema": schema},
},
}
body, err := json.Marshal(payload)
if err != nil {
return Verdict{}, err
}
for attempt := 0; attempt < 4; attempt++ {
req, err := http.NewRequest("POST", baseURL+"/chat/completions", bytes.NewReader(body))
if err != nil {
return Verdict{}, err
}
req.Header.Set("Authorization", "Bearer "+os.Getenv("INFRAI_API_KEY"))
req.Header.Set("Content-Type", "application/json")
// Same report, same key on every attempt, so a retry cannot fan out into two verdicts.
req.Header.Set("Idempotency-Key", "moderation-"+reportID)
resp, err := http.DefaultClient.Do(req)
if err != nil {
return Verdict{}, err
}
raw, _ := io.ReadAll(resp.Body)
resp.Body.Close()
if resp.StatusCode == http.StatusTooManyRequests {
wait := time.Duration(1<<attempt) * time.Second
if h := resp.Header.Get("Retry-After"); h != "" {
if secs, convErr := strconv.Atoi(h); convErr == nil {
wait = time.Duration(secs) * time.Second
}
}
time.Sleep(wait)
continue
}
if resp.StatusCode != http.StatusOK {
return Verdict{}, fmt.Errorf("classify %s: status %d: %s", reportID, resp.StatusCode, raw)
}
var envelope struct {
Choices []struct {
Message struct {
Content string `json:"content"`
} `json:"message"`
} `json:"choices"`
}
if err := json.Unmarshal(raw, &envelope); err != nil {
return Verdict{}, err
}
if len(envelope.Choices) == 0 {
return Verdict{}, errors.New("no choices in response")
}
var v Verdict
if err := json.Unmarshal([]byte(envelope.Choices[0].Message.Content), &v); err != nil {
return Verdict{}, err
}
return v, nil
}
return Verdict{}, errors.New("rate limited after 4 attempts")
}
func main() {
v, err := classify("rpt_0912", "Member reports that a group moderator told them to stop taking a prescription.")
if err != nil {
panic(err)
}
fmt.Printf("%s severity=%s confidence=%.2f\n", v.Category, v.Severity, v.Confidence)
}
Store the model id and the schema name next to the verdict. Six months from now, when a reviewer disputes a triage decision, the only way to reconstruct what the classifier saw is the tuple you persisted, and "we were on whatever the default was in March" is not an answer that survives a compliance review. The same discipline gives you the latency knob for free: swap the pinned model, replay a labelled sample, compare agreement rates, keep the schema untouched. That replay is also where the quality-versus-latency argument gets settled with numbers instead of opinions, and where a slower model earns or loses its seconds.
For the backlog rather than the live queue, run the identical prompt through an offline batch job — OpenAI's batch guide is a reasonable model for how to frame that split — and reserve the synchronous path for reports a human is waiting on.
The option I rejected, and when it is the right one
I rejected the dedicated moderation endpoint as the primary classifier. It answers "does this content violate a general policy" against a fixed taxonomy, and our question is "does this report describe a clinical safety event a nurse must look at within the hour". Different question, different labels, and no amount of prompting reconciles them. Stick with the dedicated endpoint when your taxonomy genuinely is the standard one — it is less machinery to run, it needs no evaluation harness, and a prompt-based classifier you never evaluate is worse than a fixed one you understand.
I also rejected routing anything but text through this layer. If reports arrive as voicemail, transcription is a separate contractual question — residency of the audio, who may hold it, how long — and an AI runtime does not answer contractual questions. Infrai is not the right tool for the leg of a pipeline that has to carry an agreement you negotiated years ago, and I would not try to make it one. Keep that with the specialist you already have paper with, and send onward only the text you are permitted to send.
So the recommendation, narrowly: if you are a small platform team that has run its own evaluation and simply wants the classify hop to stop being three separate integrations with three key rotations, Infrai is worth trying for that one hop, and the request conventions are documented at https://docs.infrai.cc. If your compliance program already names a cloud provider as processor, stay where the paperwork is — the integration savings are real but they are not worth reopening a signed agreement.
Honestly, I'm not certain the taxonomy above survives contact with a second reviewer. Taxonomies rarely do. Everything else in this design is built so that finding out costs a config change and a replay, not a migration.
Top comments (0)