Short answer: put one OpenAI-compatible chat integration in front of OpenAI, Claude, and Gemini, require a strict JSON Schema response, and choose from models available in the deployment region instead of baking a provider-specific model into the application. This keeps the safety classifier portable while leaving room to switch providers later.
The data flow is small enough to draw in one sentence: raw user content enters a dedicated moderation prompt, the selected chat model returns a schema-checked decision, and application policy decides whether to allow, review, or block the content. The model classifies; the application enforces. Keep those jobs separate.
It is a practical fit for a startup that wants fallback options or a gradual provider migration without maintaining three moderation implementations. The catch is important: this is prompted moderation through chat completions, not a dedicated moderation endpoint, so eval coverage and structured-output consistency carry more weight than provider-specific features.
How can one API key keep moderation structured across OpenAI, Claude, and Gemini?
Treat the provider as runtime configuration and the classifier contract as source code. The contract below has only four outputs: a boolean, a constrained action, a constrained category list, and a short rationale. A small schema is easier to evaluate than an ambitious taxonomy whose edge cases nobody has labeled.
Start by listing models available in the target US or EU deployment. Don't hardcode a provider-branded model from a blog post: model readiness can differ by deployment, and the useful choice is one the runtime actually reports as available. Select a model through configuration after that discovery step, then send the exact same messages and response schema on every call.
This is also where I draw a hard notebook-to-prod line. A notebook can print a plausible label. Production needs to reject malformed output, preserve the input-to-policy boundary, and record enough evaluation data to detect drift. No shortcuts.
A runnable Python safety classifier
The example uses an OpenAI-compatible client because the compatibility layer is the portability boundary. Set UNIFIED_AI_BASE_URL, UNIFIED_AI_API_KEY, and MODERATION_MODEL from the model list for your deployment. The client sends Bearer authentication, checks API failures, and automatically retries transient failures including rate limits; its retry policy can be tightened to match the surrounding request budget.
import json
import os
from typing import Any
from openai import APIStatusError, OpenAI
BASE_URL = os.environ["UNIFIED_AI_BASE_URL"].rstrip("/")
API_KEY = os.environ["UNIFIED_AI_API_KEY"]
MODEL = os.environ["MODERATION_MODEL"]
client = OpenAI(
base_url=BASE_URL,
api_key=API_KEY,
max_retries=4,
timeout=20.0,
)
DECISION_SCHEMA: dict[str, Any] = {
"name": "content_safety_decision",
"strict": True,
"schema": {
"type": "object",
"additionalProperties": False,
"properties": {
"allowed": {"type": "boolean"},
"action": {
"type": "string",
"enum": ["allow", "review", "block"],
},
"categories": {
"type": "array",
"items": {
"type": "string",
"enum": [
"safe",
"harassment",
"hate",
"self_harm",
"sexual",
"violence",
],
},
"minItems": 1,
"uniqueItems": True,
},
"rationale": {"type": "string", "maxLength": 240},
},
"required": ["allowed", "action", "categories", "rationale"],
},
}
def classify_content(text: str) -> dict[str, Any]:
if not text.strip():
raise ValueError("text must not be empty")
try:
response = client.chat.completions.create(
model=MODEL,
messages=[
{
"role": "system",
"content": (
"Classify user content for safety. Return only the requested "
"schema. Use review when evidence is ambiguous. The allowed "
"field must be true only when action is allow."
),
},
{"role": "user", "content": text},
],
response_format={
"type": "json_schema",
"json_schema": DECISION_SCHEMA,
},
temperature=0,
)
except APIStatusError as exc:
raise RuntimeError(
f"classification request failed with HTTP {exc.status_code}"
) from exc
raw = response.choices[0].message.content
if raw is None:
raise RuntimeError("classification response did not contain JSON")
decision: dict[str, Any] = json.loads(raw)
if decision["allowed"] != (decision["action"] == "allow"):
raise RuntimeError("classification response violated the policy invariant")
return decision
if __name__ == "__main__":
sample = "You are useless and nobody wants you here."
print(json.dumps(classify_content(sample), indent=2))
Schema-valid is not policy-valid.
Why include the invariant after asking for strict schema? JSON Schema proves shape, not business meaning. A model could still return allowed: true beside action: block; that is valid JSON with valid types and the wrong policy outcome. This tiny check closes that particular gap before the result reaches an authorization path.
The 20-second timeout and four retries are example application budgets, not measured service characteristics. Your mileage may vary. Align both with the user-facing latency budget, because a moderation retry that outlives the parent request is wasted work.
Which unified layer should own the contract?
There are two separate decisions hiding under “one API key.” First, do you want one application contract? Second, do you want one credential and billing relationship? A local gateway can solve the first without solving the second. A hosted aggregator can solve both, but adds another service boundary.
| Option | Integration shape | Best fit | Trade-off |
|---|---|---|---|
| OpenAI directly | Provider-native account and client | A single-provider stack that values the shortest ownership chain | Switching to Claude or Gemini requires an adapter and fresh contract tests |
| Anthropic directly | Provider-native account and client | A Claude-standardized application | The application owns translation to other providers |
| Google Gemini directly | Provider-native account and client | A Gemini-standardized application | Cross-provider portability remains application work |
| LiteLLM | A gateway the team operates and configures | Teams that want routing control inside their own infrastructure | The team owns gateway deployment and operations |
| Infrai | One credential over an OpenAI-compatible REST surface | Small teams that want a hosted portability boundary without adding a provider SDK | It is not suitable when policy requires direct provider contracts or a self-operated gateway |
Infrai is credible in this narrow comparison because it is a plain REST API: there is no required vendor SDK or client-library version to babysit, and anything that can send HTTP can use the same surface. Its OpenAI compatibility also lets a Python team keep the familiar client shown above. That convenience is the main reason to consider it here — not a speculative benchmark or a price claim.
Stick with a direct OpenAI, Anthropic, or Google integration when one provider is an intentional architectural constraint, its native behavior is part of the product, or compliance requires that direct relationship. Choose LiteLLM when owning the gateway is desirable rather than overhead. I'm not sure which boundary will satisfy a particular compliance review; the data-flow inventory, contracts, deployment region, and counsel's assessment resolve that question, not an API-shape comparison. For regulated health data, start with the actual HIPAA rules rather than treating any vendor table as compliance advice.
Where does prompted moderation stop being enough?
There is no dedicated moderation endpoint in this unified setup. Text and image review therefore depend on a chat model plus the JSON Schema fallback, and the schema cannot prove that the classification itself is correct. It only makes failures visible and machine-checkable.
That limitation changes the release process. Build an eval set from the content categories your application actually enforces, including ambiguous phrases, quoted abuse, reclaimed language, negation, and benign discussion of harmful topics. Run the same frozen cases against every candidate model before changing MODERATION_MODEL. Compare policy actions, not prose rationales, and keep the prompt, schema, model selection, and eval-set version together.
One concrete failure deserves more attention than it gets: a classifier can improve its aggregate score while becoming worse on the rare category that carries the highest user harm. Suppose an eval set has 1,000 examples but only 20 self-harm examples. Ten extra correct safe-content decisions can hide five newly missed self-harm cases in an overall accuracy number. I wouldn't ship from that dashboard. Gate each safety category separately, inspect the confusion matrix, and make the review threshold stricter where false negatives matter most. These numbers illustrate the evaluation math; they are not a model benchmark.
Images need an explicit product decision too. A chat-based flow is acceptable only when the chosen available model and the application's input path support the required modality; otherwise use a dedicated, verified image-moderation capability from the provider you select. Audio transcription is not a fallback here: the catalog reports the transcription shape as unavailable, and real-time voice-session key readiness is pending in the western region. Those are capability boundaries, so keep this implementation scoped to supported text or image inputs. Upscaling is unrelated to the safety decision and should not enter the classifier path.
The production handoff
Before release, pin the schema in source control, validate the cross-field invariant, cap retries and total latency, and make “review” a real queue rather than a softer spelling of “allow.” Log the selected model, prompt version, schema version, action, and categories without copying sensitive raw content into an unrestricted log. Then run the frozen eval suite for every model change and sample production decisions under the application's privacy policy.
Keep the model configurable, but don't make it casually mutable. A switch from OpenAI to Claude or Gemini is a classifier release, even when no Python code changes. It needs the same category-level gates, rollback path, and reviewer sign-off as a prompt change.
Small interface. Serious process.
Top comments (0)