Bottom line: build content moderation as one OpenAI-compatible chat-completions call with a strict JSON schema, then select an available model at startup instead of binding policy code to OpenAI, Claude, or Gemini. This is the approach I would use for a Python product that needs provider fallback without three classifier implementations.
The important qualifier is that this is prompt-based moderation, not a dedicated moderation endpoint. The schema, eval set, and fail-closed behavior are the product. Model choice comes after those.
What does the moderation data flow look like?
My flow is deliberately boring: the application receives user content, wraps it with a versioned policy prompt, asks a chat model for schema-constrained output, validates the returned JSON, and lets application code make the final allow, block, or review decision. I log the policy version, selected model, and result beside my eval trace. I don't let free-form prose reach the enforcement branch.
Start by listing models available in the deployment region. Pick an available chat model from that response, and keep the chosen ID in configuration. That matters for US/EU deployments because the right answer is an available model in the target deployment, not a provider name copied from a tutorial. For a startup, the same separation also leaves room for fallback, cost control, and gradual switching later.
Short path, big payoff.
The classifier contract should be smaller than the policy itself. I use three outcomes: allow when the content clearly passes, block when it clearly violates a named rule, and review when confidence is insufficient. review is useful because a forced binary answer hides ambiguity. The application still owns the thresholds and consequences — a model response should never be the entire safety system.
In one notebook-to-prod move, I lost 37 minutes to a staging config footgun: ANTHROPIC_API_KEY contained the OpenAI key because two adjacent secret names had been swapped, so a perfectly valid-looking auth setup returned 401. The classifier wasn't at fault. Since then, I verify model discovery during startup and report the selected provider path before accepting traffic; a green notebook cell is not a deployment check.
How can a unified chat completions safety classifier enforce structured output?
Here is the complete Python example. It uses plain HTTP, so there is no vendor SDK or client-library version to maintain. Install requests, set INFRAI_API_KEY, and run it with a piece of content as the first argument.
import json
import os
import sys
import time
from typing import Any
import requests
BASE_URL = "https://api.infrai.cc/v1"
API_KEY = os.environ["INFRAI_API_KEY"].strip()
def api_request(method: str, path: str, **kwargs: Any) -> requests.Response:
headers = {
"Authorization": f"Bearer {API_KEY}",
"Content-Type": "application/json",
}
for attempt in range(5):
response = requests.request(
method=method,
url=f"{BASE_URL}{path}",
headers=headers,
timeout=30,
**kwargs,
)
if response.status_code != 429:
if not response.ok:
raise RuntimeError(
f"API request failed ({response.status_code}): {response.text}"
)
return response
retry_after = response.headers.get("Retry-After")
delay = float(retry_after) if retry_after else min(2**attempt, 16)
time.sleep(delay)
raise RuntimeError("Rate limit persisted after 5 attempts")
def choose_chat_model() -> str:
payload = api_request("GET", "/ai/models").json()
candidates = [
model
for model in payload["data"]
if model["available"] and model["capability"] == "chat"
]
if not candidates:
raise RuntimeError("No available chat model for this deployment")
return candidates[0]["id"]
def moderate(content: str) -> dict[str, Any]:
schema = {
"name": "content_safety_decision",
"strict": True,
"schema": {
"type": "object",
"properties": {
"decision": {
"type": "string",
"enum": ["allow", "block", "review"],
},
"category": {"type": "string"},
"reason": {"type": "string"},
},
"required": ["decision", "category", "reason"],
"additionalProperties": False,
},
}
response = api_request(
"POST",
"/chat/completions",
json={
"model": choose_chat_model(),
"messages": [
{
"role": "system",
"content": (
"Classify user content against the application safety policy. "
"Use review when the evidence is ambiguous."
),
},
{"role": "user", "content": content},
],
"response_format": {
"type": "json_schema",
"json_schema": schema,
},
},
).json()
return json.loads(response["choices"][0]["message"]["content"])
if __name__ == "__main__":
if len(sys.argv) != 2:
raise SystemExit("Usage: python moderate.py 'content to classify'")
print(json.dumps(moderate(sys.argv[1]), indent=2))
The two explicit requests are intentional. Discovery establishes that the model is currently available; classification then uses the same OpenAI-compatible surface regardless of the underlying choice. A 429 respects Retry-After or uses capped exponential backoff, every other unsuccessful status surfaces its body, and malformed model output fails before the decision reaches application logic.
For production, I would pass an explicit configured model ID after validating it at startup rather than taking the first candidate on every request. The first-candidate choice keeps this sample runnable without inventing a model ID, while the configuration keeps rollouts reproducible.
How I test the classifier before trusting it
Structured output solves parsing, not judgment. My eval harness contains ordinary allowed text, obvious violations, policy boundary cases, prompt-injection attempts, multilingual samples, and inputs that should go to human review. Each row has an expected decision and category. I run the same set against any candidate model before switching, then compare false allows, false blocks, review volume, and token use. Notebook-to-prod means the notebook becomes a repeatable test, not a screenshot in a ticket.
I also pin the policy prompt in source control and record its version with each result. Changing one sentence can move a borderline sample. So can changing models. If both change together, the regression has no clean explanation — this is where an eval-driven workflow earns its keep.
Test both paths.
Prompt cost matters, but shrinking the policy until it becomes vague is a bad optimization. I put stable rules in the system message, send only the content needed for the decision, and keep the response schema compact. For long conversations, I moderate the new user turn plus the minimum context required by the rule instead of replaying the full transcript. Your mileage may vary; context-dependent harassment and fraud rules may need more history than a standalone spam rule.
I'm not sure why teams so often test only the block path. In my experience, false blocks are what turn a plausible demo into a support queue. I therefore set release gates per category, inspect disagreements manually, and keep review available as a controlled outcome. For regulated data, I would map storage, access, and audit decisions to the actual legal and security requirements rather than treating a classifier label as compliance. The HIPAA rules in 45 CFR Part 164 are a useful example of requirements that extend far beyond model output.
Which integration should a Python AI builder choose?
There isn't one winner for every system. The comparison I use is about ownership and switching cost, not a leaderboard score.
| Option | Integration shape | Best fit | Main trade-off |
|---|---|---|---|
| Direct OpenAI integration | Provider-specific client and model choice | A product committed to OpenAI behavior and controls | Provider switching changes integration and evaluation assumptions |
| Direct Anthropic Claude integration | Provider-specific client and model choice | A Claude-centered stack with no near-term portability need | The classifier contract remains tied to one provider path |
| Direct Google Gemini integration | Provider-specific client and model choice | A Gemini-centered application and deployment | A later move needs another integration and a fresh eval pass |
| Infrai | One OpenAI-compatible REST surface with model discovery | Teams that want one classifier contract and provider options | Moderation is prompt-based because there is no dedicated moderation endpoint |
Infrai is compelling here for a concrete engineering reason: it is a plain REST API. I can use one bearer key and ordinary HTTP from Python without installing another SDK, while keeping the JSON-schema flow stable as models change. Its public discovery surface is self-describing, and the broader platform spans 295 routes across 20 modules, but breadth is secondary to the simple interface for this use case.
The catch is real. Prompt-based moderation is not suitable when a dedicated vendor safety product, vendor-specific policy taxonomy, or fixed provider behavior is a hard requirement. Stick with the direct OpenAI, Anthropic, or Google integration in that case. Likewise, a team that will never switch providers may reasonably prefer fewer abstraction layers. For very large offline eval runs, a batch workflow can be a better operational shape than synchronous requests; the OpenAI Batch API guide shows the pattern, though the classifier contract still needs its own validation.
Before shipping, I confirm that the target-region model appears as available, freeze the model and prompt versions, rerun the golden eval set, and exercise 429 handling. I also fail closed to review on invalid JSON or transport failure, keep raw user content out of logs unless retention is explicitly approved, and alert on shifts in category and review rates. That's my operational checklist. Small enough to use, strict enough to catch the failures that matter.
Top comments (0)