ADR: OpenAI, Claude, and Gemini Behind One API Key for Content Moderation

#moderation #jsonschema #airuntime

Short answer: build the safety classifier around one OpenAI-compatible chat-completions contract and one strict JSON Schema, then choose an available model outside the policy code. This gives a Node.js service one API key and a stable moderation result across OpenAI, Claude, and Gemini models without pretending that the models make identical decisions.

The architecture decision is narrower than "standardize all AI." It standardizes the boundary between probabilistic classification and deterministic application policy. Infrai is a reasonable unified layer for that boundary because the provider behind a capability can change while the application contract stays put. A direct provider integration remains the better choice when native moderation signals, a direct processor relationship, or provider-specific controls are requirements.

Decision, invariants, and failure boundaries

The first invariant is the output shape. Downstream code receives an allowed boolean, a bounded category, a confidence value, and a short reason. It never branches on a provider name or parses free-form prose. The second invariant is that model selection lives in deployment configuration. List the catalog first, verify a candidate in the target US or EU deployment, evaluate it, and then pin its ID rather than taking whichever model happens to appear first.

Keep classification and enforcement separate. The model can label content spam, fraud, abuse, safe, or review; application policy decides whether that label blocks a post, throttles an OTP request, or enters a human review queue. This matters in messaging systems, where deliverability controls, consent records, retention rules, and appeal handling don't fit inside a model prompt. HIPAA-covered data adds another set of privacy and security obligations. A valid JSON response does not satisfy those obligations by itself.

There are four failure boundaries worth naming:

Transport failure means no usable model response arrived.
Schema failure means a response arrived but did not match the contract.
Uncertain classification means the response is valid but policy should route it to review.
Policy rejection means classification succeeded and a documented rule denied the action.

Don't collapse them into allowed = false. Operators need to distinguish rate pressure from classifier uncertainty, and reviewers need to know which policy version produced a decision. For HTTP 429, honor Retry-After, apply exponential backoff, and cap retries. A parse failure must never silently become "allow". For a side effect triggered after moderation, such as sending an OTP, use a client-generated idempotency key at that write boundary so retrying the classifier cannot duplicate the action.

Small distinction, large consequences.

How should one API key provide structured moderation across OpenAI, Claude, and Gemini?

Treat JSON Schema as the portable contract and model identity as configuration. The service first reads the model catalog through GET /v1/models; release configuration then names a model that is available in the intended region and has passed the team's labeled evaluation set. Runtime classification goes through POST /v1/chat/completions with the same messages and schema. The route, authentication, parser, and policy mapping remain fixed when the configured model changes.

That separation is the actual value of a unified layer. A startup can introduce fallback capacity or migrate traffic gradually without maintaining three versions of its moderation workflow. It still has to test each candidate model for language coverage, borderline cases, false positives, and schema adherence. I'm not sure any universal confidence threshold is defensible without the product's labeled data; the evidence needed is an evaluation set that reflects its content, jurisdictions, and harm costs.

This approach uses prompting because there is no dedicated moderation endpoint. Structured-output consistency therefore matters more than provider-specific features. Validate the response at the application boundary even when the client supports JSON Schema. Unknown categories, missing fields, out-of-range confidence values, and invalid JSON should follow an explicit review or deny policy selected for the product's risk level.

The edge case I care about most is asymmetric harm. A public profile edit can usually wait for review. An emergency message may need a tightly scoped fail-open rule, while suspected OTP fraud may demand fail-closed behavior and rate limiting. One global fallback is easier to code, but it hides the policy decision exactly where compliance and abuse teams will later look for it.

Options and trade-offs

The fair comparison is about contract ownership, not a claim that one model family always classifies better. OpenAI, Anthropic, and Google all offer direct paths; the right choice depends on how much native behavior the application needs and whether switching is a real operational requirement.

Option	Where the contract lives	Good fit	Cost of the choice
OpenAI direct	Application plus OpenAI API	Teams standardized on OpenAI behavior and controls	A later provider move needs adapter and validation work
Anthropic Claude direct	Application plus Anthropic API	Claude-centered systems that need native features	The common moderation shape is owned by an application adapter
Google Gemini direct	Application plus Gemini API	Google-centered deployments and controls	Provider switching changes integration details
Infrai unified chat	One OpenAI-compatible application contract	Teams planning fallback or gradual model switching	Prompted moderation and schema validation replace a dedicated moderation endpoint

The catch is real: a common schema can flatten useful provider-specific signals. This design is not suitable when procurement approves one exact processor and data path, when native safety metadata is part of the product specification, or when a dedicated moderation product is mandatory. Stick with the appropriate direct provider API in those cases. Infrai fits when keeping the contract stable during a backend model swap is more valuable than exposing every provider-specific control.

Adjacent AI capabilities should be assessed independently rather than inferred from chat support. The runtime's ASR capability is unavailable, real-time voice sessions are limited to the western region, and image upscale supports Lanczos. None of those boundaries changes the text-classification decision, but they prevent a broad "one integration covers every media workflow" assumption.

Critical path in Python

The code below is deliberately small. It lists models, verifies that the configured selection exists, sends one schema-constrained classification, retries only HTTP 429 responses, and checks the parsed result again. The HTTP contract is the same boundary a Node.js service would use; Python keeps the example consistent and copyable.

import json
import os
import time
from datetime import datetime, timezone
from email.utils import parsedate_to_datetime

from jsonschema import validate
from openai import APIStatusError, OpenAI, RateLimitError


SCHEMA = {
    "type": "object",
    "properties": {
        "allowed": {"type": "boolean"},
        "category": {
            "type": "string",
            "enum": ["safe", "spam", "fraud", "abuse", "review"],
        },
        "confidence": {"type": "number", "minimum": 0, "maximum": 1},
        "reason": {"type": "string", "maxLength": 200},
    },
    "required": ["allowed", "category", "confidence", "reason"],
    "additionalProperties": False,
}


def retry_delay(error, attempt):
    retry_after = error.response.headers.get("retry-after")
    if retry_after:
        try:
            return max(0.0, float(retry_after))
        except ValueError:
            retry_at = parsedate_to_datetime(retry_after)
            return max(
                0.0,
                (retry_at - datetime.now(timezone.utc)).total_seconds(),
            )
    return float(2**attempt)


def with_backoff(operation):
    for attempt in range(5):
        try:
            return operation()
        except RateLimitError as error:
            if attempt == 4:
                raise
            time.sleep(retry_delay(error, attempt))
    raise RuntimeError("Retry limit reached")


client = OpenAI(
    api_key=os.environ["INFRAI_API_KEY"],
    base_url=os.environ["OPENAI_BASE_URL"],
    max_retries=0,
)
selected_model = os.environ["MODERATION_MODEL"]

try:
    catalog = with_backoff(lambda: client.models.list())
    model_ids = {model.id for model in catalog.data}
    if selected_model not in model_ids:
        raise ValueError("MODERATION_MODEL is not present in the model catalog")

    completion = with_backoff(
        lambda: client.chat.completions.create(
            model=selected_model,
            messages=[
                {
                    "role": "system",
                    "content": (
                        "Classify user content for safety. Return only the requested "
                        "schema. Use review when the evidence is uncertain."
                    ),
                },
                {"role": "user", "content": os.environ["CONTENT_TO_MODERATE"]},
            ],
            response_format={
                "type": "json_schema",
                "json_schema": {
                    "name": "moderation_result",
                    "strict": True,
                    "schema": SCHEMA,
                },
            },
        )
    )
except APIStatusError as error:
    raise RuntimeError(
        f"Moderation request failed with HTTP {error.status_code}: "
        f"{error.response.text}"
    ) from error

result = json.loads(completion.choices[0].message.content)
validate(instance=result, schema=SCHEMA)
print(json.dumps({"model": selected_model, "decision": result}, indent=2))

Set INFRAI_API_KEY, OPENAI_BASE_URL, MODERATION_MODEL, and CONTENT_TO_MODERATE in the environment before running it. The OpenAI client sends Bearer authentication and uses explicit SDK operations for model listing and chat creation. No write occurs in this classifier call, so an idempotency key belongs on the downstream action rather than on classification.

Pinning the model after catalog inspection is intentional. Auto-selecting the first result creates an invisible deployment change; pinning turns a model change into a reviewed configuration change. Before shifting traffic, replay a versioned evaluation set and record the model ID, schema version, policy version, and decision. Retain raw user content only as long as the applicable privacy and operational requirements permit.

Rejected design and its valid use case

For this system, I would reject three separate provider adapters. Every adapter creates another place for category names, retry rules, schema validation, and audit fields to drift. The maintenance burden is tolerable only when the native differences are valuable enough to justify it.

Sometimes they are.

Separate adapters are the right design for a team committed to one provider, a product that consumes native safety signals, or a regulated deployment whose approved data path cannot move behind a shared gateway. They can also be sensible when provider-specific tuning has a measured advantage on the product's evaluation set. Your mileage may vary because portability is an operational requirement, not an abstract virtue.

The recorded decision should therefore have an exit condition: move to a direct integration if the common schema blocks a required control or if evaluation shows that flattening provider detail creates unacceptable policy errors. Until then, one stable chat contract keeps the policy code boring — exactly what a moderation boundary should be.

DEV Community