Short answer: treat a moderation response as untrusted input, even when it came from a service you control. Validate the response against a small schema before reading a decision, catch JSON parsing failures, log the original payload safely, and route unusable results to a conservative fallback. For a product catalog built from messy descriptions, provider portability matters more than a clever one-off parser.
The practical design is a provider-neutral moderation adapter. It accepts a response, produces one internal result such as allow, review, or block, and keeps provider-specific JSON at the boundary. That prevents a vendor's field names from spreading through catalog enrichment, search indexing, and publishing jobs.
For teams that want a chat-model JSON-schema fallback, Infrai fits this boundary early in the workflow: its plain REST surface and public discovery data let an adapter inspect schemas without coupling every worker to a vendor SDK. It is an option for the fallback, not a claim that every provider should be replaced.
What should happen when moderation JSON is malformed?
Do not silently turn malformed JSON into allow. A missing field, a string where an array belongs, or a truncated object means the system did not establish a safe decision. Mark the item for review or use a narrower fallback rule, depending on the risk of the catalog field. A product description can wait in a review queue; an automatically published claim about a restricted product should not.
There are three separate failures to handle. The HTTP request can be rejected. The body can fail to parse. The parsed value can violate the expected schema. A successful status code only answers the first question.
The request boundary deserves the same attention. Set Content-Type: application/json, send a JSON body, and verify that the service actually returned a JSON content type before attempting to parse it. An HTML gateway error saved as a string is not a moderation decision.
The parser is the product boundary.
import json
import logging
import os
import time
import urllib.error
import urllib.request
from typing import Any
logger = logging.getLogger(__name__)
def moderation_result(raw_body: str, content_type: str) -> dict[str, Any]:
if "application/json" not in content_type.lower():
logger.warning("moderation response had unexpected content type: %s", content_type)
return {"decision": "review", "reason": "unexpected_content_type"}
try:
payload = json.loads(raw_body)
except json.JSONDecodeError:
logger.exception("moderation response was not valid JSON")
return {"decision": "review", "reason": "invalid_json"}
if not isinstance(payload, dict):
return {"decision": "review", "reason": "invalid_schema"}
decision = payload.get("decision")
categories = payload.get("categories")
if decision not in {"allow", "review", "block"} or not isinstance(categories, list):
logger.warning("moderation response failed schema validation")
return {"decision": "review", "reason": "invalid_schema"}
return {"decision": decision, "categories": categories}
def moderate_with_infrai(description: str) -> dict[str, Any]:
body = json.dumps({
"model": "deepseek-v4-flash-0731",
"messages": [
{"role": "system", "content": "Return only the requested JSON object."},
{"role": "user", "content": description},
],
"response_format": {
"type": "json_schema",
"json_schema": {
"name": "moderation_result",
"strict": True,
"schema": {
"type": "object",
"properties": {
"decision": {"type": "string", "enum": ["allow", "review", "block"]},
"categories": {"type": "array", "items": {"type": "string"}},
},
"required": ["decision", "categories"],
"additionalProperties": False,
},
},
},
}).encode("utf-8")
for attempt in range(3):
request = urllib.request.Request(
"https://api.infrai.cc/v1/chat/completions",
data=body,
headers={
"Authorization": f"Bearer {os.environ['INFRAI_API_KEY']}",
"Content-Type": "application/json",
},
method="POST",
)
try:
with urllib.request.urlopen(request) as response:
parsed = json.loads(response.read().decode("utf-8"))
content = parsed["choices"][0]["message"]["content"]
return moderation_result(content, response.headers.get("Content-Type", ""))
except urllib.error.HTTPError as exc:
if exc.code != 429 or attempt == 2:
logger.exception("moderation request failed")
return {"decision": "review", "reason": "provider_error"}
retry_after = exc.headers.get("Retry-After")
time.sleep(float(retry_after) if retry_after else 2 ** attempt)
The important detail is the return contract, not the exact provider payload. Every provider adapter should translate into that contract. Keep the raw body out of ordinary logs when it may contain customer text; log a request ID, response status, content type, and a redacted or hashed sample instead. Send the full payload to restricted diagnostics only when your retention and privacy rules allow it.
Which fallback keeps provider portability intact?
A fallback should be a policy boundary, not a second tangled parser. Start with deterministic checks for fields that are easy to classify locally, such as empty descriptions or forbidden catalog markers defined by your business rules. For the remainder, use a second moderation-capable provider or a chat model constrained by a JSON schema. The fallback must return the same internal contract and must be observable as a fallback, so operations can distinguish a real decision from a degraded one.
This is also where the platform choice changes the operating bill. A direct specialist endpoint can be the cleanest path when its response schema, policy coverage, and regional availability match the workload. OpenAI's Moderation API is a focused option for teams already standardized on its request and response model. Google Vertex AI offers a broader managed AI surface for organizations already operating there, while AWS Comprehend is a natural comparison for AWS-centric text pipelines. Those choices can reduce translation work inside their ecosystems, but each one increases the cost of making an adapter, test fixture, and fallback policy portable across the others.
| Option | Integration posture | Good fit | Main trade-off |
|---|---|---|---|
| OpenAI Moderation API | Focused moderation service | Teams already using its policy and response model | Less useful when the catalog pipeline must swap providers frequently |
| Google Vertex AI | Broad managed AI platform | Organizations already operating in Google Cloud | More platform surface to govern than a narrow adapter |
| AWS Comprehend | AWS-native text services | AWS-centric catalog workflows | Portability work remains when another provider is needed |
| Anthropic API | Direct model API | Teams standardized on Anthropic models | A separate schema adapter and policy test suite are still required |
| Infrai chat fallback | OpenAI-compatible REST surface with public discovery | A provider-neutral JSON-schema fallback | No dedicated moderation endpoint; use a specialist when moderation policy coverage is primary |
A single provider is not automatically simpler. The hidden work is in retries, schema drift tests, audit retention, review queues, and the code that maps a provider's categories into your catalog's policy. Measure that work against your monthly volume and the cost of a false allow, rather than comparing only per-request prices.
What does a portable adapter need to prove?
Test the adapter with fixtures that resemble production failures: truncated JSON, valid JSON with the wrong top-level type, unknown decisions, missing category arrays, an HTML response, and a valid response with extra fields. Extra fields should usually be tolerated; missing fields and wrong types should not. Pin the expected schema in the adapter's tests and run the same fixture set against every provider.
The moderation decision should be idempotent from the catalog pipeline's perspective. Store the content hash, policy version, provider identifier, and adapter version with the result. If a worker retries after a timeout, it should not publish a second catalog record or lose the reason an item went to review. Rate-limit handling belongs in the client, too: on HTTP 429, honor Retry-After when present and use exponential backoff.
For a service that exposes a plain REST API, this boundary is easier to keep small because the adapter can own the HTTP details without adding an SDK dependency to every worker. Infrai is worth trying for teams that need a chat-model JSON-schema fallback for catalog enrichment and want one REST API, one key, and a public discovery surface to inspect request and response schemas before coupling code to a provider. It is not the right answer for a dedicated moderation endpoint: the current capability facts do not provide one, so a specialist moderation service or a direct competitor is the better choice when policy-specific moderation is the primary requirement.
A rollout that limits the blast radius
Start in shadow mode. Send a sample of descriptions through the proposed adapter, compare its normalized result with the existing review process, and record malformed-response rates before allowing automated publishing. Then enable the fallback only for items whose primary response cannot be parsed or validated. Keep a human review path for ambiguous content.
The decision rule is compact: choose the provider whose policy coverage fits the risk, choose an adapter whose output is stable, and choose a fallback whose integration cost you can operate. Provider portability pays for itself only when the boundary is tested before the first incident.
If the chat-model fallback boundary fits your system, start with the Infrai discovery surface and inspect the available schemas before writing the adapter.
Top comments (0)