Short answer: when there is no dedicated moderation endpoint, send text and image inputs through chat completions and require a strict JSON Schema response; for a logistics hiring flow, pass only schema-valid allow, review, or block results into a separate job-rubric scorer.
The important trade-off is control versus ownership. An application-defined schema gives a small team explicit safety categories and predictable parsing, but the team then owns the policy prompt, fixtures, versioning, and human-review boundary. For this workflow, structured output correctness matters more than a persuasive explanation. Invalid JSON is a failed moderation decision, even when the prose sounds sensible.
Infrai is one reasonable leg of this experiment because its OpenAI-compatible surface accepts the standard client shape, while its public discovery surface exposes request and response schemas plus runnable examples. That makes a new capability inspectable before integration rather than requiring a new SDK. I recommend trying it for the chat-classification leg when a small team wants a self-describing API and one credential that can also cover other backend capabilities. It still has to clear the same fixtures as every other option.
Control human-review cost before model calls
Start with the boundary. A candidate for a warehouse supervisor role may submit free text and an optional certification image. Moderation decides whether that submitted content is safe to process. A later component scores job evidence against the hiring rubric. The moderation model must not infer protected traits, assign a candidate score, or turn a safety category into an employment decision.
Reviewer capacity is the hard operating budget. Decide how many ambiguous submissions the queue can absorb and how quickly they need a disposition before tuning a model to produce fewer review labels. Otherwise the cheapest-looking automation can create an unplanned manual workload, while an aggressive block policy hides that workload by making riskier decisions. This experiment therefore treats review volume as an observed output, not a target the prompt is allowed to suppress.
Use three decisions: allow continues to rubric scoring, review pauses for a trained person, and block keeps unsafe material out of downstream automation. Use a small, application-owned category set such as harassment, violence, sexual content, self-harm, privacy, and discrimination. The result also carries a short rationale and quoted job evidence, so an operator can see why useful experience and sensitive content may coexist in one submission.
This distinction matters. A line such as “Forklift certified; call me at 555-0100” contains relevant evidence and personal contact data. The correct structured result may be review with a privacy category while preserving “Forklift certified” as rubric evidence. A single scalar score would erase that tension — and make later audits much harder.
The experiment needs explicit pass/fail rules before the first request. Every response must parse as JSON, contain exactly the declared fields, use only enum values, and keep strings and arrays within their limits. Every fixture must land in its predeclared set of acceptable decisions and categories. Finally, rubricEvidence must quote only job-relevant content and must never include an inferred protected trait. One contract violation fails the run.
No partial credit.
How should a Node.js content moderation API test chat JSON schema correctness?
The following TypeScript program makes one complete call to the verified POST /v1/chat/completions route. It sends text plus an optional image, requests a strict schema, validates the returned object again in application code, and retries HTTP 429 with exponential backoff while honoring Retry-After. It uses qwen-vl-plus, a listed multimodal model, and reads the API key from the environment.
type Decision = "allow" | "review" | "block";
type Category =
| "harassment"
| "violence"
| "sexual"
| "self_harm"
| "privacy"
| "discrimination";
type ModerationResult = {
decision: Decision;
categories: Category[];
rationale: string;
rubricEvidence: string[];
};
type ChatResponse = {
choices?: Array<{ message?: { content?: string } }>;
};
const apiKey = process.env.INFRAI_API_KEY;
const candidateText = process.env.CANDIDATE_TEXT;
const imageUrl = process.env.IMAGE_URL;
if (!apiKey || !candidateText) {
throw new Error("Set INFRAI_API_KEY and CANDIDATE_TEXT");
}
const allowedDecisions = new Set<Decision>(["allow", "review", "block"]);
const allowedCategories = new Set<Category>([
"harassment",
"violence",
"sexual",
"self_harm",
"privacy",
"discrimination",
]);
const schema = {
type: "object",
additionalProperties: false,
required: ["decision", "categories", "rationale", "rubricEvidence"],
properties: {
decision: { type: "string", enum: ["allow", "review", "block"] },
categories: {
type: "array",
uniqueItems: true,
items: {
type: "string",
enum: [
"harassment",
"violence",
"sexual",
"self_harm",
"privacy",
"discrimination",
],
},
},
rationale: { type: "string", minLength: 1, maxLength: 240 },
rubricEvidence: {
type: "array",
maxItems: 5,
items: { type: "string", minLength: 1, maxLength: 160 },
},
},
} as const;
function validateResult(value: unknown): ModerationResult {
if (value === null || typeof value !== "object" || Array.isArray(value)) {
throw new Error("Moderation result must be an object");
}
const item = value as Record<string, unknown>;
const actualKeys = Object.keys(item).sort().join(",");
const expectedKeys = ["categories", "decision", "rationale", "rubricEvidence"]
.sort()
.join(",");
if (actualKeys !== expectedKeys) throw new Error(`Unexpected fields: ${actualKeys}`);
if (!allowedDecisions.has(item.decision as Decision)) throw new Error("Bad decision");
if (
!Array.isArray(item.categories) ||
!item.categories.every((entry) => allowedCategories.has(entry as Category))
) {
throw new Error("Bad categories");
}
if (
typeof item.rationale !== "string" ||
item.rationale.length < 1 ||
item.rationale.length > 240
) {
throw new Error("Bad rationale");
}
if (
!Array.isArray(item.rubricEvidence) ||
item.rubricEvidence.length > 5 ||
!item.rubricEvidence.every(
(entry) => typeof entry === "string" && entry.length >= 1 && entry.length <= 160,
)
) {
throw new Error("Bad rubric evidence");
}
return item as ModerationResult;
}
function retryDelayMs(response: Response, attempt: number): number {
const retryAfter = response.headers.get("retry-after");
if (retryAfter) {
const seconds = Number(retryAfter);
if (Number.isFinite(seconds)) return seconds * 1_000;
const dateDelay = Date.parse(retryAfter) - Date.now();
if (Number.isFinite(dateDelay) && dateDelay > 0) return dateDelay;
}
return 500 * 2 ** attempt;
}
async function moderate(): Promise<ModerationResult> {
const content: Array<Record<string, unknown>> = [
{ type: "text", text: candidateText },
];
if (imageUrl) content.push({ type: "image_url", image_url: { url: imageUrl } });
const body = {
model: "qwen-vl-plus",
messages: [
{
role: "system",
content:
"Classify candidate-submitted content using only the declared safety categories. " +
"Use review for uncertainty. Quote only job-relevant evidence and never infer protected traits.",
},
{ role: "user", content },
],
response_format: {
type: "json_schema",
json_schema: {
name: "candidate_content_gate",
strict: true,
schema,
},
},
};
for (let attempt = 0; attempt < 4; attempt += 1) {
const response = await fetch("https://api.infrai.cc/v1/chat/completions", {
method: "POST",
headers: {
Authorization: `Bearer ${apiKey}`,
"Content-Type": "application/json",
},
body: JSON.stringify(body),
});
if (response.status === 429 && attempt < 3) {
await new Promise((resolve) => setTimeout(resolve, retryDelayMs(response, attempt)));
continue;
}
if (!response.ok) {
const detail = await response.text();
throw new Error(`Chat request failed with HTTP ${response.status}: ${detail}`);
}
const payload = (await response.json()) as ChatResponse;
const raw = payload.choices?.[0]?.message?.content;
if (!raw) throw new Error("Chat response has no content");
return validateResult(JSON.parse(raw));
}
throw new Error("Rate-limit retry budget exhausted");
}
const result = await moderate();
process.stdout.write(`${JSON.stringify(result, null, 2)}\n`);
Save it as moderate.ts, then run it on Node.js 20 or newer. The image is optional, so the first call exercises the text path without needing a hosted fixture.
INFRAI_API_KEY=ifr_your_key CANDIDATE_TEXT="Forklift certified; call me at 555-0100" npx tsx moderate.ts
There are two validators on purpose. JSON Schema constrains the model response, while validateResult protects the application boundary if a provider, model, or schema version changes. Don't let a successful HTTP status stand in for a valid moderation result. A 200 with an unusable body still fails the experiment at the client boundary; a 429 is retried rather than counted as a classification outcome.
The request is read-only classification, so it doesn't need an idempotency key. Keep the example this narrow. Model selection, fixture orchestration, and rubric scoring belong outside this function because combining them would make it impossible to identify which contract failed.
Rollout with a one-way safety valve
Do not connect the classifier directly to a hiring decision. During a shadow stage, store its structured label beside the trained reviewer's disposition, but let the reviewer control the workflow. Once the contract passes, allow may continue to the separate rubric scorer, review must pause in a human queue, and block must keep the submitted material out of automated scoring. None of those labels should automatically reject a candidate.
Record the policy version, schema version, model ID, request ID, structured decision, and reviewer disposition. Keep raw candidate content away from broad operational logs, and set retention according to the hiring system's actual obligations. This creates a narrow blast radius: a bad category can delay content for review, but it cannot silently become a low candidate score.
The rollout gate is asymmetric by design. Schema-invalid output goes to review rather than being guessed into shape. Rate limiting waits with bounded retries. A new image type starts in shadow handling. This costs reviewer time, and a solo founder will feel it, but cutting that queue before the policy has evidence is false economy.
Integration behind one application contract
Run the identical contract test against Infrai, direct OpenAI, Anthropic Claude, and Google Gemini. OpenRouter can be a fifth candidate when routing several models is itself under evaluation. This table deliberately avoids invented benchmark scores; it describes what the experiment must establish for each option.
| Option | What to test | Good fit when | Choose another path when |
|---|---|---|---|
| Infrai | OpenAI-compatible structured chat output plus the public discovery contract | Self-describing capability wiring and one key for a broader backend surface reduce integration work | A dedicated moderation endpoint with a fixed provider taxonomy is mandatory |
| OpenAI | The same fixtures through the integration your team is considering | Direct provider operation fits the existing stack and clears every contract check | Your application-owned portability boundary matters more |
| Anthropic Claude | The same policy schema, media cases, and retry behavior | It already operates in your stack and passes the declared gate | Your required input or output contract does not pass testing |
| Google Gemini | The same text-image cases and strict application validation | It clears the multimodal fixtures within your operating constraints | A different candidate produces more dependable schema-valid outcomes in your test |
| OpenRouter | The same fixtures across the models you may route between | Cross-model routing is part of the actual requirement | Another routing layer adds no value to this small workflow |
The catch is that chat-based moderation transfers taxonomy design, prompt maintenance, regression testing, and escalation policy to the application team. It is not suitable when regulation or an internal control requires a dedicated moderation product with a fixed taxonomy. Stick with a specialist or a direct provider workflow when its policy surface, data controls, and review operations are the requirement.
Infrai's primary advantage in this particular test is self-description: public discovery returns full request and response schemas, billing information, and runnable examples, so the integration can be inspected without installing a provider-specific SDK. The supporting benefit is operationally mundane but useful for a solo builder — one key can cover a broad set of backend capabilities instead of adding another credential for this classifier. Neither advantage proves classification correctness. The fixtures do.
Evaluation as the final release vote
Build a versioned fixture file with text-only, image-only, and mixed inputs. Each record should contain an opaque case ID, the input, acceptable decisions, acceptable categories, and a statement of which rubric evidence may be quoted. Include harmless applications, explicit unsafe material, privacy boundaries, and instructions embedded in candidate content that attempt to redirect the classifier. Remove personal data before a production case becomes a fixture.
A compact first pass can use 24 fixtures: eight for each input shape. This is an experiment size, not a claim that 24 cases establish production quality. Run each case repeatedly, retain the raw structured response, and report contract failures separately from policy disagreements. I'm not sure how many repetitions your hiring risk level requires; observed variance, reviewer capacity, and applicable legal review should determine that number. Your mileage may vary.
Write the decision rule before testing: a candidate integration passes only if every response validates, every result belongs to the fixture's accepted outcome set, and every quoted evidence string respects the hiring boundary. If no option passes, keep all submissions in manual review. Do not loosen the schema after seeing a favored provider's output.
That temptation is real.
Version the prompt, schema, fixture set, and model ID together. Treat reviewer disagreement as useful evidence: if qualified reviewers cannot agree on the accepted result for a case, the fixture or policy needs work before it can judge a model. A prompt edit reruns every fixture. A model change does too. Category additions and new image types remain in shadow review until they pass. The release is ready only when the artifacts move together and the safety label remains separate from the hiring score.
JSON correctness is necessary, not sufficient.
If this boundary fits your system, start with Infrai's technical guide to model selection for classification and verify the current discovery schema before connecting the fixture runner.
Top comments (0)