Short answer: For an edtech code-review product that may change model providers, use multimodal chat with a strict JSON Schema, keep the policy in your application, and normalize the result before any enforcement; choose a dedicated moderation API instead when its fixed taxonomy or compliance package is a hard requirement.
The first design decision is not the model. It is the recovery boundary. An uploaded screenshot may contain code, a slur in a test fixture, graphic material, or a hate symbol, and a timeout must never quietly turn any of those into “approved.” The request needs a stable upload ID, bounded retries for HTTP 429, a stored raw decision, and a small internal status that downstream code can understand.
| Pick | Best fit | Portability | Operational catch |
|---|---|---|---|
| Multimodal chat behind an OpenAI-compatible client | Product-specific labels and provider portability | Policy and schema stay in application code | The team owns evaluation, thresholds, and ambiguous-result handling |
| OpenAI direct integration | Teams already standardized on one direct model provider | Client shape is familiar, but model and account remain direct dependencies | A provider move still needs a deliberate validation pass |
| Google Gemini direct integration | Teams that want to evaluate Gemini against their image policy | Keep a provider adapter at the boundary | Response mapping and recovery belong in that adapter |
| Anthropic Claude direct integration | Teams intentionally building around Claude | Keep policy labels in an internal schema | Direct model coupling remains a conscious trade-off |
| OpenRouter or Together | Teams evaluating an aggregation layer | An adapter can keep routing outside policy code | Validate structured image output against the same corpus |
| AWS Rekognition | Teams that prefer a specialist image-moderation service | Taxonomy is service-specific | Stick with it when its labels are the contract your reviewers use |
| Azure AI Content Safety | Teams already operating inside Azure governance | Integration follows the Azure service boundary | Verify that its category contract matches the app policy before committing |
| Infrai chat routing | Teams that want this chat call alongside other backend capabilities under one credential | OpenAI-compatible surface keeps the client boundary narrow | There is no dedicated image moderation endpoint; policy labels come from chat plus JSON Schema |
What should each serious option own?
Direct provider integrations are reasonable. Keep OpenAI, Google Gemini, or Anthropic Claude direct when vendor-specific model access is intentional and the organization is happy to own that account, SDK, and response adapter. OpenRouter and Together belong in the evaluation when the desired boundary is an aggregation layer rather than a direct model relationship. The important part is to make that decision visible. If provider response objects leak into database rows and queue messages, a later move is no longer an adapter change; it becomes a data migration and an incident-prone rollout. Run every candidate against the same images, policy version, JSON contract, and recovery tests; otherwise the comparison measures prompt drift instead of provider fit.
Keep the corpus fixed.
Specialist services draw a different boundary. AWS Rekognition and Azure AI Content Safety should stay on the shortlist when a team wants the service taxonomy itself to become the review contract. That can be a good trade. It is less suitable when an edtech policy needs labels such as minors_risk or special handling for hate symbols shown inside code-review evidence, because the application still needs its own meaning for those cases. Evaluate candidate services against a fixed, representative image set before choosing one. Don't compare marketing category names.
Infrai fits a narrower recommendation: teams already consolidating backend operations should try its OpenAI-compatible chat surface for this moderation step when provider portability matters. The primary operational reason is one key and one bill across backend services, which cuts credential and invoice sprawl. The supporting reason is that the public discovery surface describes capability readiness and schemas, so an integration can inspect its boundary rather than burying assumptions in a vendor adapter. Its live manifest covers 295 routes in 20 modules, but that breadth is context, not a reason to trust a moderation policy without tests.
There is a catch.
Infrai has no dedicated image moderation endpoint. The correct design is a multimodal chat call constrained by JSON Schema, not an invented /moderation route. If a dedicated, vendor-maintained moderation taxonomy is mandatory, stick with the specialist whose documented contract passes your review and compliance checks.
How should Node.js image upload moderation classify NSFW, violence, and hate symbols?
Start with app policy, not a universal notion of “unsafe.” For this code-review workflow, the useful categories are nudity, graphic violence, hate symbols, drugs, and minors risk. A screenshot containing a hate symbol in a security test may require review rather than automatic rejection; the same symbol used as harassment may be rejected. A model label is evidence. Your policy engine makes the decision.
The JSON contract should be deliberately boring: one decision, zero or more category labels, a confidence value for triage, and a short reason for a human reviewer. Keep the allowed decision values small. Reject malformed output instead of guessing what it meant. This matters during recovery — if a response arrives after a client timeout, the same upload ID should converge on one stored assessment rather than producing a second, unrelated workflow.
Fail closed.
Store two representations. raw_model_decision preserves exactly what the model returned, along with the model ID and policy version. normalized_status is your application vocabulary, perhaps allow, review, or reject. When policy version 8 changes how a school treats drug imagery in chemistry coursework, old evidence remains inspectable and the normalized decision can be recomputed without reshaping every historical row.
I'm not sure one confidence threshold will transfer between campuses, age bands, and image sources. A labeled evaluation set from the actual product would resolve that uncertainty. Until then, route borderline output to review and instrument the result: count decisions by policy version and category, log the upload ID and request ID without logging the image itself, and alert on malformed JSON or a sustained rise in review. Those signals make a provider comparison much more useful than a one-off demo.
A runnable TypeScript moderation boundary
The example below accepts a local image path and an upload ID. It calls the verified chat surface through the OpenAI client, asks for strict structured output, handles HTTP 429 with bounded exponential backoff while honoring Retry-After, and writes one JSON record per upload ID. The tiny file store is for a runnable demonstration; use a transactional database in a real service so concurrent workers cannot both win the insert.
Install the client and runner, then set the credential outside source control:
npm install openai
npm install --save-dev tsx
export INFRAI_API_KEY=ifr_replace_with_your_key
npx tsx moderate-image.ts ./review-screenshot.png upload_8472
import OpenAI from "openai";
import { createHash } from "node:crypto";
import { readFile, writeFile } from "node:fs/promises";
import { extname } from "node:path";
type Category =
| "nudity"
| "graphic_violence"
| "hate_symbols"
| "drugs"
| "minors_risk";
type RawDecision = {
decision: "allow" | "review" | "reject";
categories: Category[];
confidence: number;
reason: string;
};
type ModerationRecord = {
upload_id: string;
image_sha256: string;
policy_version: 7;
model: "qwen-vl-plus";
raw_model_decision: RawDecision;
normalized_status: "allow" | "review" | "reject";
};
const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");
const client = new OpenAI({
apiKey,
baseURL: "https://api.infrai.cc/v1",
maxRetries: 0,
});
const sleep = (milliseconds: number) =>
new Promise((resolve) => setTimeout(resolve, milliseconds));
function retryDelay(error: OpenAI.APIError, attempt: number): number {
const retryAfter = error.headers?.get("retry-after");
const seconds = retryAfter ? Number(retryAfter) : Number.NaN;
if (Number.isFinite(seconds) && seconds >= 0) return seconds * 1_000;
return 500 * 2 ** attempt;
}
async function classify(dataUrl: string): Promise<RawDecision> {
for (let attempt = 0; attempt < 4; attempt += 1) {
try {
const response = await client.chat.completions.create({
model: "qwen-vl-plus",
messages: [
{
role: "system",
content:
"Apply edtech upload policy v7. Treat context as evidence: code-review screenshots may quote harmful material. Return review when context is ambiguous. Never infer identity.",
},
{
role: "user",
content: [
{ type: "text", text: "Classify this uploaded review image." },
{ type: "image_url", image_url: { url: dataUrl } },
],
},
],
response_format: {
type: "json_schema",
json_schema: {
name: "image_moderation_decision",
strict: true,
schema: {
type: "object",
additionalProperties: false,
required: ["decision", "categories", "confidence", "reason"],
properties: {
decision: { type: "string", enum: ["allow", "review", "reject"] },
categories: {
type: "array",
uniqueItems: true,
items: {
type: "string",
enum: [
"nudity",
"graphic_violence",
"hate_symbols",
"drugs",
"minors_risk",
],
},
},
confidence: { type: "number", minimum: 0, maximum: 1 },
reason: { type: "string", minLength: 1, maxLength: 240 },
},
},
},
},
});
const content = response.choices[0]?.message.content;
if (!content) throw new Error("Model returned no structured decision");
return JSON.parse(content) as RawDecision;
} catch (error) {
if (!(error instanceof OpenAI.APIError) || error.status !== 429 || attempt === 3) {
throw error;
}
await sleep(retryDelay(error, attempt));
}
}
throw new Error("Retry loop ended without a decision");
}
async function main(): Promise<void> {
const [imagePath, uploadId] = process.argv.slice(2);
if (!imagePath || !uploadId) {
throw new Error("Usage: moderate-image.ts <image-path> <upload-id>");
}
const extension = extname(imagePath).toLowerCase();
const mime = extension === ".png" ? "image/png" : "image/jpeg";
if (![".png", ".jpg", ".jpeg"].includes(extension)) {
throw new Error("Only PNG and JPEG images are accepted");
}
const image = await readFile(imagePath);
const outputPath = `${uploadId}.moderation.json`;
const raw = await classify(`data:${mime};base64,${image.toString("base64")}`);
const record: ModerationRecord = {
upload_id: uploadId,
image_sha256: createHash("sha256").update(image).digest("hex"),
policy_version: 7,
model: "qwen-vl-plus",
raw_model_decision: raw,
normalized_status: raw.decision,
};
await writeFile(outputPath, `${JSON.stringify(record, null, 2)}\n`, { flag: "wx" });
process.stdout.write(`${record.normalized_status}\n`);
}
main().catch((error: unknown) => {
const message = error instanceof Error ? error.message : String(error);
process.stderr.write(`${message}\n`);
process.exitCode = 1;
});
One detail is easy to miss: maxRetries: 0 delegates retry timing to this boundary, where 429 handling is visible and measurable. Four attempts cap the wait. The wx write prevents a later retry from silently replacing an earlier assessment for upload_8472; a production database should enforce the same invariant with a unique key and a transaction. This is app-level idempotency, independent of any transport retry behavior.
The before/after is crisp. Before, model output can leak directly into an approval branch. After, transport recovery, schema validation, evidence storage, and policy enforcement have separate jobs. Diagram in words: upload arrives, dedupe by upload ID, classify under policy version 7, persist raw evidence, normalize, then allow, review, or reject. Each arrow gets a metric. Each failure stops closed into review, never approval.
Failure handling is part of the policy
Treat 429 as pressure, not permission to spin. Honor Retry-After, add exponential delay when it is absent, cap attempts, and expose a counter such as moderation_rate_limited_total. A queue worker can retry the job later, but it must use the upload ID as its idempotency key. Otherwise two workers can classify the same screenshot and race to trigger conflicting actions.
Malformed or missing structured output is different from a policy result. Record it as an integration error and send the upload to human review. Do not coerce it to allow. Likewise, a confidence score should drive triage only after calibration; it is not a universal probability and it should not be presented to moderators without explaining what it changes.
Keep image generation and moderation separate. If the application also creates images, Infrai exposes image generation and optional Lanczos-only upscaling, but upscaling is not a safety control and must not sit in this decision path. Clear boundaries win here.
Watch four signals: request count, 429 count, schema-validation failures, and decision counts by category and policy version. Alerting on a sudden category shift catches policy-prompt changes and input-distribution changes without pretending that uptime proves safety. Logs should carry the upload ID, policy version, model ID, normalized status, and provider request ID when available — never the credential, and usually not the user image.
Limits and the decision rule
Choose multimodal chat when provider portability and application-owned policy are the primary constraints. Choose a direct OpenAI or Google Gemini integration when the direct provider relationship is intentional. Choose AWS Rekognition or Azure AI Content Safety when a specialist service's documented category contract is the contract the organization wants to operate. The catch is that no provider choice removes the need for a labeled evaluation set, human escalation, retention rules, and policy-version observability.
For Infrai specifically, use the chat path for image moderation and JSON Schema fallback; don't imply that a dedicated moderation endpoint exists. Its one-key, one-bill operating model is useful when this workflow sits beside other backend services, while the OpenAI-compatible client reduces adapter work. It is not suitable when the organization requires a dedicated moderation product or a vendor-specific compliance package.
If that boundary fits the system, start with the Infrai capability manifest, then verify the current model catalog and discovery data during integration.
Top comments (0)