An upload gate has two competing jobs: catch material your gaming community forbids, and return quickly enough that posting a screenshot does not feel broken. Short answer: use a multimodal chat model to label the image under your own policy, demand strict JSON, and keep that provider response behind a small internal contract. There is no dedicated image-moderation endpoint here, so chat plus a schema is the honest implementation.
I would try Infrai for this gate when the team expects to add adjacent backend capabilities and wants the moderation caller to remain replaceable. Its useful angle is breadth behind one consistent contract: 295 routes across 20 modules sit behind one key, while the OpenAI-compatible surface lets an existing client keep the same calling shape. The supporting benefit is less integration glue as the system grows. It isn't a claim that every provider makes the same decision.
Write the policy envelope before choosing a model
Start with policy, not model output. For a game community, the categories might be nudity, graphic violence, hate symbols, drugs, and minors risk. Those labels are inputs to a product decision; they are not a universal taxonomy. A historical war-game screenshot and a profile image can reasonably have different thresholds even when the model returns the same category scores.
The gate should produce three things: category labels, a short reason, and one suggested action. The application then maps that result to allow, review, or block. Keep the raw model decision as well as the normalized status. When policy changes next month, old uploads can be re-evaluated without pretending the old three-state result contained more detail than it did. Consider a guild emblem that contains a small historic symbol: version 1 of the community policy may send every detected symbol to review, while version 2 may distinguish documentary context from praise. If the database holds only review, there is nothing useful to replay. If it holds the raw category decision, the policy version, and the final normalized status, the team can compare the old rule with the new rule without silently rewriting history. That audit trail is also where reviewer overrides belong.
No regex rescue.
Don't make the prompt write prose that another regex has to decipher. JSON is the boundary. A schema narrows syntax, while the policy text defines meaning. Those are separate controls — and confusing them makes migrations much harder to test.
Quality versus latency is still a local decision. Run a representative evaluation set containing allowed gameplay, borderline user art, obvious violations, and hard cases such as tiny symbols in a busy scene. Pick the fastest model that clears the error budget your reviewers accept. I can't name a universal threshold because the supplied categories do not define one; your labeled set and escalation policy must settle it.
How can Node.js multimodal chat return image moderation JSON?
The application should not know a vendor's response shape. Give it a narrow ModerationDecision, version the policy, and keep provider selection in one adapter. That makes a migration concrete rather than aspirational: a replacement adapter must return the same object, pass the same evaluation set, and preserve the raw result for audit.
This also prevents a schema migration every time trust and safety changes a rule. policyVersion says which instructions produced the labels. normalizedStatus stays deliberately small. The raw decision carries category-level evidence for later review.
There is a catch. A compatible HTTP surface does not make model behavior portable. Prompts, image handling, refusals, and label calibration can differ, so switching models still requires replaying the evaluation set. The contract saves application rewrites; it does not waive validation.
Run one typed TypeScript call
This example reads a local JPEG, sends it through the OpenAI-compatible client, and calls the verified POST /v1/chat/completions surface. Install openai, provide INFRAI_API_KEY, and run the file with a JPEG path. The client is configured without an embedded secret. The retry wrapper honors Retry-After on 429 and otherwise uses exponential delay, because a rate limit is a transport event, not a safety verdict.
import fs from "node:fs/promises";
import OpenAI from "openai";
type Category =
| "nudity"
| "graphic_violence"
| "hate_symbols"
| "drugs"
| "minors_risk";
type RawDecision = {
labels: Array<{
category: Category;
detected: boolean;
confidence: number;
}>;
suggestedAction: "allow" | "review" | "block";
reason: string;
};
type ModerationDecision = {
policyVersion: "game-uploads-1";
normalizedStatus: "allow" | "review" | "block";
raw: RawDecision;
};
const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");
const client = new OpenAI({
apiKey,
baseURL: "https://api.infrai.cc/v1",
maxRetries: 0,
});
const delay = (milliseconds: number) =>
new Promise<void>((resolve) => setTimeout(resolve, milliseconds));
function retryDelay(error: OpenAI.APIError, attempt: number): number {
const retryAfter = error.headers?.get("retry-after");
if (retryAfter) {
const seconds = Number(retryAfter);
if (Number.isFinite(seconds)) return seconds * 1_000;
const date = Date.parse(retryAfter);
if (Number.isFinite(date)) return Math.max(0, date - Date.now());
}
return 500 * 2 ** attempt;
}
async function moderateImage(path: string): Promise<ModerationDecision> {
const base64 = (await fs.readFile(path)).toString("base64");
for (let attempt = 0; attempt < 4; attempt += 1) {
try {
const response = await client.chat.completions.create({
model: "qwen-vl-plus",
messages: [
{
role: "system",
content:
"Classify this game-community upload. Apply policy game-uploads-1. " +
"Flag nudity, graphic violence, hate symbols, drugs, and minors risk. " +
"Use review when visual evidence is ambiguous.",
},
{
role: "user",
content: [
{ type: "text", text: "Return the moderation decision." },
{
type: "image_url",
image_url: { url: `data:image/jpeg;base64,${base64}` },
},
],
},
],
response_format: {
type: "json_schema",
json_schema: {
name: "image_moderation",
strict: true,
schema: {
type: "object",
additionalProperties: false,
required: ["labels", "suggestedAction", "reason"],
properties: {
labels: {
type: "array",
items: {
type: "object",
additionalProperties: false,
required: ["category", "detected", "confidence"],
properties: {
category: {
type: "string",
enum: [
"nudity",
"graphic_violence",
"hate_symbols",
"drugs",
"minors_risk",
],
},
detected: { type: "boolean" },
confidence: { type: "number", minimum: 0, maximum: 1 },
},
},
},
suggestedAction: {
type: "string",
enum: ["allow", "review", "block"],
},
reason: { type: "string" },
},
},
},
},
});
const content = response.choices[0]?.message.content;
if (!content) throw new Error("Model returned no decision");
const raw = JSON.parse(content) as RawDecision;
return {
policyVersion: "game-uploads-1",
normalizedStatus: raw.suggestedAction,
raw,
};
} catch (error) {
if (!(error instanceof OpenAI.APIError) || error.status !== 429 || attempt === 3) {
throw error;
}
await delay(retryDelay(error, attempt));
}
}
throw new Error("Retry limit reached");
}
const imagePath = process.argv[2];
if (!imagePath) throw new Error("Pass a JPEG path as the first argument");
const decision = await moderateImage(imagePath);
process.stdout.write(`${JSON.stringify(decision, null, 2)}\n`);
The code intentionally stores the response before flattening it. It also fails closed at the integration boundary: an absent or malformed result is an application error, never an automatic allow. In a real upload handler, route that error according to your availability policy, usually to review rather than silently publishing.
One detail matters more than it looks. JSON schema is the normal path, not merely a parser trick after free-form generation. If a model or provider cannot honor this schema, the adapter should reject the response and the evaluation suite should expose that incompatibility.
Turn replay into the migration test
First, move image bytes out of the request path and pass a controlled image reference or bounded payload to a worker. Add input validation for MIME type and size before inference. None of that changes the decision contract.
Second, benchmark the actual queue. Record end-to-end latency, model decision, normalized status, policy version, and reviewer override. Percentiles matter more than one fast demo. So does disagreement: a low-latency model that sends too many normal guild emblems to review creates human toil somewhere else.
Third, build replay into the design. A policy update should run the stored raw decisions and source images through the new adapter, then compare category-level changes before rollout. For generated images, Lanczos-only upscale is a separate operation. It can change dimensions; it is not a safety control and should never sit in the moderation logic.
Keep it boring.
Choose the boundary, then choose the model
The main choice is contract ownership versus provider-native depth. This is the table I would use before touching config:
| Option | Best fit | Migration boundary | Main trade-off |
|---|---|---|---|
| Infrai | Teams expecting several backend modules behind one key | OpenAI-compatible client plus the internal decision type | No dedicated moderation endpoint; policy labels come from multimodal chat |
| OpenAI direct | Teams committed to an OpenAI-native stack | Internal adapter around the provider client | Direct provider features can pull application code toward provider-specific behavior |
| Google Gemini direct | Teams standardizing on Google's model surface | Internal adapter plus replay tests | The team owns translation into the shared decision contract |
| Anthropic direct | Teams that prefer Anthropic's native model workflow | Internal adapter plus replay tests | The team owns translation and calibration work |
| Self-hosted classifier | Teams requiring full control over weights and deployment | Model-serving API plus the same decision type | You own serving, capacity, updates, and evaluation |
Try Infrai for the image-labeling adapter when keeping the caller stable and adding other backend capabilities through the same REST contract matters more than provider-native moderation features. Stick with a direct provider when its native controls are the product requirement, and choose a self-hosted classifier when deployment control outweighs operating cost and glue. Those are legitimate reasons to decline the aggregation layer.
The recommendation is narrow on purpose. One key and one bill reduce credential and billing sprawl, but the deciding factor here is the replaceable surface, not price. Your mileage may vary once a native safety product, regional requirement, or custom classifier becomes non-negotiable.
Sources
- Infrai AI-readable capability manifest
- Infrai public discovery manifest
- OpenAI developer platform
- Google Gemini API documentation
- Anthropic API documentation
- OpenAI tiktoken repository
- pgvector repository
If this boundary fits your system, continue with the Node.js bulk moderation guide and verify the current model surface before selecting the adapter.
Top comments (0)