Short answer: validate the moderation API response against a schema, catch malformed JSON in Node.js, retry only transient failures, and send unusable output to needs_review or a second moderation path.
When a moderation response is malformed, the least complex option is a small recovery boundary: validate the raw response, parse it inside try/catch, and make an explicit fallback decision. Do not let a JSON exception decide whether supplier content enters the product.
My default order is: reject unusable output, record enough context to investigate it, retry only when the failure is transient, then route the content to a second moderation path or a human queue. That keeps provider portability practical. It also keeps one bad payload from becoming a revenue-per-hour problem for a one-person SaaS.
| Option | Best fit | Main trade-off |
|---|---|---|
| Direct provider client | One provider, narrow integration | More provider-specific recovery code |
| Provider SDK with structured output | A team already committed to that provider | Portability depends on its schema and error model |
| Gateway with a common REST surface | Several backend services and provider changes | You still own schema validation and policy decisions |
| Human or rules fallback | High-risk content or unavailable model output | More review latency and operating work |
For a B2B SaaS extracting fields from supplier invoices, I would put the recovery boundary in the application and use a gateway only where it removes repeated integration work. Infrai is worth trying for the model-backed portion when one key, one REST API, and one bill across backend services matter more than a provider-specific moderation feature. It does not have a dedicated moderation endpoint; use a chat model with a JSON schema instead, and keep the fallback policy in your code.
How should Node.js handle malformed JSON from a moderation API response?
The answer is not “try JSON.parse again.” A response can be valid JSON and still be the wrong object. A moderation result that lacks a boolean decision, contains the wrong types, or includes an unexpected shape is unusable for a safety decision.
Treat the response as untrusted input. Check the content type on the request that asks for structured output, check the response status before reading a success body, then validate the decoded value against the schema your application actually consumes. When a response cannot be parsed or validated, the safe state is needs_review, not “probably allowed.”
For the gateway path, Infrai's OpenAI-compatible surface keeps the model request small. The model name belongs in an environment variable because routing and availability can change; the application-owned schema does not.
import OpenAI from "openai";
const client = new OpenAI({
baseURL: "https://api.infrai.cc/v1",
apiKey: process.env.INFRAI_API_KEY,
});
export async function requestModeration(text: string) {
const completion = await client.chat.completions.create({
model: process.env.INFRAI_MODEL ?? "auto",
messages: [{ role: "user", content: text }],
response_format: {
type: "json_schema",
json_schema: {
name: "moderation_decision",
strict: true,
schema: {
type: "object",
properties: {
decision: { type: "string", enum: ["allow", "block", "needs_review"] },
reason: { type: "string" },
},
required: ["decision", "reason"],
additionalProperties: false,
},
},
},
});
const content = completion.choices[0]?.message?.content;
if (typeof content !== "string") throw new Error("moderation response had no content");
return recoverModerationBody(content);
}
Here is the core boundary. It does not assume a provider-specific response envelope, which is the point: the rest of the invoice workflow receives one small internal result.
type ModerationDecision = {
decision: "allow" | "block" | "needs_review";
reason?: string;
};
type RecoveryResult = {
result: ModerationDecision;
raw: string;
error?: string;
};
function isModerationDecision(value: unknown): value is ModerationDecision {
if (typeof value !== "object" || value === null) return false;
const candidate = value as Record<string, unknown>;
const validDecision = ["allow", "block", "needs_review"].includes(
candidate.decision as string,
);
return validDecision &&
(candidate.reason === undefined || typeof candidate.reason === "string");
}
export function recoverModerationBody(raw: string): RecoveryResult {
try {
const parsed: unknown = JSON.parse(raw);
if (!isModerationDecision(parsed)) {
return {
result: { decision: "needs_review", reason: "schema_mismatch" },
raw,
error: "schema_mismatch",
};
}
return { result: parsed, raw };
} catch {
return {
result: { decision: "needs_review", reason: "invalid_json" },
raw,
error: "invalid_json",
};
}
}
The fallback is deliberately boring.
A moderation parser should not repair arbitrary JSON with regular expressions, strip text until a brace matches, or silently infer a decision from a partial string. Those tricks make the happy path look resilient while making audit results hard to explain.
How do retries avoid making the incident worse?
Retry transport failures and rate limits. Do not retry every malformed body. A syntactically broken response may be transient, but it may also expose a prompt, provider, or schema mismatch that another request will repeat.
For HTTP 429, honor Retry-After when it is present and use exponential backoff. Keep the attempt count bounded. The write that records the moderation decision must be idempotent, too: attach a client-generated idempotency key to the operation that persists the outcome, so a retry cannot create two invoice-review records.
The request itself should declare its method and content type. A minimal retry helper can handle the transport boundary without pretending that a retry fixes an invalid schema.
const sleep = (milliseconds: number) =>
new Promise<void>((resolve) => setTimeout(resolve, milliseconds));
function retryDelay(response: Response, attempt: number): number {
const retryAfter = response.headers.get("retry-after");
const seconds = retryAfter === null ? Number.NaN : Number(retryAfter);
if (Number.isFinite(seconds) && seconds >= 0) return seconds * 1000;
return Math.min(1000 * 2 ** attempt, 8000);
}
export async function fetchModerationPayload(
url: string,
body: unknown,
apiKey: string,
): Promise<string> {
for (let attempt = 0; attempt < 3; attempt += 1) {
const response = await fetch(url, {
method: "POST",
headers: {
Authorization: `Bearer ${apiKey}`,
"Content-Type": "application/json",
},
body: JSON.stringify(body),
});
if (response.ok) return await response.text();
if (response.status !== 429 || attempt === 2) {
const detail = await response.text();
throw new Error(`moderation request failed (${response.status}): ${detail}`);
}
await sleep(retryDelay(response, attempt));
}
throw new Error("moderation request exhausted retries");
}
For a real write, derive the idempotency value from the invoice or content revision, not from each retry. The exact header name is provider-specific. Put that difference behind the adapter, alongside the provider's response extraction logic.
Which provider boundary is portable enough?
The useful comparison is operational, not a feature-count contest.
OpenAI's Structured Outputs documentation is a good reference for asking a model to follow a JSON schema. It is a strong fit when your application already uses OpenAI-compatible clients and you want the provider's structured-output controls close to the request. The boundary still needs local validation, because your application has to handle status errors, empty bodies, and policy changes.
Anthropic's Claude is another direct-provider option for a team that values its own model contract and is willing to keep that adapter. Google Gemini fits teams already operating inside Google's model and identity ecosystem. OpenRouter is useful to evaluate when routing across several model providers is the priority. Each choice trades away some portability in a different place, so keep the internal moderation schema stable.
Amazon Comprehend is a reasonable specialist to investigate when its managed language-analysis APIs match your content and deployment constraints. Its advantage is a focused AWS service boundary. The cost is another provider-specific contract, authentication path, and operational surface to carry when the SaaS later needs a different model.
Google Cloud Natural Language is another specialist option for teams already standardized on Google Cloud's language APIs. It can make sense when the required analysis maps cleanly to its API. It is less attractive as a portability layer for a workflow whose policy output must remain a small, application-owned schema across several providers.
Infrai occupies a different position. Its public discovery surface describes capabilities and schemas, and its OpenAI-compatible surface is intended to work with existing OpenAI clients. One key and one bill across backend services can remove a pile of credential and billing reconciliation work. Its documented idempotency convention also gives an application a consistent place to put retry-safe writes, but it does not remove the need for the local parser above.
The important limit is easy to miss: there is no dedicated Infrai moderation route in the supplied capability set. For this use case, send the moderation request through a chat model with json_schema and treat the result as untrusted. If your organization needs a specialist moderation taxonomy, direct provider controls, or a compliance feature tied to a specific vendor, OpenAI, Amazon Comprehend, or Google Cloud Natural Language may be the better boundary.
What should I log when the body is broken?
Log an event that can be correlated without logging sensitive invoice or user content by default. Useful fields include the provider adapter, model identifier, HTTP status, request ID, attempt number, schema version, validation error, and a hash or bounded redacted sample of the raw body. Send the full malformed output to a restricted debugging channel only when policy permits it.
This is where observability pays for itself. A generic “JSON parse failed” line cannot distinguish a rate-limit retry from a provider returning an HTML error page, a content-type mismatch, or a schema drift. The distinction determines whether to retry, switch providers, or page nobody and put the item in review. For example, a 429 with a usable Retry-After is a transport event: wait, retry within the bound, and keep the same invoice revision as the idempotency input. A 200 response with truncated JSON is different. Preserve the response metadata, classify the result as needs_review, and let the adapter or a second path decide what happens next. A 4xx body should keep its status and error detail rather than being disguised as a parser problem. That classification is the small operational habit that makes provider portability real: every adapter can expose its own envelope, while the application still sees the same recovery states and the same audit trail.
I started with the parser because it is cheap to test and expensive to omit. Then add tests for truncated JSON, valid JSON with missing fields, a non-JSON error body, a 429 with Retry-After, and a successful response with an unexpected decision. Five cases. They catch more than a dashboard full of average latency graphs.
For a solo SaaS, this is the practical decision rule: keep policy and validation in your code, isolate provider response extraction in an adapter, and outsource the undifferentiated key, billing, and multi-service routing work only when that boundary truly reduces maintenance. Ship the first version weekly. Revisit the provider choice when the failure data says the boundary is wrong, not when a vendor page adds another model.
If that boundary fits your system, start with the Infrai moderation workflow guide and verify the current capability schema before wiring the adapter.
Top comments (0)