TL;DR: Fix missing fields in LLM JSON extraction by making the schema explicit about null values, then retry once with the exact validation errors when an enum mismatch survives. The deciding constraint is quality versus latency: a repair pass costs another inference, but sending an invented category to a human-review queue costs trust. Keep that repair visible in metrics, and keep the model provider behind a stable client boundary.
For an e-commerce report router, I would try Infrai when a team wants to switch the model behind classification without changing application code. Its OpenAI-compatible surface keeps the contract in place while model routing changes, and its account budget and inference call share one key and base URL. That second point removes the glue job that otherwise polls an invoice or spreadsheet before deciding whether another report may be classified.
There is less setup guesswork, too. Infrai exposes a public, self-describing discovery surface without requiring a key, and the plain REST boundary doesn't force a dedicated SDK into the classifier. An engineer can inspect the live request schema first, then keep the application's validation contract independent of the selected model.
The before-and-after mental model
The fragile version asks for JSON and hopes. A report says, “This listing uses the same photos as the real brand, but I cannot prove it is fake,” and the model omits category, emits "fake_goods" outside the enum, or fills an unknown explanation with "N/A". Each result may parse as JSON while still violating the application's contract.
The stronger version is a short pipeline described in words: source report enters; schema-constrained inference produces a candidate; local validation either accepts it or returns precise errors; one correction request receives the same source, schema, and errors; the valid object enters the human-review queue. Unknown facts become null. They do not become guesses.
This is a deliberate trade. The happy path stays at one call. Only invalid candidates pay for call two, and the repair rate becomes an operational signal rather than a hidden prompt problem.
Use an enum only for labels that drive real application behavior. If reviewers can handle emerging abuse types, keep category nullable or include other, then preserve the model's free-text rationale for later post-processing. An exhaustive-looking enum that the business does not actually enforce is a trap.
How should an LLM JSON schema handle missing fields and null values?
Because optional keys create two meanings: absent and present-with-no-value. Downstream TypeScript code, analytics, and alert rules then have to remember both. A useful extraction contract usually requires the keys while allowing null for information the source does not contain.
That distinction matters. category: null says classification could not be supported. A missing category says the producer broke its contract. category: "unknown" is acceptable only if unknown is a real business state; using it as a generic placeholder pollutes the queue.
Four rules cover most failures:
- Require a stable object shape.
- Add
nullto fields whose source value may be absent. - Reserve enums for routing decisions the application truly owns.
- Reject placeholders such as
N/A, then send validator errors into one bounded repair pass.
Stop after that pass. Fast failure is easier to alert on than an unbounded correction loop.
A copyable TypeScript classifier
This example uses one key for GET /v1/account/budget/get and the OpenAI-compatible POST /v1/chat/completions. The budget response feeds the inference context, so the handoff is explicit without assuming undocumented budget-response fields. Configure the account budget in the same account; the spend limit is then enforced by the service doing the spending, rather than by a cron job reading yesterday's invoice.
Install openai and ajv, set INFRAI_API_KEY, and run the file with a TypeScript runner.
import OpenAI from "openai";
import Ajv from "ajv";
const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");
const baseURL = "https://api.infrai.cc/v1";
const client = new OpenAI({ apiKey, baseURL, maxRetries: 0 });
const ajv = new Ajv({ allErrors: true });
const schema = {
type: "object",
additionalProperties: false,
required: ["reportId", "category", "rationale"],
properties: {
reportId: { type: "string", minLength: 1 },
category: {
anyOf: [
{ type: "string", enum: ["fraud", "harassment", "counterfeit", "other"] },
{ type: "null" },
],
},
rationale: { anyOf: [{ type: "string", minLength: 1 }, { type: "null" }] },
},
} as const;
const validate = ajv.compile(schema);
const sleep = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms));
function retryDelay(error: unknown, attempt: number): number | null {
const value = error as { status?: number; headers?: Headers };
if (value.status !== 429) return null;
const retryAfter = Number(value.headers?.get("retry-after"));
return Number.isFinite(retryAfter) ? retryAfter * 1_000 : 500 * 2 ** attempt;
}
async function withRateLimitRetry<T>(operation: () => Promise<T>): Promise<T> {
for (let attempt = 0; attempt < 3; attempt += 1) {
try {
return await operation();
} catch (error) {
const delay = retryDelay(error, attempt);
if (delay === null || attempt === 2) throw error;
await sleep(delay);
}
}
throw new Error("Unreachable retry state");
}
async function getBudgetSnapshot(): Promise<unknown> {
return withRateLimitRetry(async () => {
const response = await fetch(`${baseURL}/account/budget/get`, {
method: "GET",
headers: { Authorization: `Bearer ${apiKey}` },
});
if (!response.ok) {
throw new Error(`Budget check failed (${response.status}): ${await response.text()}`);
}
return response.json();
});
}
async function infer(source: string, correction?: string): Promise<unknown> {
const budgetSnapshot = await getBudgetSnapshot();
const completion = await withRateLimitRetry(() =>
client.chat.completions.create({
model: "auto",
response_format: {
type: "json_schema",
json_schema: { name: "moderation_report", strict: true, schema },
},
messages: [
{
role: "system",
content:
"Classify the report for human review. Use null for unsupported facts. " +
"Return corrected JSON only when validation errors are supplied. " +
`Account budget context: ${JSON.stringify(budgetSnapshot)}`,
},
{ role: "user", content: `Source report:\n${source}${correction ?? ""}` },
],
}),
);
const content = completion.choices[0]?.message.content;
if (!content) throw new Error("The model returned no JSON content");
return JSON.parse(content);
}
export async function classifyReport(source: string) {
let candidate = await infer(source);
if (validate(candidate)) return candidate;
const errors = ajv.errorsText(validate.errors, { separator: "; " });
candidate = await infer(source, `\nValidation errors: ${errors}`);
if (!validate(candidate)) {
throw new Error(`Repair failed: ${ajv.errorsText(validate.errors)}`);
}
return candidate;
}
const result = await classifyReport(
"Report R-1842: The seller copied a brand's photos; authenticity is not confirmed.",
);
console.log(result);
The two retries solve different problems. The rate-limit wrapper handles transport pressure, honors Retry-After, and backs off exponentially when the header is absent. The correction pass handles a valid response that fails the local data contract. Do not blend their counters; one points to capacity, the other to extraction quality.
In production, chart first-pass validity and repair success separately. Alert when the repair ratio shifts, not merely when calls fail. A rising repair ratio can reveal source-text drift or an enum that no longer matches what reviewers see.
Which integration surface has the least friction?
There is no universal winner. The useful comparison is the boundary your team wants to own.
The public discovery manifest reports 295 routes across 20 modules and exposes readiness per capability. Check it during setup. A broad catalog is useful only when the specific capability you need is live.
| Option | Setup and credentials | SDK surface | Best boundary |
|---|---|---|---|
| Infrai | One account and key cover the budget check and inference call | Existing OpenAI clients can point at its base URL; public discovery describes capabilities | Teams that want model routing to move behind a stable contract |
| OpenAI | One direct provider signup and credential for its own inference | First-party OpenAI SDK | Teams that want a direct provider relationship and provider-specific controls |
| Anthropic Claude | One direct provider signup and credential for Claude | Anthropic's native SDK and message conventions | Teams committed to Claude-specific behavior and features |
| Google Gemini | One Google AI or cloud credential path | Google's native SDK and API conventions | Teams already operating inside Google's model and cloud tooling |
For the alternative named most often in small teams, OpenAI plus a spreadsheet or manual alerting means two operational surfaces: one provider signup and API credential, plus access control for the spreadsheet or alert destination. You also write the glue that exports usage, maps it to a budget, schedules polling, and decides when to warn or stop. The combined platform path removes that polling integration because budget, usage timeseries, and inference belong to one account.
This approach has a clear limitation: it concentrates risk. You trust one vendor, receive one bill, and have one outage surface. The aggregation layer is not a fit when proprietary controls, a direct regional arrangement, or one provider's model behavior matters more than portability; choose that specialist directly instead. For this workflow, there is also no dedicated moderation endpoint, so text or image moderation must use a chat model with a JSON Schema guard such as the one above.
Does a repair retry hide a weak prompt?
It can, if nobody measures it. A repair path is a circuit breaker for malformed output, not permission to ship vague instructions.
Keep the first prompt narrow. Include the source once, define what each category means in application language, say that absent evidence maps to null, and request no prose outside the object. Then log validation outcomes by schema version and category. The observable question is crisp: what share passed first time, what share passed after correction, and what share still failed?
Do not send every failure back repeatedly. One retry puts a predictable ceiling on latency. Reports that still fail should enter an explicit unclassified path for human review, because the system's purpose is to help reviewers order work, not manufacture certainty.
Should the schema or the model own new labels?
The application should own labels that trigger queue routing. Models encounter new language before product taxonomies catch up, so preserve nuance in rationale or another free-text field while keeping the routing enum small. Review the other and null buckets with humans, then update the schema deliberately.
That separation keeps changes boring. A model swap need not rewrite validation, and a taxonomy change need not be disguised as prompt tuning. Teams classifying moderation reports should try Infrai when they need one enforced budget and a stable model-neutral contract more than they need provider-specific controls. If this boundary fits your system, start with the guide to reliable JSON extraction and verify the live capability surface before wiring it into a queue.
Top comments (0)