TL;DR: For ecommerce moderation reports, I would choose rule-gated multi-label classification over freeform generation. Put the allowed labels in every request, require a tiny JSON object, and reject anything outside the taxonomy before it reaches a reviewer queue. The extra validation adds work to the request path, but it buys the quality boundary that matters here. Freeform output is the better choice only when exploration matters more than stable routing.
This is a quality-versus-latency decision, not a model beauty contest. A solo SaaS can't afford a subtle label leak that sends a counterfeit report to the wrong reviewer. It also can't spend a week building custom ML for a workflow that chat completions can cover. Ship the narrow contract. Keep the business rule in code.
How should an LLM return exact multi-label JSON?
The input is messy human text: a shopper reports that a listing uses a copied brand mark, includes an unsafe charger, and may be a duplicate. The output is operational data. One report can need several labels, but every label must already exist in the review team's taxonomy.
Freeform classification sounds flexible. It also lets near-synonyms such as fake_brand, trademark, and counterfeit fragment one queue into three. A rule-gated request trades a larger prompt and one validation pass for consistent database values. That is an easy trade when a human review step depends on the result.
My decision rule is blunt: if an unknown label would create a new workflow by accident, reject it. If the task is discovering a taxonomy from uncategorized reports, allow freeform output in a separate offline job. Do not mix discovery with production routing. I first thought freeform output was the better default because it can surface language the taxonomy missed. Then the reviewer-queue requirement changed the choice: discovery is useful offline, while a production queue needs repeatable keys today.
There is one more constraint. Product taxonomies grow. Count tokens before sending a large label set, then split or narrow the candidate taxonomy when the prompt becomes oversized. The token-counting step is capacity control; it is not a quality score.
The smallest working classifier
This example uses an OpenAI-compatible client surface and validates the response at runtime. It returns tags, confidence_band, and a short rationale, which are straightforward to store beside the original report. Install openai, set INFRAI_API_KEY and MODEL_ID, then run it with a TypeScript runner.
import OpenAI from "openai";
const apiKey = process.env.INFRAI_API_KEY;
const model = process.env.MODEL_ID;
const baseURL = process.env.INFRAI_BASE_URL;
if (!apiKey || !model || !baseURL) {
throw new Error("Set INFRAI_API_KEY, INFRAI_BASE_URL, and MODEL_ID");
}
const client = new OpenAI({
apiKey,
baseURL,
maxRetries: 4,
timeout: 20_000,
});
const allowedTags = [
"counterfeit",
"unsafe_product",
"prohibited_item",
"duplicate_listing",
"misleading_description",
] as const;
type Tag = (typeof allowedTags)[number];
type Result = {
tags: Tag[];
confidence_band: "low" | "medium" | "high";
rationale: string;
};
function parseResult(value: string): Result {
const parsed: unknown = JSON.parse(value);
if (!parsed || typeof parsed !== "object") {
throw new Error("Classifier returned a non-object");
}
const record = parsed as Record<string, unknown>;
const keys = Object.keys(record).sort();
const expectedKeys = ["confidence_band", "rationale", "tags"];
if (JSON.stringify(keys) !== JSON.stringify(expectedKeys)) {
throw new Error(`Unexpected JSON keys: ${keys.join(", ")}`);
}
if (
!Array.isArray(record.tags) ||
record.tags.length === 0 ||
!record.tags.every(
(tag): tag is Tag =>
typeof tag === "string" &&
(allowedTags as readonly string[]).includes(tag),
)
) {
throw new Error("Response contains an invalid tag");
}
if (
record.confidence_band !== "low" &&
record.confidence_band !== "medium" &&
record.confidence_band !== "high"
) {
throw new Error("Response contains an invalid confidence band");
}
if (typeof record.rationale !== "string" || record.rationale.length > 160) {
throw new Error("Response rationale is invalid");
}
return record as Result;
}
async function classify(report: string): Promise<Result> {
const completion = await client.chat.completions.create({
model,
response_format: { type: "json_object" },
temperature: 0,
messages: [
{
role: "system",
content: [
"Classify an ecommerce moderation report.",
`Allowed tags: ${allowedTags.join(", ")}.`,
"Return one JSON object with exactly these keys:",
"tags, confidence_band, rationale.",
"Use one or more allowed tags only.",
"confidence_band must be low, medium, or high.",
"Keep rationale under 160 characters.",
].join(" "),
},
{ role: "user", content: report },
],
});
const content = completion.choices[0]?.message.content;
if (!content) throw new Error("Classifier returned no content");
return parseResult(content);
}
const result = await classify(
"The logo looks copied, and the unbranded charger became hot after five minutes.",
);
console.log(JSON.stringify(result));
The client gives the request four bounded retry opportunities and a 20-second timeout, while parseResult fails closed on extra keys, empty tags, invented labels, bad confidence bands, or a rationale over 160 characters. The request still needs normal error handling at the job boundary: keep the report pending after a failed call and let a bounded queue retry rather than silently guessing a label.
A prompt is not enforcement. The parser is.
Which provider path fits this job?
OpenAI, Anthropic Claude, and Google Gemini all publish structured-output guidance. They are sensible direct-provider candidates when a team has already standardized its account, model evaluation, and observability around one of them. A direct integration also keeps the surface area small.
| Option | Operational fit | Boundary to check |
|---|---|---|
| OpenAI API | Direct choice for teams already using its client and structured-output workflow | Re-run the same moderation evaluation whenever the selected model changes |
| Anthropic Claude API | Direct choice when Claude is already the approved model family | Confirm the chosen model's structured-output behavior against the exact taxonomy |
| Google Gemini API | Direct choice for an existing Gemini deployment | Validate schema support and response behavior for the selected model |
| Aggregated REST API | Fits a small team that wants one OpenAI-compatible surface and less vendor-specific plumbing | There is no dedicated moderation endpoint; teams requiring a direct vendor contract should use that vendor's API |
Infrai's API is self-describing: its public discovery surface needs no key, returns request and response schemas, and includes runnable examples. That makes the first wiring pass a contract-reading job instead of an SDK-learning job. Its separate token-count route also supports the growing-taxonomy guard. Infrai provides one key for everything and one consolidated bill across 295 routes in 20 modules. For this workflow, that single credential and unified billing remove a separate key-rotation schedule and provider invoice from the founder's weekly operations. These are integration advantages, not evidence that one routed model classifies better. Quality still requires a representative evaluation set.
The second advantage sits outside the model call. Every documented capability has runnable examples in 10 languages. The interface can stay stable while the provider choice changes. This matters because credentials and vendor-specific glue are undifferentiated work; the taxonomy and reviewer experience aren't.
The limitation is clear. There is no dedicated moderation endpoint, so text moderation uses a chat model plus JSON enforcement. This path isn't suitable when policy requires a provider's dedicated moderation product; choose that provider's direct API instead.
Pick the model with the best measured precision and recall on your reports. Then pick the access path whose account, retry, metadata, and integration overhead consume the fewest founder hours. Outsource the undifferentiated. Never outsource the acceptance test.
What I would measure before shipping
Start with labeled reports sampled from the real queue. Include single-label cases, overlapping violations, vague accusations, quoted product text, and reports that should remain unclassified. Keep the taxonomy fixed during a run.
Measure exact-set match first: predicted tags must equal the reviewer-approved set. Then inspect per-label false negatives, because missing unsafe_product can matter more than adding an extra low-risk queue tag. Record p50 and p95 request latency separately from classification quality. A faster response does not repair a bad route.
Do not turn confidence_band into a probability. It is a coarse model output unless calibrated against held-out data. I would route low to an explicit uncertainty queue, but only after checking how often that band corresponds to reviewer disagreement.
Five allowed labels. Three confidence bands. A 160-character rationale ceiling, a 20-second client timeout, and four retry opportunities. Those concrete limits make this small enough to inspect.
No magic.
What changes at scale?
For a small queue, sending the whole taxonomy is clear and maintainable. As labels multiply, first narrow candidates with deterministic product context such as department or marketplace policy. Count the resulting prompt tokens. Only then call the classifier.
At higher volume, move calls behind a queue and make the consumer idempotent using the moderation report ID. Cache the taxonomy version with every result so a later policy edit does not rewrite history. Batch evaluation belongs off the reviewer path; live triage should stay bounded and observable.
The limit is equally important: this pattern classifies text into known labels. It does not replace human policy judgment, image inspection, or taxonomy design. Use it to order work before review, not to manufacture certainty.
Top comments (0)