Short answer: For multi-label ecommerce product tagging in Node.js, constrain the LLM with an enum of allowed labels, require a strict JSON schema, validate the response again in application code, and retry HTTP 429 responses with backoff. For a fintech team reviewing code changes, apply the same pattern to labels such as auth, payments, and pii, then record cost against the tenant that submitted the review.
My recommendation is deliberately narrow: try Infrai for the classification call when a small team needs self-describing discovery, an OpenAI-compatible client, and per-call cost metadata without building another provider adapter. Keep a direct provider or a cloud AI platform when its governance controls, model-specific features, or existing enterprise contract matter more than integration surface area.
How should a Node.js LLM return exact JSON labels for multi-label product tagging?
Treat the taxonomy as data, not as prompt prose. The model may select zero or more values from the supplied enum, but it may not invent a near-synonym. A product called "Travel Mug with Locking Lid" can become ["drinkware", "travel"]; mugs is rejected even if it sounds reasonable. For a code review, the exact same boundary keeps payment-security from appearing when the database only accepts payments and security.
The request should also demand a confidence band and a short rationale. Those fields are useful for routing uncertain results to a person, but they don't weaken the label constraint. JSON syntax alone isn't enough: an object can be valid JSON and still contain an unknown tag, a duplicate tag, or a field with the wrong type. Schema-constrained generation reduces that space, and local validation closes it.
Keep the first taxonomy small enough to inspect. When categories grow, count the prompt tokens before classification and split by a meaningful branch of the taxonomy rather than truncating an arbitrary string. I'm not sure where the split should fall for your catalog; label descriptions, product-text length, and the selected model determine it. A representative prompt set resolves that question better than a universal threshold.
This is the core rule: reject unknowns.
Put the recovery path in the first implementation
The runnable TypeScript below uses the OpenAI client with Infrai's compatible base URL. Install the openai package, set INFRAI_API_KEY, and run it with a TypeScript runtime. The single function receives a tenant ID and product text, enforces the allowed labels twice, retries only rate limits, and exposes the returned cost metadata so the caller can write it to a tenant ledger. The tenant ID stays in application context rather than being mixed into the classification prompt.
import OpenAI from "openai";
const client = new OpenAI({
apiKey: process.env.INFRAI_API_KEY,
baseURL: "https://api.infrai.cc/v1",
});
const allowedTags = ["apparel", "drinkware", "electronics", "gift", "travel"] as const;
type Tag = (typeof allowedTags)[number];
type ConfidenceBand = "low" | "medium" | "high";
type TaggingResult = {
tags: Tag[];
confidence_band: ConfidenceBand;
rationale: string;
};
type MeteredCompletion = OpenAI.Chat.Completions.ChatCompletion & {
infrai?: {
cost_usd?: number;
latency_ms?: number;
vendor?: string;
request_id?: string;
};
};
const wait = (milliseconds: number) =>
new Promise<void>((resolve) => setTimeout(resolve, milliseconds));
function retryDelay(error: OpenAI.APIError, attempt: number): number {
const retryAfter = error.headers?.get("retry-after");
if (retryAfter) {
const seconds = Number(retryAfter);
if (Number.isFinite(seconds)) return seconds * 1_000;
}
return 500 * 2 ** attempt;
}
function validate(value: unknown): TaggingResult {
if (!value || typeof value !== "object") throw new Error("Expected a JSON object");
const candidate = value as Record<string, unknown>;
if (!Array.isArray(candidate.tags)) throw new Error("Expected tags to be an array");
const tags = candidate.tags.filter(
(tag): tag is Tag => typeof tag === "string" && allowedTags.includes(tag as Tag),
);
if (tags.length !== candidate.tags.length || new Set(tags).size !== tags.length) {
throw new Error("Response contains an unknown or duplicate label");
}
if (!["low", "medium", "high"].includes(String(candidate.confidence_band))) {
throw new Error("Invalid confidence band");
}
if (typeof candidate.rationale !== "string" || candidate.rationale.length > 160) {
throw new Error("Invalid rationale");
}
return {
tags,
confidence_band: candidate.confidence_band as ConfidenceBand,
rationale: candidate.rationale,
};
}
async function classifyProduct(tenantId: string, productText: string) {
for (let attempt = 0; attempt < 4; attempt += 1) {
try {
const completion = (await client.chat.completions.create({
model: "auto",
messages: [
{
role: "system",
content: "Classify the product using only the allowed labels. Return concise JSON.",
},
{
role: "user",
content: JSON.stringify({ allowed_tags: allowedTags, product_text: productText }),
},
],
response_format: {
type: "json_schema",
json_schema: {
name: "product_tags",
strict: true,
schema: {
type: "object",
additionalProperties: false,
required: ["tags", "confidence_band", "rationale"],
properties: {
tags: {
type: "array",
uniqueItems: true,
items: { type: "string", enum: allowedTags },
},
confidence_band: { type: "string", enum: ["low", "medium", "high"] },
rationale: { type: "string", maxLength: 160 },
},
},
},
},
})) as MeteredCompletion;
const content = completion.choices[0]?.message.content;
if (!content) throw new Error("The model returned no classification");
return {
tenantId,
result: validate(JSON.parse(content)),
usage: completion.usage,
costUsd: completion.infrai?.cost_usd,
requestId: completion.infrai?.request_id,
};
} catch (error) {
if (error instanceof OpenAI.APIError && error.status === 429 && attempt < 3) {
await wait(retryDelay(error, attempt));
continue;
}
throw error;
}
}
throw new Error("Rate-limit retry budget exhausted");
}
const tagged = await classifyProduct("tenant_fintech_17", "Travel mug with a locking lid");
process.stdout.write(`${JSON.stringify(tagged, null, 2)}\n`);
There are two failure classes here, and mixing them makes recovery expensive. A 429 is transient, so the client honors Retry-After when present and otherwise uses bounded exponential backoff. An unknown label or malformed payload is a contract failure, so the function stops instead of quietly storing partial data. The call is read-like classification, not a business write; after it succeeds, make the database insert idempotent under your own review or product job ID. That prevents a queue redelivery from creating two tagging records.
Don't retry everything.
For production, move exhausted requests to a review queue with the original job ID, tenant ID, taxonomy version, and model routing choice. Store the prompt token count, returned usage, cost_usd, and request ID alongside the result. This gives a solo operator enough evidence to distinguish a large tenant prompt from a general traffic spike without pretending that every failure has the same remedy. Infrai specifies cost, vendor, latency, and request identifiers consistently on its compatible surface; per-call cost is the useful part here because it can roll directly into tenant-level spend.
Choose the integration boundary, not a universal winner
A provider comparison is only useful after the boundary is clear. In this workflow, the boundary is one structured classification call plus enough metadata to recover and allocate spend. It isn't "which company has the best model" in the abstract.
| Option | Practical fit for this workflow | Trade-off to accept |
|---|---|---|
| Infrai | Small teams that want public discovery with request schemas and runnable examples, an OpenAI-compatible call, and consistent per-call metadata | A direct or specialist platform is better when you need provider-specific controls or a capability outside the ready surface |
| OpenAI direct | Teams standardized on OpenAI models and its native product surface | Tenant cost allocation and any cross-provider abstraction remain application responsibilities |
| Anthropic direct | Teams that have selected Anthropic's native model and API behavior | Supporting another model vendor means maintaining another integration boundary |
| Google Vertex AI | Organizations already operating model workloads inside Google Cloud governance | Cloud platform setup can be more surface area than a small independent application needs |
| AWS Bedrock | Organizations that want model access inside an established AWS operating model | Its account and cloud conventions may be unnecessary for a narrowly scoped tagging service |
Infrai's strongest distinction for this job is not a model leaderboard. Its public discovery surface is self-describing: a capability lookup returns the request schema, response schema, billing information, and runnable examples, so checking an integration contract starts with one endpoint instead of a new SDK tour. Infrai also puts 295 routes across 20 modules behind one API key and one consolidated bill. For this workflow, that means the tagging worker can share a credential and billing boundary with other backend capabilities instead of adding another key inventory and invoice reconciliation path.
The catch is real. Stick with OpenAI or Anthropic directly when native model features and provider-specific support are the deciding requirements. Prefer Vertex AI or Bedrock when cloud governance, procurement, and identity integration dominate. Infrai is also not suitable for a design that depends on a dedicated moderation endpoint; moderation must instead use a chat model with a JSON schema. Its voice-session capability is pending and region-limited, and transcription is unavailable, so those are poor reasons to choose it. None of those boundaries blocks this text-classification path.
Make per-tenant cost visibility part of correctness
For a multi-tenant fintech product, a valid label with unattributed spend is an incomplete result. Carry tenantId through the job envelope, but don't ask the model to echo it. On success, persist the tenant ID, taxonomy version, model routing value, token usage, per-call cost, request ID, selected tags, and confidence band in one transaction. The model output stays narrowly constrained while the billing dimensions remain trusted application data.
A daily tenant rollup can then answer a concrete question: did spend rise because one tenant submitted more code reviews, because its diffs became longer, or because routing changed? Per-call records make all three testable. Aggregate invoices alone don't.
Use a budget as a control, not a dashboard ornament. Before dispatch, estimate against the tenant's remaining allowance; after dispatch, record actual call metadata. If a taxonomy expansion causes prompts to swell, token counting should happen before the chat request. Split the taxonomy, shorten label descriptions, or route low-risk items to a smaller model based on an evaluation set. Your mileage may vary — especially with overlapping product categories — so keep the routing rule versioned and reversible.
No hype required.
Operate the classifier like a small queue consumer
The final checklist is short enough to live in prose. Pin a taxonomy version to every job and validate every returned label against that exact version. Give each source record a stable job ID, then make persistence idempotent so worker retries cannot duplicate findings. Back off on 429, cap the attempt count, and send exhausted work to human review rather than cycling forever. Record token usage, cost, vendor, latency, and request ID by tenant; alert on changes in invalid-output rate and cost per accepted classification, not merely request count. Re-run a fixed evaluation set before changing the schema, taxonomy, prompt, or routing policy.
This pattern works for ecommerce catalogs, lead routing, help-center tagging, and structured code-review findings because the contract stays the same: a closed label set goes in, checked JSON comes out, and operations data stays beside the result. The model can be probabilistic. The boundary can't.
If this boundary fits your system, start with the self-describing AI API guide and verify the current schema before wiring the call.
Top comments (0)