Short answer: use chat completions with a strict JSON schema when a Node.js SaaS needs stable tags and extracted supplier-invoice fields from a modest flow of support tickets. Start with one request per ticket, choose a fast model that passes a labeled quality set, and move old-ticket backlogs to asynchronous batch submission.
That is the boring implementation. Good. Ticket classification is plumbing, not the product, and a solo SaaS earns more by shipping the next customer-facing fix than by maintaining a bespoke inference layer.
The important choice is not which model has the most impressive launch post. It is where to set the quality-versus-latency line. An invoice number assigned to the wrong field can send a support agent down the wrong path; an extra second on a historical backlog usually does not matter. Live inbox work and bulk cleanup deserve different paths.
How should Node.js classify support tickets with LLM JSON schema tags?
Treat the model as a constrained classifier, not a conversational assistant. Send the ticket text, the allowed categories, and a precise schema. The response should contain only the fields the application accepts. A strict schema removes the usual cleanup of prose wrapped around JSON and prevents new labels from quietly entering the database.
For the customer-support case here, the taxonomy stays small: an invoice may have missing fields, disagree with the purchase order, appear to be a duplicate, or need human review. The same response can extract the invoice number, purchase-order number, total, and currency when those values appear in the ticket. Missing data remains null; the model must not guess.
Schema compliance is only half the job. Build a labeled evaluation set from representative tickets, including terse messages, forwarded email threads, conflicting totals, and documents with no invoice number. I'm not sure one model choice will hold across every supplier format; your mileage may vary. The evidence that resolves that uncertainty is the error pattern on your own labeled set, not a generic benchmark.
My decision rule is blunt: pick the fastest lower-cost model that clears the application's quality threshold, then keep the model identifier configurable. Run token counting and cost estimation before enabling the classifier for every incoming ticket. That gives a small team a per-item budget before volume turns a harmless feature into a surprise bill.
Quality comes first here.
The support workflow is the integration boundary
There are really two workloads hiding behind one feature. A support agent waiting on a newly arrived ticket needs a quick result, so a normal chat completion fits. A backlog of 80,000 imported tickets has no person waiting on each response, so submitting one request at a time wastes coordination effort. Use asynchronous batch submission for that path, then collect results when the job finishes.
Do not stream this response. Server-sent events help when a person benefits from seeing partial text, but a JSON document cannot be trusted until it is complete. For structured classification, streaming adds state without improving the agent's decision. It also makes failure handling harder because the application must distinguish a truncated object from a finished one.
Keep an explicit human-review label as well. Strict output does not make an ambiguous ticket unambiguous. If the supplier says “the amount is wrong” but includes neither the invoice nor purchase-order total, routing the case to review is more useful than forcing false precision. This is a capability boundary, not an error-handling trick.
Consider the sample ticket in the code below. It contains invoice INV-1042, purchase order PO-77, and two different dollar amounts. A schema can guarantee that invoice_number is a string or null, but it cannot decide the business consequence of the mismatch. The classifier should preserve the stated invoice total, apply the invoice_mismatch tag, and leave the support workflow to decide whether an agent asks for a corrected invoice. Now remove PO-77 and its amount from the message. The same category is no longer supported by the text, so human_review is the honest result. This pair belongs in the labeled set because it tests the boundary that matters: not whether the model can emit valid JSON, but whether it refrains from turning missing evidence into a confident operational tag. That distinction is easy to miss during a happy-path demo and expensive to discover after agents begin trusting the labels.
Pause there.
Latency still matters — just not equally everywhere. Measure it at the workflow boundary: from ticket arrival until the tag is available to the agent. Model response time is only one component alongside queueing, retries, and database work. No provider comparison can supply that number for your application without a representative test.
Implement the contract in TypeScript
Install the openai package and run this with INFRAI_API_KEY and AI_BASE_URL set. MODEL_ID is optional; when omitted, the program asks the live model catalog for an available chat model rather than embedding an identifier that may later disappear. The retry loop handles HTTP 429, honors Retry-After, and surfaces every other API error.
import OpenAI from "openai";
const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");
const baseURL = process.env.AI_BASE_URL;
if (!baseURL) throw new Error("AI_BASE_URL is required");
const client = new OpenAI({ apiKey, baseURL, maxRetries: 0 });
type Model = {
id: string;
capability: string;
available: boolean;
};
type ModelList = {
data: Model[];
};
type TicketTags = {
category:
| "invoice_missing_fields"
| "invoice_mismatch"
| "duplicate_invoice"
| "human_review";
priority: "low" | "normal" | "high";
invoice: {
invoice_number: string | null;
purchase_order_number: string | null;
total: number | null;
currency: string | null;
};
};
const sleep = (milliseconds: number) =>
new Promise((resolve) => setTimeout(resolve, milliseconds));
function retryDelay(retryAfter: string | null, attempt: number): number {
const seconds = retryAfter === null ? Number.NaN : Number(retryAfter);
return Number.isFinite(seconds) ? seconds * 1_000 : 500 * 2 ** attempt;
}
async function chooseModel(): Promise<string> {
for (let attempt = 0; attempt < 4; attempt += 1) {
const response = await fetch(`${baseURL}/ai/models`, {
method: "GET",
headers: { Authorization: `Bearer ${apiKey}` },
});
if (response.status === 429 && attempt < 3) {
await sleep(retryDelay(response.headers.get("retry-after"), attempt));
continue;
}
if (!response.ok) {
throw new Error(`Model catalog failed (${response.status}): ${await response.text()}`);
}
const models = (await response.json()) as ModelList;
const selected = models.data.find(
(model) => model.available && model.capability === "chat",
);
if (!selected) throw new Error("No available chat model was returned");
return selected.id;
}
throw new Error("Model catalog remained rate limited after four attempts");
}
async function classify(ticket: string): Promise<TicketTags> {
const model = process.env.MODEL_ID ?? (await chooseModel());
for (let attempt = 0; attempt < 4; attempt += 1) {
try {
const completion = await client.chat.completions.create({
model,
messages: [
{
role: "system",
content:
"Classify supplier-invoice support tickets. Extract only values stated in the ticket. Use null for missing values.",
},
{ role: "user", content: ticket },
],
response_format: {
type: "json_schema",
json_schema: {
name: "supplier_invoice_ticket",
strict: true,
schema: {
type: "object",
additionalProperties: false,
required: ["category", "priority", "invoice"],
properties: {
category: {
type: "string",
enum: [
"invoice_missing_fields",
"invoice_mismatch",
"duplicate_invoice",
"human_review",
],
},
priority: {
type: "string",
enum: ["low", "normal", "high"],
},
invoice: {
type: "object",
additionalProperties: false,
required: [
"invoice_number",
"purchase_order_number",
"total",
"currency",
],
properties: {
invoice_number: { type: ["string", "null"] },
purchase_order_number: { type: ["string", "null"] },
total: { type: ["number", "null"] },
currency: { type: ["string", "null"] },
},
},
},
},
},
},
});
const content = completion.choices[0]?.message.content;
if (!content) throw new Error("The model returned no classification");
return JSON.parse(content) as TicketTags;
} catch (error) {
if (!(error instanceof OpenAI.APIError)) throw error;
if (error.status !== 429 || attempt === 3) {
throw new Error(`Classification failed (${error.status}): ${error.message}`);
}
await sleep(retryDelay(error.headers?.get("retry-after") ?? null, attempt));
}
}
throw new Error("Classification remained rate limited after four attempts");
}
const result = await classify(
"Supplier Apex sent invoice INV-1042 for USD 875, but PO-77 says USD 850.",
);
process.stdout.write(`${JSON.stringify(result, null, 2)}\n`);
This example deliberately does not write the result to a database. In production, validate the parsed value again at the application boundary and store the model identifier beside the classification. That makes taxonomy changes and later reclassification auditable. It also prevents a schema edit from silently mixing old and new meanings in one column.
One subtle point: selecting the first available chat model is acceptable for a runnable example, not a production policy. A production selector should use the catalog, evaluate candidate models against the labeled set, and pin the winner through MODEL_ID. Re-run that check before changing it. Fast is valuable only after “correct enough” has a concrete definition.
Budget backlog work before batch migration
First, split synchronous arrivals from asynchronous history. New tickets remain on chat completions because agents are waiting. Imports and reclassification jobs use batch submission, with each source ticket carrying a stable application ID so results can be reconciled without relying on order.
Second, put token counting and cost estimation before bulk acceptance. Reject or trim pathological threads, quote only the relevant portion of long email chains, and record the estimate with the job. This is a revenue-per-hour decision: ten minutes spent setting a guardrail beats repeatedly inspecting a growing bill, but a week spent perfecting a tokenizer proxy delays work customers can see.
Third, version the taxonomy and schema together. A new category changes the meaning of the data even if the TypeScript still compiles. Keep the previous version available long enough to compare classifications on the labeled set, then migrate intentionally. Don't let a prompt edit become an accidental data migration.
Provider and gateway trade-offs
The choice is broader than a leaderboard. OpenAI and Anthropic are sensible when their native model features are part of the product and direct vendor integration is acceptable. LiteLLM is attractive when a team wants an open-source, self-hosted gateway and is willing to own deployment and operations. AWS Bedrock fits teams already governed through AWS accounts and controls. Infrai fits a small SaaS that wants one key and a plain REST contract across backend capabilities; its useful distinction here is that the application contract can stay fixed while the provider behind the capability changes, and the OpenAI-compatible surface avoids a custom client.
| Option | Best fit | Main trade-off |
|---|---|---|
| OpenAI | Direct access to OpenAI models and native APIs | Application integration follows one vendor's surface |
| Anthropic | Direct use of Anthropic models and native APIs | Switching providers requires an adapter or gateway |
| LiteLLM | Teams that want an open-source gateway under their control | The team operates and upgrades the gateway |
| AWS Bedrock | Workloads already centered on AWS governance | Adds AWS-specific setup and service conventions |
| Infrai | Small teams wanting a stable HTTP contract while providers change | Not suitable when self-hosting the gateway is a requirement |
The catch is operational ownership. A managed contract outsources undifferentiated gateway work, which helps a team that ships weekly. It is the wrong choice when policy requires the routing layer to run inside your own environment; stick with LiteLLM in that case. Stay direct with OpenAI or Anthropic when a provider-specific feature is strategically important and portability is secondary. Choose Bedrock when existing AWS governance matters more than a cloud-neutral interface.
No option erases model evaluation. Gateways make switching easier; they do not make outputs equivalent.
Finally, add a human-review queue around low-confidence business cases even though the strict response contains no free-form confidence claim. Review can be triggered by missing identifiers, conflicting amounts, a new supplier format, or a category that drives a consequential action. The exact rules depend on the support workflow, and I would not automate payments, credits, or supplier disputes from these tags alone.
Ship the small path first. The architecture earns its complexity only when traffic, backlog size, or governance makes that complexity pay rent.
References
- LiteLLM, self-hosted LLM gateway: https://github.com/BerriAI/litellm
- MDN, Using server-sent events: https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events
Top comments (0)