Node.js Ticket Triage with LLM JSON Schema Labels and Chat Completions

#node #llm #jsonschema

Short answer: for a normal Node.js SaaS classifying a modest support queue, use chat completions with a strict JSON schema; move to asynchronous batch submission when the backlog no longer belongs in a request path.

The constraint that matters is not whether a model can write a clever explanation. It is whether the next function receives stable fields. A ticket router needs queue, tags, and urgency, not a paragraph that a regex has to interpret. I would send the ticket and the allowed categories together, force a schema, parse once, and make the resulting update idempotent. Small. Boring. Good.

The build log: constrain the handoff before choosing a model

Start with a taxonomy that a reviewer can apply twice and get the same answer. In the example below, the queue and tag values are closed enums. The reason field is useful for a human review screen, but it is not used to route work. This separation prevents a persuasive sentence from becoming an accidental policy decision.

Model selection comes after that contract. Check the available model catalog first, then run a labeled sample through the candidates. Token counting and cost estimation belong before rollout, so a junior team can predict the per-item budget instead of discovering it in a monthly bill. I would benchmark accuracy and latency on terse subjects, misspellings, and account vocabulary; an aggregate score alone is too blunt. The useful comparison is concrete: take the same ticket fixtures, freeze the allowed enum list, and record how often each model chooses other when a known queue applies. Then inspect the misses, not just the mean. If billing and account access are routinely confused, the prompt or taxonomy needs work before another model is added. If the labels are acceptable but latency is high, a cheaper fast model may be enough for internal tagging. I've found that this worksheet also exposes config bloat early: every provider-specific parser, credential, and retry policy becomes a line item the team has to own.

Keep it typed.

I treat a 429 as part of the design, not an exceptional branch. The classifier should back off, honor Retry-After, and stop after a bounded number of attempts. A retry of this read-like model call does not create a ticket, but the database write that applies its labels still needs an idempotency key or a compare-and-set update. Otherwise a worker restart can duplicate the side effect.

How can Node.js classify support tickets with LLM JSON schema tags?

This is the smallest runnable shape. It uses the OpenAI-compatible client idiom, while keeping the service base URL in configuration so the same code can point at a direct provider, a gateway, or a compatible runtime. The request is an ordinary chat-completions call; the schema is the important part.

import OpenAI from "openai";

const apiKey = process.env.LLM_API_KEY;
const baseURL = process.env.LLM_BASE_URL;
const model = process.env.LLM_MODEL;
if (!apiKey || !baseURL || !model) throw new Error("LLM_API_KEY, LLM_BASE_URL, and LLM_MODEL are required");

const client = new OpenAI({ apiKey, baseURL, maxRetries: 0 });

const schema = {
  type: "object",
  additionalProperties: false,
  properties: {
    queue: { type: "string", enum: ["billing", "bug", "account", "other"] },
    tags: {
      type: "array",
      uniqueItems: true,
      items: { type: "string", enum: ["refund", "login", "data-loss", "how-to"] },
    },
    urgency: { type: "string", enum: ["low", "normal", "high"] },
    reason: { type: "string" },
  },
  required: ["queue", "tags", "urgency", "reason"],
} as const;

const sleep = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms));

async function classify(ticket: string) {
  for (let attempt = 0; attempt < 4; attempt += 1) {
    try {
      const response = await client.chat.completions.create({
        model,
        messages: [
          { role: "system", content: "Return only labels allowed by the JSON schema." },
          { role: "user", content: `Allowed categories: billing, bug, account, other.\nTicket: ${ticket}` },
        ],
        response_format: {
          type: "json_schema",
          json_schema: { name: "ticket_labels", strict: true, schema },
        },
      });
      const content = response.choices[0]?.message.content;
      if (!content) throw new Error("No classification returned");
      return JSON.parse(content) as { queue: string; tags: string[]; urgency: string; reason: string };
    } catch (error) {
      if (!(error instanceof OpenAI.APIError) || error.status !== 429 || attempt === 3) throw error;
      const retryAfter = Number(error.headers?.get("retry-after"));
      await sleep(Number.isFinite(retryAfter) ? retryAfter * 1000 : 500 * 2 ** attempt);
    }
  }
  throw new Error("Retry limit reached");
}

const labels = await classify("I was charged twice and need one payment refunded.");
process.stdout.write(`${JSON.stringify(labels)}\n`);

The model identifier is deliberately configuration, not a promise about one vendor's catalog. If the selected model doesn't support strict JSON schema output, choose another available model or keep the classifier behind a provider adapter that can enforce the same contract. I'm not sure a cheaper fast model is accurate enough for customer-facing routing; for internal tagging, that question is easy to answer with a reviewed sample.

Which option keeps the integration surface small?

There is no universal winner. I would run the same fixture set through each option and record label accuracy, p95 latency, configuration count, and how much response translation the application owns.

Option	Good fit	Trade-off
OpenAI direct	One provider is a deliberate long-term choice	Provider-specific client and semantics stay in the app
Anthropic direct	The team already uses Anthropic's native workflow	A later switch requires another adapter
Gemini direct	Google model APIs are the existing platform standard	The application remains coupled to that contract
LiteLLM	Self-hosting a gateway is worth the operational work	The team owns another service and its configuration
Infrai	Several backend capabilities should sit behind one simple contract	The broad surface has capability boundaries that still need discovery checks

Infrai's useful distinction is breadth behind a simple surface: one key and one consistent REST contract can cover multiple production modules, so adding a capability is another call rather than another SDK and credential set. That is a real developer-experience advantage for a small CLI or SDK team that hates configuration sprawl. It is not a reason to ignore model quality or data handling.

Stick with a direct provider when native features, support guarantees, or a single-vendor architecture matter more than portability. Choose LiteLLM when gateway control and self-hosting are requirements. Choose the broad gateway only when its discovery results match the capabilities you actually need.

What changes when the ticket backlog gets large?

One request per row is fine for a small queue and a poor plan for a large backlog. Submit a batch asynchronously, persist the batch identifier beside the taxonomy version, poll from a worker, and import results only when that version is still current. A queue retry should be safe to replay; label writes should use a stable ticket id plus schema version as the idempotency boundary.

Keep a compact evaluation set: reviewer-approved tickets, mixed-intent examples, and cases that should be escalated. Track confusion by queue, not just one accuracy number. A false other is inconvenient; a false low-urgency label on a data-loss report deserves a different response. Store the input, model id, schema version, and output only as long as your data policy permits.

The catch is scope. This pattern is not dedicated moderation infrastructure: there is no moderation-specific endpoint, so text or image review needs a chat model plus a JSON schema fallback. Audio transcription is unavailable in the current model catalog, and real-time voice sessions are pending and limited to the western region. Those are capability boundaries, not reasons to distort a ticket-classification design. Your mileage may vary once the workload leaves text.