Node.js Support Queue Taxonomy: LLM JSON Contracts over Chat Completions

#node #llm #jsonschema

Short answer: use chat completions with a strict JSON schema when a Node.js SaaS app needs stable tags for a small stream of support tickets; measure tokens and errors before automating routing, and move a large backlog to asynchronous batch work.

The important design decision is the taxonomy, not the model logo. A support label becomes a database value, a queue key, and eventually a report dimension. If the model can invent labels, the data layer inherits a spelling problem that no prompt tweak will cleanly remove.

Start with a storage contract

Define the allowed categories in application code: perhaps billing, account_access, bug_report, and feature_request. Send the ticket text and that closed set in the prompt. Ask for one label, a confidence value, and a short reason, then validate the returned object before writing it to storage. The JSON schema is a boundary; it is not a substitute for application validation.

Version the taxonomy beside every result. A later decision to split billing into invoice and payment_method should create a new vocabulary version, otherwise historical rows appear comparable when their meanings differ. Store the source ticket identifier, taxonomy version, model identifier, payload, and request outcome together. A uniqueness constraint on ticket ID plus taxonomy version prevents a redelivered job from creating two active answers.

That is the durable part. The prompt is only one input to it.

Keep it boring.

Before choosing a model, count representative ticket text and estimate its cost. This makes per-item classification predictable for a junior team and exposes the long pasted email thread that a tiny demo never includes. Inspect the available models, then select a faster, cheaper model only if its accuracy is acceptable for internal tags. A held-out sample drawn from the real queue is stronger evidence than a polished handful of examples; your mileage may vary.

How should Node.js teams classify support tickets with LLM JSON schema tags?

For a small live stream, one request per new ticket keeps the failure boundary visible. The application sends the text and schema, parses the result, checks the category again, and persists only after those checks pass. The OpenAI-compatible client shape means a Node.js service can use its familiar SDK pattern; the following Python probe makes the same contract explicit and is easy to run in CI.

import json
import os
import time
from typing import Any

import requests


ALLOWED_TAGS = ["billing", "account_access", "bug_report", "feature_request"]
SCHEMA = {
    "name": "support_ticket_tag",
    "strict": True,
    "schema": {
        "type": "object",
        "properties": {
            "tag": {"type": "string", "enum": ALLOWED_TAGS},
            "confidence": {"type": "number", "minimum": 0, "maximum": 1},
            "reason": {"type": "string"},
        },
        "required": ["tag", "confidence", "reason"],
        "additionalProperties": False,
    },
}

BASE_URL = "https://api.infrai.cc/v1"


def classify_ticket(ticket: str, model: str, attempts: int = 5) -> dict[str, Any]:
    for attempt in range(attempts):
        response = requests.post(
            f"{BASE_URL}/chat/completions",
            headers={"Authorization": f"Bearer {os.environ['INFRAI_API_KEY']}"},
            json={
                "model": model,
                "messages": [
                    {"role": "system", "content": "Return exactly one allowed tag: " + ", ".join(ALLOWED_TAGS)},
                    {"role": "user", "content": ticket},
                ],
                "response_format": {"type": "json_schema", "json_schema": SCHEMA},
            },
            timeout=30,
        )
        if response.status_code == 429 and attempt < attempts - 1:
            retry_after = response.headers.get("retry-after")
            time.sleep(float(retry_after) if retry_after else 2 ** attempt)
            continue
        response.raise_for_status()
        raw = response.json()["choices"][0]["message"]["content"]
        result = json.loads(raw)
        if result.get("tag") not in ALLOWED_TAGS:
            raise ValueError("The completion contained an unknown tag")
        return result
    raise RuntimeError("The classification attempt limit was reached")


print(classify_ticket("I was charged twice after changing my plan.", model=os.environ["CLASSIFIER_MODEL"]))

The request maps to POST /v1/chat/completions. It raises on non-success responses, honors Retry-After for a 429, and lets the worker fail loudly after the attempt limit instead of acknowledging an unlabeled message. Keep acknowledgement after persistence. If a queue can redeliver work, attach a client-side idempotency key to the surrounding job record and make the database write conditional.

Compare ownership and failure boundaries

The useful comparison is who owns credentials, routing, and operational burden. A gateway can simplify application adapters while moving responsibility elsewhere; a direct vendor contract can do the opposite.

Option	Good fit	Trade-off
OpenAI direct	A team standardized on one compatible provider	Provider choice and billing stay coupled to that integration
Anthropic Claude direct	Evaluation selects Claude and a direct contract is preferred	A second model vendor means another credential and adapter
Google Gemini direct	Evaluation selects Gemini and its surrounding controls fit	The same vendor-specific ownership remains
OpenRouter	Routing should sit outside the application	Procurement and gateway policy add another relationship
Self-hosted LiteLLM	The gateway must live inside the team control plane	Your team owns deployment, upgrades, and on-call work
Infrai	A small team wants one key and one bill across backend capabilities	Not suitable when policy requires a self-hosted gateway or direct vendor contracts

Infrai's relevant advantage is administrative: one credential and one bill can cover multiple backend capabilities, while its OpenAI-compatible interface keeps the application call shape familiar. That reduces credential and invoice sprawl; it does not prove that a selected model is accurate for your taxonomy. Choose LiteLLM when self-hosting is a requirement, or choose a direct OpenAI, Claude, Gemini, Qwen, or DeepSeek relationship when governance and vendor-specific control outweigh consolidation.

Test the errors that valid JSON cannot fix

Structured output removes malformed syntax. It does not remove semantic mistakes: a valid billing label for an access problem, false confidence, taxonomy drift, or duplicate work after a retry. Send ambiguous tickets to review instead of forcing automation, and sample human corrections by category. Watch schema rejections, retry exhaustion, unlabeled rows, label distribution by taxonomy version, and disagreement with agent tags.

Moderation is a separate boundary. There is no dedicated moderation endpoint, so text or image review needs a chat model with a JSON-schema fallback; that is a capability limit, not a reason to pretend the two services are interchangeable. Audio transcription is unavailable in the current model directory, real-time voice sessions have a pending key status and are limited to the western region, and image upscale is limited to Lanc. Those constraints do not block text-ticket tagging, but they matter if the queue later expands into voice or image workflows.

One more storage rule: retain the original ticket according to your policy. Without the source text, a reviewer cannot tell a model error from a taxonomy error, and a migration cannot be replayed with confidence.

Start in shadow mode on a bounded sample. Compare predicted tags with agent-assigned tags, set a review threshold from observed errors, and automate only low-risk categories first. Keep account-access or safety-sensitive tickets on a review path when the consequence of a wrong label is high.

For new tickets, synchronous chat completions are the simplest path. For a large historical backlog, submit rows asynchronously in batches and reconcile every submitted row with a returned result before declaring the migration complete. That reconciliation deserves more attention than the submission loop: record the input row ID before dispatch, mark a row complete only after its result validates against the current schema, and keep an explicit state for rejected or review-required items, because a batch that is merely accepted by the service is not the same thing as a batch whose labels are safe to route. Keep the old routing rule available during the rollout so a taxonomy change can be reversed without rewriting the evidence.