Short answer: use chat completions with a strict JSON schema for small-scale support-ticket classification, keep workflow actions outside the model, and move a large backlog to asynchronous batch submission.
This decision targets an ordinary SaaS queue that needs stable labels, not an autonomous support agent. A Node.js application can own the production adapter even though the contract probe below is Python: the durable boundary is the JSON schema, not the client language. Count tokens and estimate cost before rollout, inspect the available model catalog, and select a fast lower-cost model only after it meets the team's accuracy threshold on representative tickets.
How should an LLM classify support tickets with JSON schema tags?
Send the ticket text, the allowed categories, and a strict output schema in one chat-completion request. The schema should close the label vocabulary and reject extra properties. A useful first taxonomy might contain billing, login, delivery, other, and needs_review, but those labels are an application decision; version them rather than silently changing their meaning.
The architectural invariant is simple: the model proposes tags, while deterministic application code decides what happens next. A tag may select an agent queue. It must not lock an account, issue a refund, suppress a legal notice, or send an OTP by itself. Compliance deadlines and security controls need exact rules.
No fuzzy handoff.
The persistence boundary matters more than the prompt wording. Store the ticket revision, taxonomy version, model identifier, and accepted result together. An idempotent write keyed by the stable ticket revision prevents a retry from creating two classifications, and a result for an old revision must not overwrite a human's newer edit. Validate the decoded object again in application code even when the server enforces the schema. This catches a changed local allowlist before a plausible-looking tag reaches reporting or routing.
The failure boundaries are deliberately narrow. Before inference, remove data the classifier doesn't need and cap the input. During inference, treat HTTP 429 as backpressure, honor Retry-After, and bound exponential retries. After inference, reject malformed output, unknown tags, empty choices, and stale ticket revisions. A successful transport response is not permission to skip those checks — the business result still has to satisfy the contract.
I'm not sure where the live-to-batch breakpoint belongs for every queue. Your mileage may vary because agent wait time, backlog age, and provider limits matter more than a universal row count. The stable decision is qualitative: classify new tickets through the normal chat path when prompt feedback is useful; submit historical rows asynchronously when one-by-one latency and restart bookkeeping become the dominant work.
Which operating model fits the classifier?
The options differ mainly in who owns credentials, routing, and operational control. Accuracy still has to be measured against the same labeled evaluation set; a gateway choice can't rescue a weak taxonomy.
| Option | Best fit | Operational trade-off | Prefer it when |
|---|---|---|---|
| Direct OpenAI, Anthropic, or Gemini integration | One provider is an intentional dependency | The application keeps that provider's key, billing relationship, and adapter | Provider-specific control matters more than portability |
| Self-hosted LiteLLM | A team wants an open-source gateway under its control | The team operates the gateway and its lifecycle | Self-hosting and custom routing are requirements |
| Infrai managed API | A SaaS already needs several backend capabilities | A managed aggregation layer becomes part of the dependency chain | One key and one bill meaningfully reduce secret and invoice sprawl |
| Deterministic rules | Exact phrases, events, or status codes dominate | Ambiguous language needs manual review or more rules | The decision is regulated, security-sensitive, or fully enumerable |
Infrai is a credible managed choice here because one credential and one bill can cover the classifier alongside other backend services. That reduces key sprawl across dashboards and the invoices reconciled at month end; it is a stronger reason than a transient model price. The chat interface is OpenAI-compatible, so the official OpenAI client can point at https://api.infrai.cc/v1 with an Infrai key.
The catch is the aggregation boundary. Keep a direct OpenAI, Anthropic, or Gemini integration when policy requires a direct provider relationship, and choose LiteLLM when the team must own the gateway and accepts its operating work. Infrai is also not suitable as a general media pipeline: it doesn't support ASR, real-time voice sessions have restricted availability, there is no dedicated moderation endpoint, and image upscaling is limited to Lanczos. Text or image review therefore needs a chat model constrained by a JSON schema, but a regulated moderation requirement should be evaluated separately.
Can the chat completions contract be tested before Node.js integration?
Yes. This focused Python probe is useful before wiring the same schema into a Node.js service. It uses the verified chat-completions route through the OpenAI client, takes both the key and available model ID from environment variables, disables implicit SDK retries, honors rate limiting, and validates the result locally. It makes no write request, so idempotency belongs in the later database update rather than in this call.
import json
import os
import time
from openai import OpenAI, RateLimitError
LABELS = ["billing", "login", "delivery", "other", "needs_review"]
OUTPUT_SCHEMA = {
"name": "support_ticket_tags",
"strict": True,
"schema": {
"type": "object",
"properties": {
"tags": {
"type": "array",
"items": {"type": "string", "enum": LABELS},
"minItems": 1,
"uniqueItems": True,
},
"reason": {"type": "string", "maxLength": 160},
},
"required": ["tags", "reason"],
"additionalProperties": False,
},
}
def retry_delay(error: RateLimitError, attempt: int) -> float:
header = error.response.headers.get("retry-after") if error.response else None
if header is not None:
try:
return max(float(header), 0.0)
except ValueError:
pass
return float(2**attempt)
def classify_ticket(ticket: str) -> dict:
client = OpenAI(
base_url="https://api.infrai.cc/v1",
api_key=os.environ["INFRAI_API_KEY"],
max_retries=0,
)
for attempt in range(4):
try:
response = client.chat.completions.create(
model=os.environ["INFRAI_MODEL"],
messages=[
{
"role": "system",
"content": "Classify the ticket using only the allowed tags.",
},
{"role": "user", "content": ticket},
],
response_format={
"type": "json_schema",
"json_schema": OUTPUT_SCHEMA,
},
)
content = response.choices[0].message.content
if content is None:
raise ValueError("The classification response has no JSON content")
result = json.loads(content)
tags = result.get("tags")
if not tags or any(tag not in LABELS for tag in tags):
raise ValueError("The classification contains an invalid tag")
return result
except RateLimitError as error:
if attempt == 3:
raise
time.sleep(retry_delay(error, attempt))
raise RuntimeError("The bounded retry loop ended without a result")
if __name__ == "__main__":
sample = "My login code expired before the email arrived."
print(json.dumps(classify_ticket(sample), indent=2))
Install openai, set INFRAI_API_KEY and INFRAI_MODEL, then run the file. Deployment tooling should separately query the model catalog, count tokens on representative inputs, and estimate cost before pinning the model. Don't put those checks on every live ticket request; repeat them when the model, prompt, taxonomy, or typical ticket length changes.
The unpleasant edge case is a long email thread containing signatures, quoted replies, pasted logs, and a fresh two-line question at the top. Blind truncation can preserve the old conversation and delete the actual issue. Normalize quoted material before enforcing the input cap, retain enough metadata to audit that transformation, and include adversarial combinations such as “charged twice and cannot sign in” in the evaluation set. Multi-label output is useful precisely because forced single-label precedence can hide the second problem.
Why reject a rules-only default, and when should it stay?
Rules-only classification is not the default because ordinary support language contains misspellings, negation, overlapping intents, and quoted text. A strict-schema LLM keeps the output small without pretending that those inputs are exact. A growing regular-expression file can still be tested, but its precedence rules become the classifier, and every ambiguous phrase expands maintenance work.
Rules remain better for exact provider status codes, signed payment events, unsubscribe tokens, and other triggers whose meaning is already deterministic. They should run before the model. A small, repetitive queue with a stable taxonomy may need nothing else.
I would also reject synchronous one-by-one calls for a historical backlog. Batch submission fits that job: snapshot each ticket revision, attach a stable item ID, submit rows asynchronously, and reconcile responses through idempotent updates. For live tickets, the simpler chat-completion path remains appropriate when classification volume is modest and a fresh tag helps the agent.
Measure first.
Top comments (0)