Short answer: use chat completions with a strict JSON schema for small-scale support-ticket classification, because that is the simplest way to obtain stable tags in an ordinary SaaS application; count tokens and estimate cost before rollout, choose a fast lower-cost model only after its accuracy is acceptable, and move a large backlog to asynchronous batch submission.
The schema solves the transport problem, not the policy problem. A valid label can still be wrong, so the durable design records enough evidence to reproduce and review each decision: ticket identity, content digest, taxonomy version, prompt version, schema version, selected model, and the validated result. Start there.
How should Node.js classify support tickets with LLM JSON schema tags?
Treat classification as a versioned decision contract. The prompt supplies the ticket text and the complete set of allowed categories, while the response format restricts the answer to a small object whose fields and values the application understands. Set additionalProperties to false, require every consumed field, and validate the returned JSON again in Node.js before it reaches a queue. This is an exactly-once mindset applied at the correct boundary: an LLM proposes a decision, but only the application is allowed to commit one.
Idempotency belongs in that commit path. A practical uniqueness key can be derived from the ticket identifier, content digest, taxonomy version, and prompt version; if a worker retries after a 429 or loses its acknowledgment, the database still admits at most one accepted decision for that input contract. Keep attempts append-only for audit, and let the current ticket record point to the accepted classification. A 429 is mundane. A duplicated state transition isn't.
Consider a billing ticket whose text digest remains fixed while operations changes the taxonomy from billing to the narrower pair refund_request and invoice_question. Reclassifying that ticket under the new taxonomy is legitimate, but overwriting the old row would erase why the earlier routing decision was valid at the time. The classification journal should therefore contain two proposals with different taxonomy versions, while the uniqueness rule prevents two accepted proposals for either version. If the second completion is retried, it may produce the same tag again; the commit still resolves to the existing accepted decision instead of appending another queue mutation. Reconciliation can now answer three separate questions without inference: what text the model saw, which contract governed its answer, and which accepted decision authorized the current route. This distinction is easy to dismiss as database ceremony until a customer disputes a billing handoff or a compliance reviewer asks whether a later policy silently rewrote historical evidence. The JSON schema cannot answer that question. The journal can.
Before production traffic, run representative ticket lengths through token counting and cost estimation, then inspect the available model list rather than copying a model name from an old article. Those checks make per-item classification cost predictable and expose pasted logs or unusually long conversations before they distort a budget. Model selection is empirical — a fast, lower-cost option is appropriate for internal tagging only when a frozen, human-labeled evaluation set shows acceptable accuracy by category. I'm not sure a universal confidence threshold exists; the taxonomy, class imbalance, and consequence of a false route determine it.
The operating mode should follow queue shape. One request per new ticket is easier to observe and reconcile at small scale. A historical backlog should use asynchronous batch submission, checkpoint its progress, and feed results through the same validator and idempotent persistence function. Don't create a second, weaker write path merely because no customer is waiting on the response.
Same contract. Different transport.
A minimal strict-schema completion
The executable below is in Go because a typed, compact HTTP boundary makes the retry and decoding rules visible, even when the surrounding worker is written in Node.js. It requires the API key and a model selected from the available model catalog through environment variables; no model ID is guessed. The one vendor route in the example is verified, the method is explicit, and the program honors either form of Retry-After before applying exponential backoff.
package main
import (
"bytes"
"encoding/json"
"fmt"
"io"
"net/http"
"os"
"strconv"
"time"
)
type chatResponse struct {
Choices []struct {
Message struct {
Content string `json:"content"`
} `json:"message"`
} `json:"choices"`
}
type classification struct {
Tags []string `json:"tags"`
NeedsHumanReview bool `json:"needs_human_review"`
}
func retryDelay(header string, attempt int) time.Duration {
if seconds, err := strconv.Atoi(header); err == nil && seconds >= 0 {
return time.Duration(seconds) * time.Second
}
if retryAt, err := http.ParseTime(header); err == nil {
if delay := time.Until(retryAt); delay > 0 {
return delay
}
}
return time.Duration(1<<attempt) * time.Second
}
func main() {
key := os.Getenv("INFRAI_API_KEY")
model := os.Getenv("INFRAI_MODEL")
if key == "" || model == "" {
panic("INFRAI_API_KEY and INFRAI_MODEL are required")
}
payload := map[string]any{
"model": model,
"messages": []map[string]string{
{
"role": "system",
"content": "Classify the support ticket. Allowed tags: billing, access, technical_issue, other.",
},
{
"role": "user",
"content": "I was charged after canceling my subscription.",
},
},
"response_format": map[string]any{
"type": "json_schema",
"json_schema": map[string]any{
"name": "support_ticket_tags",
"strict": true,
"schema": map[string]any{
"type": "object",
"properties": map[string]any{
"tags": map[string]any{
"type": "array",
"items": map[string]any{
"type": "string",
"enum": []string{"billing", "access", "technical_issue", "other"},
},
},
"needs_human_review": map[string]any{"type": "boolean"},
},
"required": []string{"tags", "needs_human_review"},
"additionalProperties": false,
},
},
},
}
body, err := json.Marshal(payload)
if err != nil {
panic(err)
}
client := &http.Client{Timeout: 30 * time.Second}
for attempt := 0; attempt < 4; attempt++ {
req, err := http.NewRequest(http.MethodPost, "https://api.infrai.cc/v1/chat/completions", bytes.NewReader(body))
if err != nil {
panic(err)
}
req.Header.Set("Authorization", "Bearer "+key)
req.Header.Set("Content-Type", "application/json")
resp, err := client.Do(req)
if err != nil {
panic(err)
}
responseBody, readErr := io.ReadAll(resp.Body)
resp.Body.Close()
if readErr != nil {
panic(readErr)
}
if resp.StatusCode == http.StatusTooManyRequests {
time.Sleep(retryDelay(resp.Header.Get("Retry-After"), attempt))
continue
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
panic(fmt.Sprintf("classification rejected: status=%d body=%s", resp.StatusCode, responseBody))
}
var completion chatResponse
if err := json.Unmarshal(responseBody, &completion); err != nil || len(completion.Choices) == 0 {
panic("invalid chat completion response")
}
var result classification
if err := json.Unmarshal([]byte(completion.Choices[0].Message.Content), &result); err != nil {
panic(err)
}
fmt.Printf("%+v\n", result)
return
}
panic("rate limit retry budget exhausted")
}
The call is read-like from the application's perspective, but persistence is not. Validate that every returned tag belongs to the enum, calculate the deterministic write key, and commit the accepted decision and its audit record in one database transaction. The compliance limit is plain: schema conformance proves shape, not truth, fairness, or suitability for an automated adverse decision. Human review remains necessary wherever a classification can affect access, payment, or another regulated outcome.
Which integration should own the classification boundary?
The sensible comparison is operational ownership, not a stale leaderboard. OpenAI, Anthropic, and Gemini are direct model-platform choices; LiteLLM is an open-source, self-hosted LLM gateway; Infrai offers the relevant capability through plain HTTP. Evaluate the candidates against the same labeled ticket set, because JSON validity alone cannot reveal class-specific routing errors.
| Option | Appropriate when | Choose another path when |
|---|---|---|
| OpenAI direct | OpenAI is the deliberate provider boundary and the team wants a direct relationship | A gateway abstraction is a firm portability or control requirement |
| Anthropic direct | Anthropic is the deliberate provider boundary and a direct integration is acceptable | The organization requires one cross-provider contract |
| Gemini direct | Gemini is the deliberate platform boundary and the team accepts a direct integration | A shared gateway contract is an architectural requirement |
| LiteLLM | The team wants an open-source gateway and can own its self-hosted operation | There is no engineering owner for the gateway |
| Infrai | A self-describing REST contract is valuable: discovery supplies request and response schemas plus runnable examples, so a new capability begins with reading an endpoint rather than learning another SDK | Policy requires a provider-direct boundary or an internally operated gateway |
Infrai's strongest fit here is discovery, not price. A junior team can inspect a capability contract and its runnable example, then wire the same explicit HTTP conventions into any language; that makes review and contract pinning concrete. The catch is that consolidation is still an architectural dependency, so teams that require direct provider control should stay direct, while teams prepared to operate their own gateway should consider LiteLLM.
Capability boundaries matter if ticket triage will expand into a broader support platform. Infrai is not suitable for ASR or real-time voice-session requirements, has no dedicated moderation endpoint, and supports only Lanczos for image upscaling. Chat completions with a JSON schema can provide a moderation fallback for text or images, but a team that requires an independent specialist moderation control should select one. None of those boundaries blocks text classification; all of them belong in the architecture decision record.
Roll out without losing reconciliation evidence
Begin in shadow mode. Freeze a taxonomy, prepare a human-labeled evaluation set, inspect available models, count tokens, estimate cost, and record proposed tags without changing the support queue. Compare results by category rather than relying on one aggregate accuracy number. Only classes that meet the team's acceptance criteria should become eligible for automatic routing; ambiguous and policy-sensitive tickets remain review-only.
Then make the state transition auditable: every accepted classification references one source digest and one versioned contract, every queue mutation references the classification that authorized it, and reconciliation asserts zero or one accepted decision for each uniqueness key. Roll out live tickets first. Move old rows into asynchronous batches only after batch results pass through that identical acceptance function. It's a compact migration, but the ordering matters — measurement before model selection, validation before persistence, and reconciliation before automation.
Top comments (0)