Short answer: put one typed classification boundary between your Node.js app and chat completions, validate every JSON result there, and record raw usage against the tenant before any tag reaches the support queue.
| Choice | Tenant attribution | Configuration surface | Best fit |
|---|---|---|---|
| Direct runtime adapter | Explicit in application code | Small | One runtime and a latency-sensitive queue |
| Self-hosted gateway | Centralized across callers | Medium | Several runtimes or shared policy |
| Asynchronous batch worker | Explicit at job level | Larger | Backlogs that do not need an immediate tag |
For a B2B SaaS knowledge-base helpdesk, start with the direct adapter. It keeps the tenant ID, schema version, usage record, and accepted tags in one transaction-shaped flow. The recommendation changes when several teams need the same routing policy; then the gateway is the cleaner runner-up. Cost is not the opening argument here. Attribution is.
Privacy starts at the classification contract
Treat classification as a narrow function, not a conversation. Its input is a tenant-scoped ticket and a closed taxonomy. Its output is a small object that either passes local validation or does not enter the queue.
Private knowledge-base text should cross only the boundary required for classification. Keep tenant IDs out of prompts unless they change the classification task, redact fields the taxonomy does not need, and avoid copying ticket bodies into usage events. The schema limits the output; data minimization limits the input.
The JSON Schema should forbid extra fields and constrain tags to values your routing system already understands. A model-generated label such as urgent-ish may look reasonable to a person, but it is useless if the only accepted urgency values are low, normal, and high. Keep the rationale short as well. You are building an operational label, not asking for an essay.
There are two validation layers. The runtime is asked to produce output matching the schema, and the Node.js boundary validates the parsed value again. Don't let the first layer substitute for the second. Providers, adapters, and schema dialect support can change independently, while your queue contract still has to stay boring.
The taxonomy is the hard part. I'm not sure a universal support taxonomy exists; the right evidence would be your own routing rules and a reviewed sample of tenant tickets. Start with labels tied to actions, such as billing, access, product_issue, how_to, and other. If two labels trigger the same workflow, merging them removes both model ambiguity and downstream config. Your mileage may vary when a tenant brings its own queue structure, which is precisely why the schema version belongs in every stored result.
Cost attribution must follow the tenant, not the request
Per-tenant cost visibility begins before the request. Pass a stable tenant identifier into the classifier, but do not place it in the prompt unless the model needs it. Carry it as application context, attach it to the normalized usage result, and write the usage event before returning the classification. That sequence prevents a successful label from becoming detached from its accounting record. Store raw units rather than only a currency total: input units, output units, request count, runtime name, model alias, schema version, and timestamp. Pricing rules can be applied later with an effective-date version. This also makes a runtime comparison honest. A single average across all tenants hides the exact distribution a B2B operator needs to inspect. Benchmark the path with the traffic shape you expect. Measure accepted classifications per minute, p50 and p95 latency, validation rejection rate, retry count, and input/output units by tenant. Run at least one skewed case where a large tenant sends much longer tickets than the median tenant. No magic benchmark number applies across taxonomies, prompts, and runtimes, so publish the harness and the input distribution beside the results. Contract stability is the second criterion. Version the schema. Log the version with each result. Add a compatibility test that replays fixed, redacted inputs whenever the prompt, taxonomy, runtime, or model alias changes. The expected assertion is not exact prose; it is whether the output parses, uses allowed labels, and maps to the intended route.
Measure it.
Keep failures explicit — but outside the classification object. A malformed result is a validation failure, an HTTP 429 is a retryable transport outcome according to your policy, and an authentication failure should stop retries. Test those branches with injected adapter responses. Three attempts in a test fixture is a test parameter, not a universal production recommendation; your retry budget must come from the queue's latency objective.
Implement the accounting boundary in TypeScript
The code below leaves the full chat-completions URL and model alias in deployment configuration. That matters. The classifier should not guess a provider route, and swapping an adapter should not rewrite tenant metering.
const allowedTags = [
"billing",
"access",
"product_issue",
"how_to",
"other",
] as const;
const allowedUrgency = ["low", "normal", "high"] as const;
type TicketTag = (typeof allowedTags)[number];
type Urgency = (typeof allowedUrgency)[number];
type Ticket = {
tenantId: string;
ticketId: string;
subject: string;
body: string;
};
type Classification = {
tags: TicketTag[];
urgency: Urgency;
rationale: string;
};
type CompletionUsage = {
inputUnits: number;
outputUnits: number;
};
type CompletionResult = {
content: string;
usage: CompletionUsage;
runtime: string;
modelAlias: string;
};
type CompletionRequest = {
messages: Array<{ role: "system" | "user"; content: string }>;
jsonSchema: typeof ticketSchema;
};
type Complete = (request: CompletionRequest) => Promise<CompletionResult>;
type UsageEvent = CompletionUsage & {
tenantId: string;
ticketId: string;
runtime: string;
modelAlias: string;
schemaVersion: "ticket-tags-v1";
recordedAt: string;
};
type RecordUsage = (event: UsageEvent) => Promise<void>;
const ticketSchema = {
type: "object",
additionalProperties: false,
required: ["tags", "urgency", "rationale"],
properties: {
tags: {
type: "array",
items: { type: "string", enum: allowedTags },
minItems: 1,
uniqueItems: true,
},
urgency: { type: "string", enum: allowedUrgency },
rationale: { type: "string", minLength: 1, maxLength: 160 },
},
} as const;
function isOneOf<const T extends readonly string[]>(
values: T,
value: unknown,
): value is T[number] {
return typeof value === "string" && values.includes(value);
}
function parseClassification(content: string): Classification {
const value: unknown = JSON.parse(content);
if (typeof value !== "object" || value === null || Array.isArray(value)) {
throw new Error("classification must be an object");
}
const record = value as Record<string, unknown>;
const keys = Object.keys(record).sort();
const expectedKeys = ["rationale", "tags", "urgency"];
if (keys.join(",") !== expectedKeys.join(",")) {
throw new Error("classification has unexpected fields");
}
if (
!Array.isArray(record.tags) ||
record.tags.length === 0 ||
new Set(record.tags).size !== record.tags.length ||
!record.tags.every((tag) => isOneOf(allowedTags, tag))
) {
throw new Error("classification tags are invalid");
}
if (!isOneOf(allowedUrgency, record.urgency)) {
throw new Error("classification urgency is invalid");
}
if (
typeof record.rationale !== "string" ||
record.rationale.length < 1 ||
record.rationale.length > 160
) {
throw new Error("classification rationale is invalid");
}
return {
tags: record.tags,
urgency: record.urgency,
rationale: record.rationale,
};
}
export async function classifyTicket(
ticket: Ticket,
complete: Complete,
recordUsage: RecordUsage,
): Promise<Classification> {
const result = await complete({
messages: [
{
role: "system",
content: "Classify the ticket using only the supplied schema.",
},
{
role: "user",
content: `Subject: ${ticket.subject}\n\nBody: ${ticket.body}`,
},
],
jsonSchema: ticketSchema,
});
const classification = parseClassification(result.content);
await recordUsage({
tenantId: ticket.tenantId,
ticketId: ticket.ticketId,
inputUnits: result.usage.inputUnits,
outputUnits: result.usage.outputUnits,
runtime: result.runtime,
modelAlias: result.modelAlias,
schemaVersion: "ticket-tags-v1",
recordedAt: new Date().toISOString(),
});
return classification;
}
Complete is the only runtime-specific seam. An HTTP adapter can call the complete URL from CHAT_COMPLETIONS_URL, translate that runtime's request and response shapes, and normalize usage into inputUnits and outputUnits. The business function doesn't need an SDK, model ID, or provider-specific route.
One detail deserves extra scrutiny: the usage write and queue update are separate side effects in many systems. If losing either record is unacceptable, publish both through one durable application transaction or an outbox, then make consumers idempotent on ticketId plus schemaVersion. Don't pretend two unrelated network calls are atomic.
That is the boundary.
What should reliability tests prove before deployment?
Replay a redacted fixture set and inspect results by tenant. After deployment, graph validation rejection rate, latency, retries, tag distribution, and raw usage units. Alert on a sudden distribution shift even when every response remains valid JSON; a contract can be syntactically healthy while the routing behavior has changed.
Review taxonomy changes like database migrations. Add the new label, teach downstream routing about it, deploy the classifier, and only then remove an old label. Fast is good. Recoverable is better. The durable design is plain: one classification contract, one runtime adapter, one tenant-scoped usage event, and tests at every boundary. That gives a SaaS operator an answer to “which tenant consumed what?” without tying the queue to a particular runtime.
How can Node.js migrate LLM support ticket classification to a shared gateway?
Use a self-hosted gateway when several services need centralized credentials, routing policy, or a shared normalization layer. An open-source gateway can make that boundary inspectable, and it can reduce duplicated adapter code. The catch is another deployed component, another configuration surface, and another place where tenant context must be preserved. It is not suitable when the team cannot operate that layer or when one small service talks to one stable runtime.
Stick with an asynchronous worker when tickets already arrive through a durable queue and users do not wait for the label. It absorbs bursts and gives retries room to breathe. The trade-off is delayed routing plus more state: queued, attempted, classified, rejected, and dead-lettered work all need definitions and metrics.
The direct adapter is also the wrong choice once policy consistency across many callers matters more than time-to-first-call. Move the boundary outward then. Keep the same schema versions and usage-event contract, because those are application assets rather than runtime features.
Top comments (0)