Per-Tenant Media Scheduling Explained: Node.js LLM JSON Schema for Ticket Tags

#node #llm #sre

Per-tenant cost visibility changes the unit of work. A media support ticket cannot be just prompt text in a global queue; it needs a tenant, stable ticket identity, taxonomy version, and prompt version before any worker is allowed to classify it.

Short answer: enqueue normalized tickets from Node.js, schedule them with an explicit per-tenant policy, require an LLM response that passes a strict JSON Schema, and commit the tags and returned usage to one durable ledger record under an idempotency key.

Do that before tuning prompts. Otherwise a good classifier can still starve a small tenant, spend twice on a duplicate delivery, or leave an operator unable to reconcile usage with completed work.

Can a Node.js failure drill classify support tickets with LLM JSON Schema tags?

Start at admission control, not at Chat Completions. The Node.js intake handler should normalize the ticket and create a durable job; it should not hold the customer request open while the model runs. Each job needs tenant_id, ticket_id, the redacted subject and body, taxonomy_version, prompt_version, and an enqueue timestamp. A stable classification key can be derived from the two identifiers and two versions.

The scheduler then chooses a tenant before choosing that tenant's oldest ready job. That ordering matters during a broadcast incident. Imagine Tenant A contributing 800 playback reports after a player release while Tenant B contributes eight time-sensitive rights reports. Global FIFO is easy to explain, but A can occupy every classifier slot. Round-robin protects a turn for each active tenant, though it cannot express different capacity commitments. Weighted scheduling can express those commitments, at the cost of reviewed weights, per-tenant caps, and more runbook state.

There is no universally correct policy. Use global FIFO when all work has the same owner and service objective. Use round-robin when tenant isolation is more important than proportional allocation. Use weighted scheduling when capacity commitments are real enough to maintain as configuration. In every case, write the policy name and configuration version beside the result so an incident review can distinguish classifier behavior from scheduler behavior.

This is the decision rule: the simplest policy that keeps each tenant's oldest-ready age inside its objective wins. Aggregate queue depth isn't enough because it can look healthy while one tenant waits.

Implement the classification contract as a narrow API

The model contract should be smaller than the support taxonomy used by humans. For a media queue, a useful first set might allow playback, upload, rights, billing, and other, with low, normal, high, and urgent priorities. The tags are routing inputs, so reject unknown categories rather than treating a near match as close enough. Keep additionalProperties false, cap the tag count, and require every field that downstream automation reads.

The Chat Completions adapter belongs behind a narrow interface. This keeps provider-specific request translation away from queue ownership, retry logic, and the tenant ledger. The following Go example builds the classification contract and validates the returned JSON with only the standard library. A Node.js producer can enqueue the same Job shape; no vendor route or model identifier is baked into the worker.

package triage

import (
    "bytes"
    "context"
    "encoding/json"
    "errors"
    "fmt"
    "slices"
)

type Job struct {
    TenantID       string `json:"tenant_id"`
    TicketID       string `json:"ticket_id"`
    Subject        string `json:"subject"`
    Body           string `json:"body"`
    TaxonomyVersion string `json:"taxonomy_version"`
    PromptVersion   string `json:"prompt_version"`
}

type Message struct {
    Role    string `json:"role"`
    Content string `json:"content"`
}

type CompletionRequest struct {
    Messages []Message      `json:"messages"`
    Schema   map[string]any `json:"json_schema"`
}

type Usage struct {
    InputUnits  int64 `json:"input_units"`
    OutputUnits int64 `json:"output_units"`
}

type Completion struct {
    JSON  []byte
    Usage Usage
}

type Completer interface {
    Complete(context.Context, CompletionRequest) (Completion, error)
}

type Result struct {
    Category string   `json:"category"`
    Priority string   `json:"priority"`
    Tags     []string `json:"tags"`
    Summary  string   `json:"summary"`
}

func schema() map[string]any {
    return map[string]any{
        "type":                 "object",
        "additionalProperties": false,
        "required":             []string{"category", "priority", "tags", "summary"},
        "properties": map[string]any{
            "category": map[string]any{"type": "string", "enum": []string{"playback", "upload", "rights", "billing", "other"}},
            "priority": map[string]any{"type": "string", "enum": []string{"low", "normal", "high", "urgent"}},
            "tags":     map[string]any{"type": "array", "items": map[string]any{"type": "string"}, "maxItems": 5},
            "summary":  map[string]any{"type": "string"},
        },
    }
}

func classify(ctx context.Context, c Completer, job Job) (Result, Usage, error) {
    ticket, err := json.Marshal(job)
    if err != nil {
        return Result{}, Usage{}, fmt.Errorf("encode ticket: %w", err)
    }

    req := CompletionRequest{
        Messages: []Message{
            {Role: "system", Content: "Classify the media support ticket using the supplied schema."},
            {Role: "user", Content: string(ticket)},
        },
        Schema: schema(),
    }
    completion, err := c.Complete(ctx, req)
    if err != nil {
        return Result{}, Usage{}, fmt.Errorf("complete classification: %w", err)
    }

    var result Result
    decoder := json.NewDecoder(bytes.NewReader(completion.JSON))
    decoder.DisallowUnknownFields()
    if err := decoder.Decode(&result); err != nil {
        return Result{}, Usage{}, fmt.Errorf("decode classification: %w", err)
    }
    if err := validate(result); err != nil {
        return Result{}, Usage{}, err
    }
    return result, completion.Usage, nil
}

func validate(result Result) error {
    if !slices.Contains([]string{"playback", "upload", "rights", "billing", "other"}, result.Category) {
        return errors.New("category outside taxonomy")
    }
    if !slices.Contains([]string{"low", "normal", "high", "urgent"}, result.Priority) {
        return errors.New("priority outside taxonomy")
    }
    if len(result.Tags) > 5 || result.Summary == "" {
        return errors.New("classification violates local constraints")
    }
    return nil
}

Keep validation on the consumer even if constrained generation is available. The worker owns the business boundary. It must enforce enums, required fields, maximum tag count, and the unknown-field rule before a result can affect routing.

No guesswork.

Put cost attribution in the committed record

Scheduling and accounting become easier to reason about when one classification has three durable states: a ready job, a leased attempt, and a committed result. A delivery is only an attempt. It is never proof that the ticket was classified exactly once.

I've been paged by missed jobs and duplicate deliveries, so the idempotency reflex is deliberate here. Before opening an LLM request, claim the classification key with a lease. After schema validation, commit only if the worker still owns that lease. If a retry finds a completed key, return the stored classification instead of spending again. If it finds an expired lease, it may claim a new attempt. Set lease duration beyond the request deadline plus expected commit time, and expose lease renewal as a metric.

The committed record should contain the tenant and ticket identifiers, structured result, returned usage fields, taxonomy and prompt versions, scheduling policy version, attempt identifier, and completion time. Attribute usage to the tenant from the durable job, never from text generated by the model. This gives operations one record for answering two separate questions: what routing decision was made, and what reported usage produced it.

Request-level usage may still be insufficient for defensible chargeback. I'm not sure precise per-ticket allocation is possible when a runtime does not return usable accounting at that granularity; controlled reconciliation against billing exports is what resolves that uncertainty. Until then, call the numbers estimates. Don't turn inferred units into an invoice merely because they fit neatly in a table.

Taxonomy drift is a separate transition. A valid JSON object can still represent a changed routing policy after a prompt edit. Store both versions with every committed result, evaluate a fixed redacted set before promotion, and compare label behavior by tenant and language. Rollback should direct new jobs to the prior versions. It should not rewrite history.

Evaluate the queue before promotion

Test the queue as an operator would encounter it. Unit tests should reject missing fields, extra fields, invalid enum values, empty summaries, and more than five tags. Integration tests should deliver the same job twice, expire a lease, and time out an attempt while asserting that one classification key produces at most one committed result. A staging replay should use redacted tickets and reconcile committed counts and returned usage by tenant.

Four signals belong on the first dashboard:

Signal	Failure it reveals	Runbook response
Oldest-ready age by tenant	Starvation hidden by aggregate depth	Inspect tenant slots and scheduling weights
Duplicate claim count	Delivery pressure or short leases	Compare lease duration with request and commit time
Schema rejection rate by version	Contract or prompt drift	Stop promotion and inspect redacted samples
Committed usage by tenant and version	Attribution gaps	Reconcile ledger records with billing data

Dead-letter records need the last error class, attempt count, versions, tenant, and ticket identifier. Ticket text should remain under the support system's normal access and retention controls. Requeue only after the error class is understood; blind replay can convert a contained backlog into duplicate model work.

Deploy in shadow mode first, recording classifications and usage without changing production routing. Canary by a stable tenant subset so a single tenant is not split unpredictably across policies. The promotion record should name the owner, cohort, prompt and taxonomy versions, scheduling policy, evaluation result, and rollback controls. Thresholds for queue age, schema rejection, label quality, and accounting reconciliation belong to the service's own SLO and risk budget. Your mileage may vary.

Rollout, rollback, and the honest boundary

Rollback needs two independent controls. One sends new jobs to the previous prompt and taxonomy versions. The other stops automated routing while preserving queued jobs and classification records for inspection. Don't delete the queue or mutate old results during an incident; choose an explicit version before draining or replaying work.

The catch is operational weight. A durable queue, lease store, idempotency record, evaluation set, and per-tenant metrics are not suitable when ticket volume is tiny and humans already triage inside the required response time. Keep manual routing with offline suggestions in that case. A synchronous Node.js request can also be reasonable for an internal prototype with no automated action and no chargeback requirement.

Once tags trigger customer-facing routing or tenant invoices, the scheduled design earns its keep. A direct model integration keeps the component count lower, while a self-hosted gateway centralizes translation and adds a service the team must operate. LiteLLM is one open-source example of that gateway pattern, not a recommendation. Choose the boundary according to incident ownership and migration needs.