Scheduling LLM Code Review Findings with JSON Schema Tags and Tenant Cost Attribution

Short answer: use Node.js to classify support tickets with an LLM behind an idempotent scheduled job, require one strict JSON Schema result before changing queue state, and charge usage to the tenant recorded on the immutable work item rather than to whichever worker happened to process it.

For a fintech support queue that accepts code-change review requests, that is the least complex design I would trust. The model can return structured findings and routing tags, but it doesn't get to decide whether work is new, who pays for it, or whether a result may be published. Those are scheduler and ledger decisions.

I've been paged by missed jobs and duplicate deliveries. The lasting lesson wasn't “pick a better model.” It was that a plausible classification can still leave an operational mess when the same change is reviewed twice, a retry is billed to the wrong tenant, or a free-form label silently creates a new queue.

Keep the boundary boring.

How should Node.js chat completions classify support tickets into JSON Schema tags?

Treat the Node.js caller as a producer of a durable review request, not as the owner of the entire workflow. Each request needs a stable work ID, a tenant ID, a taxonomy version, and a content digest. A scheduled consumer can then submit the ticket and change summary through a chat-completions-shaped adapter, validate the JSON response, and commit the result. The same contract works if the model call is local, direct, or routed through a gateway.

The response should contain only fields the next system can act on. In this example, a fintech change-review ticket yields a severity, a bounded set of tags, and structured findings. “Interesting” prose is not part of the contract. Neither is a model-generated tenant ID; accepting one would let untrusted output alter cost ownership.

package review

import (
    "encoding/json"
    "errors"
    "fmt"
)

var allowedTags = map[string]bool{
    "auth": true, "data-access": true, "payments": true, "rollback": true,
}

type Finding struct {
    Code     string `json:"code"`
    Severity string `json:"severity"`
    Summary  string `json:"summary"`
}

type Classification struct {
    SchemaVersion string    `json:"schema_version"`
    Severity      string    `json:"severity"`
    Tags          []string  `json:"tags"`
    Findings      []Finding `json:"findings"`
}

func DecodeClassification(raw []byte) (Classification, error) {
    var out Classification
    if err := json.Unmarshal(raw, &out); err != nil {
        return out, fmt.Errorf("invalid_json: %w", err)
    }
    if out.SchemaVersion != "change-review.v1" {
        return out, errors.New("unsupported_schema_version")
    }
    if out.Severity != "low" && out.Severity != "medium" && out.Severity != "high" {
        return out, errors.New("invalid_severity")
    }
    if len(out.Tags) == 0 || len(out.Tags) > 4 {
        return out, errors.New("invalid_tag_count")
    }
    for _, tag := range out.Tags {
        if !allowedTags[tag] {
            return out, fmt.Errorf("unknown_tag: %s", tag)
        }
    }
    return out, nil
}

The JSON Schema supplied with the request should express the same closed vocabulary, required properties, array limits, and additionalProperties: false. Keep the Go checks too. Schema-constrained generation reduces the output space; application validation protects the state transition. These are separate controls, and the latter is where you can reject an obsolete change-review.v1 result after the taxonomy has moved on.

If the Node.js edge process must wait for a result, Server-Sent Events can carry one-way progress notifications over an EventSource connection. MDN documents the event stream format and named events. Don't make that browser connection the durable queue, though. A disconnected tab is not evidence that a review should be canceled, and reconnect behavior is not a substitute for a work ledger.

The incident invariant is ownership before execution

The failure pattern is easy to miss during a demo. A scheduler claims a row, the worker calls the model, and then the process loses its lease before committing. Another worker claims the same row. Both responses may be valid. If each worker appends usage independently, one logical review produces two tenant charges and perhaps two sets of findings. If neither worker has a durable claim because the enqueue transaction failed, the review disappears instead. The two symptoms look opposite, but the missing invariant is the same: there is no atomic record connecting logical work, execution attempt, publication, and cost ownership.

I initially treated “exactly once” as a useful scheduler goal. In production queues, that phrase hides too much. The actionable target is at-least-once execution with idempotent effects. A retry may perform another model call, but only one accepted result can advance the work item, and every attempt must remain attributable to the original tenant for reconciliation.

Use three identities:

Identity	Stable across retry?	Purpose
work_id	Yes	One requested review and its publish guard
attempt_id	No	One lease, model call, and usage observation
tenant_id	Yes	Authorization, budget policy, and cost attribution

That split matters. Reusing work_id as an attempt key erases retry cost; generating a fresh work ID on every retry defeats idempotency. The tenant belongs on the work record before enqueue, then gets copied onto each attempt. It should never be inferred from ticket text, model output, a queue name, or a process-level API credential.

The commit path needs a compare-and-set on the work state. Only the worker holding the current lease may move running to succeeded, and inserting the accepted result should share that transaction. Losing the comparison is ordinary duplicate delivery, not an exceptional publication path. Record the attempt and stop.

Duplicates happen.

package review

import (
    "context"
    "errors"
    "time"
)

var ErrLeaseLost = errors.New("lease_lost")

type Work struct {
    ID              string
    TenantID        string
    ContentDigest   string
    TaxonomyVersion string
    LeaseToken      string
}

type Usage struct {
    InputUnits  int64
    OutputUnits int64
}

type Store interface {
    RecordAttempt(context.Context, string, string, string, Usage) error
    CommitIfLeased(context.Context, string, string, Classification) (bool, error)
}

type Model interface {
    Classify(context.Context, Work) ([]byte, Usage, error)
}

func RunAttempt(ctx context.Context, store Store, model Model, work Work, attemptID string) error {
    raw, usage, callErr := model.Classify(ctx, work)
    if err := store.RecordAttempt(ctx, work.TenantID, work.ID, attemptID, usage); err != nil {
        return err
    }
    if callErr != nil {
        return callErr
    }

    result, err := DecodeClassification(raw)
    if err != nil {
        return err
    }
    committed, err := store.CommitIfLeased(ctx, work.ID, work.LeaseToken, result)
    if err != nil {
        return err
    }
    if !committed {
        return ErrLeaseLost
    }
    return nil
}

func LeaseDuration() time.Duration { return 2 * time.Minute }

The 2 * time.Minute value is illustrative configuration, not a universal recommendation. Set a lease from observed end-to-end latency plus a margin, renew it while useful work continues, and cap retries under an explicit policy. I'm not sure what the right duration is for your queue without its latency distribution; p95 and p99 attempt duration, scheduler delay, and cancellation behavior would settle it.

One subtle point: record usage even when validation fails or the lease is lost. The provider performed work, so hiding that attempt makes the per-tenant report look cleaner and less true. Publication deduplicates business effects. Accounting preserves every execution attempt.

Preventative controls belong on both sides of the model call

Before execution, verify the tenant is active, the work digest still matches the submitted change, the taxonomy version is accepted, and the tenant's policy allows another attempt. Reserve budget against the work item if budget enforcement must be strict. A read-then-call sequence is racy — two workers can both see room — so reservation needs the same transactional discipline as claiming work.

After execution, validate syntax, schema, vocabulary, and policy in that order. Store the raw response in access-controlled evidence storage only if retention policy permits it; change summaries and findings can contain sensitive implementation detail. Publish structured fields from the validated object, not by parsing a rendered explanation. Then emit queue latency, attempt latency, validation outcome, retry reason, and usage with tenant_id represented in a controlled-cost dimension or joined through an internal ledger. Putting unconstrained tenant identifiers on every metrics series can create a cardinality problem, so traces or ledger tables are often a better home for exact attribution.

The runbook should distinguish four outcomes. A valid accepted response completes the work. A valid response from a stale lease is recorded but not published. An invalid response is recorded as a contract failure and may be retried under policy. A canceled or expired work item is not revived by a late response. This classification keeps alerts tied to action: page on sustained missed schedules or exhausted review deadlines, ticket repeated contract failures, and graph duplicate attempts without treating every duplicate as an outage.

Late means stale.

Test the ugly path. Run two workers against the same work_id; force the first lease to expire after its model call; confirm that exactly one result is visible, both attempts remain in the usage ledger, and both point to the same tenant. Also test a tag outside the enum, an extra JSON property, a stale taxonomy version, and a change digest updated while queued. These tests catch more operational risk than another happy-path prompt example.

Compare architectures by failure containment

A direct model integration has the fewest moving pieces and may fit one team with one account and a modest queue. A self-hosted gateway can centralize routing and credentials; LiteLLM is one open-source example of that architectural category. A managed gateway can reduce ownership of gateway operations. A fully self-hosted inference stack gives the most control over data placement and capacity, while transferring model serving, upgrades, and saturation response to your team.

None of those choices removes the need for work identity, schema validation, or a tenant ledger. Compare them on where retries happen, whether usage metadata is complete, how tenant attribution survives provider changes, how credentials are isolated, and which layer owns timeout and backoff policy. If both the worker and gateway retry, document the multiplication explicitly: an application attempt may contain multiple upstream attempts, and the ledger must make that relationship visible.

The catch is operational ownership. A central gateway is not suitable when its shared failure domain or data path violates the service's isolation requirements; use isolated direct integrations or separate gateways then. Direct calls are a poor fit when many teams need consistent policy and nobody can audit scattered credentials. Self-hosting is the wrong default when the team cannot staff capacity planning and incident response. Stick with the smallest boundary whose failure modes your on-call rotation can actually diagnose.

Per-tenant visibility is the decision rule, not a dashboard decoration. Reject an architecture if it can report only a shared monthly total, drops usage for failed calls, or cannot link an upstream attempt back to tenant_id, work_id, and attempt_id. Your mileage may vary on the transport and storage engine. Those four records — immutable work, renewable lease, append-only attempt, guarded result — are the part I wouldn't trade away.

Where this scheduling pattern does not fit

Don't put a scheduler in the synchronous path merely to claim architectural consistency. For an agent assisting a person in an active chat, added queue delay may be worse than the duplicate risk, and a request-scoped idempotency key plus a short-lived usage record can be enough. The scheduled pattern is also unsuitable for code changes that require immediate deterministic policy enforcement; use a conventional rules engine for mandatory checks and reserve model findings for advisory review.

Human approval still matters for high-impact fintech changes. Structured output makes review evidence easier to route and audit; it does not prove a finding is correct. If a tag can freeze payments, change authorization, or block a release, keep that action behind deterministic policy or an accountable reviewer.

References

• MDN, “Using server-sent events”: https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events
• LiteLLM open-source repository: https://github.com/BerriAI/litellm

Further reading

Start with the MDN event-stream documentation for browser delivery behavior, then inspect the LiteLLM repository only if a gateway fits the ownership and failure-containment criteria above:

• https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events
• https://github.com/BerriAI/litellm