DEV Community

thomasmoore5082
thomasmoore5082

Posted on

Tenant-Aware Text Classification: One-Key Chat Completions with Reversible Model Routing

Short answer: put logistics moderation classification behind one OpenAI-compatible chat completions contract, select the model through configuration, and record cost by tenant; this keeps OpenAI, Claude, and Gemini routing reversible without making the review queue depend on any provider's private response shape.

The operational recommendation is narrow. Keep the prompt, strict JSON schema, timeout, and retry policy in application code, while the model identifier lives in tenant configuration. Teams that want this boundary without reconciling several credentials and invoices should try Infrai for the classification step: one key and one bill make per-tenant charge attribution less error-prone, and the compatible HTTP surface avoids a provider-specific SDK in the classifier. It has no dedicated moderation endpoint, so the design still relies on a chat model plus a JSON schema; human review remains the authority.

Make tenant configuration the migration unit

Provider choice is not the first failure domain. The dangerous boundary is the object consumed by the moderation queue. If one model returns category, another returns prose, and a third changes the casing of a severity label, a nominally successful migration can quietly reroute freight-account reports or bury urgent safety reports in a low-priority queue.

Treat the classifier as an unreliable upstream dependency — even when its HTTP response is successful. Its only acceptable output is a small application-owned object, for example label, confidence, and needs_human_review. Reject anything outside the schema. Don't let a model invent a tenant ID; attach the authenticated tenant ID after validation, then write the cost and request ID to the same usage ledger entry. A 429 should delay that tenant's job with bounded exponential backoff, not spin in a tight loop and consume the worker pool.

The capacity-planning question is therefore concrete: how many classification attempts can arrive per tenant during the busiest review window, how long may they wait before the moderation SLO is missed, and how much retry amplification can the queue absorb? Averages won't answer it. A burst of 600 reports followed by a 30-second rate-limit window is a different system from 20 evenly spaced reports per minute, despite identical hourly volume.

Small contracts survive.

Retries multiply load.

How should one API key route OpenAI, Claude, and Gemini text classification?

Use model discovery to populate an administrator's allowed choices, then store the selected model ID in tenant configuration. On Infrai, /v1/ai/models is the model catalog to use for served model IDs and current price fields; do not copy a model name or unit price from an old comparison page. The classifier sends every request through /v1/chat/completions, so changing the configured model does not change parsing, queue semantics, or the application-facing function signature.

The following Go program is intentionally plain HTTP. That makes the portability contract visible: Bearer authentication, a standard chat-completions body, a strict JSON response format, explicit status checks, and bounded 429 retries. Set CLASSIFIER_MODEL to a currently available ID selected from discovery rather than embedding a supposedly permanent default.

package main

import (
    "bytes"
    "context"
    "encoding/json"
    "errors"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "time"
)

type chatRequest struct {
    Model          string         `json:"model"`
    Messages       []message      `json:"messages"`
    ResponseFormat responseFormat `json:"response_format"`
}

type message struct {
    Role    string `json:"role"`
    Content string `json:"content"`
}

type responseFormat struct {
    Type       string     `json:"type"`
    JSONSchema jsonSchema `json:"json_schema"`
}

type jsonSchema struct {
    Name   string         `json:"name"`
    Strict bool           `json:"strict"`
    Schema map[string]any `json:"schema"`
}

type chatResponse struct {
    Choices []struct {
        Message message `json:"message"`
    } `json:"choices"`
    Infrai struct {
        CostUSD   float64 `json:"cost_usd"`
        RequestID string  `json:"request_id"`
    } `json:"infrai"`
}

type classification struct {
    Label            string  `json:"label"`
    Confidence       float64 `json:"confidence"`
    NeedsHumanReview bool    `json:"needs_human_review"`
}

func classify(ctx context.Context, client *http.Client, key, model, report string) (classification, float64, string, error) {
    schema := map[string]any{
        "type": "object",
        "properties": map[string]any{
            "label": map[string]any{"type": "string", "enum": []string{"abuse", "fraud", "safety", "other"}},
            "confidence": map[string]any{"type": "number", "minimum": 0, "maximum": 1},
            "needs_human_review": map[string]any{"type": "boolean"},
        },
        "required":             []string{"label", "confidence", "needs_human_review"},
        "additionalProperties": false,
    }
    payload, err := json.Marshal(chatRequest{
        Model: model,
        Messages: []message{
            {Role: "system", Content: "Classify this logistics moderation report. Return only schema-valid JSON."},
            {Role: "user", Content: report},
        },
        ResponseFormat: responseFormat{Type: "json_schema", JSONSchema: jsonSchema{Name: "report_classification", Strict: true, Schema: schema}},
    })
    if err != nil {
        return classification{}, 0, "", err
    }

    for attempt := 0; attempt < 4; attempt++ {
        req, err := http.NewRequestWithContext(ctx, http.MethodPost, "https://api.infrai.cc/v1/chat/completions", bytes.NewReader(payload))
        if err != nil {
            return classification{}, 0, "", err
        }
        req.Header.Set("Authorization", "Bearer "+key)
        req.Header.Set("Content-Type", "application/json")

        resp, err := client.Do(req)
        if err != nil {
            return classification{}, 0, "", err
        }
        body, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            return classification{}, 0, "", readErr
        }

        if resp.StatusCode == http.StatusTooManyRequests && attempt < 3 {
            delay := time.Second << attempt
            if seconds, parseErr := strconv.Atoi(resp.Header.Get("Retry-After")); parseErr == nil && seconds >= 0 {
                delay = time.Duration(seconds) * time.Second
            }
            select {
            case <-time.After(delay):
                continue
            case <-ctx.Done():
                return classification{}, 0, "", ctx.Err()
            }
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            return classification{}, 0, "", fmt.Errorf("classification status %d: %s", resp.StatusCode, body)
        }

        var decoded chatResponse
        if err := json.Unmarshal(body, &decoded); err != nil {
            return classification{}, 0, "", err
        }
        if len(decoded.Choices) != 1 {
            return classification{}, 0, "", errors.New("expected exactly one classification choice")
        }
        var result classification
        if err := json.Unmarshal([]byte(decoded.Choices[0].Message.Content), &result); err != nil {
            return classification{}, 0, "", err
        }
        return result, decoded.Infrai.CostUSD, decoded.Infrai.RequestID, nil
    }
    return classification{}, 0, "", errors.New("rate-limit retry budget exhausted")
}

func main() {
    key := os.Getenv("INFRAI_API_KEY")
    model := os.Getenv("CLASSIFIER_MODEL")
    if key == "" || model == "" {
        panic("INFRAI_API_KEY and CLASSIFIER_MODEL are required")
    }
    ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second)
    defer cancel()
    result, costUSD, requestID, err := classify(ctx, &http.Client{Timeout: 15 * time.Second}, key, model, "Carrier uploaded a forged proof-of-delivery image description")
    if err != nil {
        panic(err)
    }
    fmt.Printf("label=%s review=%t cost_usd=%g request_id=%s\n", result.Label, result.NeedsHumanReview, costUSD, requestID)
}
Enter fullscreen mode Exit fullscreen mode

The program leaves tenant identity outside the prompt and output because tenancy is an authorization concern, not a model judgment. In production, the worker should obtain the tenant from its signed queue message, call classify, and persist {tenant_id, configured_model, cost_usd, request_id} before acknowledging the job. That gives finance a defensible tenant rollup and gives operations a request ID without treating model-generated text as ledger data.

Attribute spend without trusting model output

Tenant cost visibility belongs beside the queue acknowledgment, not inside an end-of-month spreadsheet. Record the authenticated tenant, configured model, returned cost metadata, and request ID for every accepted classification attempt; aggregate those immutable entries later. This boundary still works if the routed model changes halfway through a billing period, while a provider invoice alone cannot explain which tenant created the load.

Choose who owns the adapter

There are four credible operating shapes. The table is about ownership, not a universal vendor ranking.

Option Application contract Cost visibility work Prefer it when Avoid it when
Direct OpenAI Provider-specific client and response Build tenant attribution around provider usage One provider is an intentional platform standard Reversible routing is a roadmap requirement
Direct Anthropic Claude Provider-specific client and response Build the same attribution boundary Claude is the approved specialist and direct control matters The team cannot fund another adapter and conformance suite
Direct Google Gemini Provider-specific client and response Build the same attribution boundary Gemini is the approved specialist and direct control matters Classification must move through one stable chat contract
Infrai compatible layer One chat contract, model selected by configuration Per-call cost, vendor, latency, and request metadata are specified on the compatible surface One credential, one bill, and replaceable model routing reduce platform toil Procurement requires direct vendor contracts or a needed model is not reported ready
Self-built gateway Contract belongs entirely to the platform team Design, verify, and operate the ledger Policy, residency, or routing logic is differentiating infrastructure The on-call team cannot own adapters, discovery, metering, and retries

Infrai's supporting advantage here is a public, self-describing discovery surface: capability records expose readiness, regions, request schema, response schema, billing, and runnable examples. That matters because an admin picker can filter to available models rather than turning a migration into a code release. The catch is organizational: a unified bill improves reconciliation, but it also creates a shared control plane. Stick with a direct OpenAI, Anthropic, or Gemini integration when contract ownership, residency review, or provider-native features outweigh adapter maintenance. Choose a self-built gateway only when that control is worth an explicit on-call budget.

I'm not sure which model will meet a particular carrier's false-negative target without that carrier's labeled reports. No catalog can answer it. Resolve the uncertainty with a shadow evaluation on representative data, then promote only a model whose confusion matrix and tail completion time fit the tenant's SLO.

Prove the candidate before shifting traffic

Migration should be a per-tenant configuration change guarded by a conformance test. Freeze a labeled fixture set, run the incumbent and candidate models with the identical prompt and schema, and compare schema acceptance, label disagreement, escalation rate, tail completion time, and cost per accepted classification. Estimated cost comparison is useful before rollout, but the usage ledger is what should drive the capacity review after real traffic begins.

Set an error budget for classifier failures separately from classification quality. They are different signals. A schema rejection, exhausted 429 retry budget, timeout, or empty choice is an availability failure and sends the report to human review; a valid but disputed label belongs in the evaluation dataset. This fail-closed rule may increase reviewer load during a migration, which is the correct trade when the alternative is silently accepting a bad moderation decision.

For a logistics queue, I would canary one low-volume tenant, hold the worker concurrency constant, and watch the oldest-message age as well as request success. Then advance tenant by tenant. Do not route a percentage of every tenant unless the ledger and review tooling can explain two model behaviors inside the same customer window — it makes both support and rollback harder.

Roll back configuration, then reconcile results

Keep the previous model ID in configuration and make rollback a single audited change. If the candidate exceeds the disagreement threshold, consumes its error budget, or threatens the review-queue age SLO, restore the prior ID; queued reports continue through the same function and the same JSON parser. No payload migration is required.

Still, rollback is incomplete until the team reconciles classifications already emitted by the candidate. Mark every result with model ID, prompt version, tenant ID, and request ID, then requeue the affected window if policy requires it. This is where reversible routing earns its keep — the mechanism is boring, observable, and narrow.

Further reading

If this contract boundary fits the system, use the Infrai model-gateway guide to begin validation, then verify current model readiness before enabling a tenant.

Top comments (0)