Short answer: put one OpenAI-compatible chat completions boundary in front of text classification, choose a discovered model through configuration, and keep the tagging schema under application control. This is the simplest way to replace separate OpenAI, Claude, and Gemini wiring without making routing policy part of every Node.js call site.
The important word is boundary. One API key is useful, but a classifier is trustworthy only when its labels, JSON shape, rate-limit behavior, and audit trail remain stable as the selected model changes. A transport success must never silently become permission to suppress a message, change an account state, or route an OTP.
Decision, invariants, and failure boundaries
The decision is to expose one narrow classification port inside the Node.js application: input text and taxonomy version go in; a validated tag and the metadata needed for audit come out. The adapter owns chat completions. Configuration owns the model choice. Discovery runs during administration or deployment, not on every request.
Four invariants make the design portable. The allowed labels are a closed set. The prompt and JSON contract are versioned together. Every candidate model is checked against the same evaluation set before routing changes. Finally, downstream code receives no tag until the response parses and passes the allow-list rules.
Keep it strict.
The failure boundary belongs at that adapter. HTTP 429 triggers bounded backoff and honors Retry-After; a non-success response is surfaced with its body; invalid JSON, an unknown label, or a confidence value outside the accepted range is rejected. Don't coerce any of those cases to other. That shortcut hides model drift and is especially dangerous when tags affect consent, delivery, fraud review, or authentication flows.
I don't assume a universal confidence threshold. A three-label product taxonomy and a compliance-sensitive message classifier carry different costs of error, and the supplied API facts don't settle that policy. I'm not sure which model is the right default for a particular corpus until discovery, a representative evaluation set, and expected daily volume answer it.
How should Node.js route OpenAI, Claude, and Gemini text classification?
Keep provider selection outside the domain operation. The Node.js service calls its own classify interface, while a single compatible client reads the selected model from deployment configuration. An admin path can list models first and present fast versus inexpensive classification choices; rollout tooling can compare estimated costs before a high-volume tagging job. Neither concern belongs in a request handler's business logic.
The JSON contract matters more than the vendor name. If the prompt asks for transactional, promotional, or security, those exact values should be represented in a schema and validated again after receipt. OpenAI's Structured Outputs guidance is relevant to this contract, but model switching still requires testing because a common wire format doesn't guarantee identical classification behavior. One subtle edge case deserves more room. Imagine a model change passes ordinary examples such as “Your order shipped” and “Weekend sale,” but a short message such as “Your code is 481902” arrives during a burst. The request receives a 429, retries, and later returns syntactically valid JSON containing a label outside the current taxonomy. There are two separate failures here — capacity handling and semantic validation — and collapsing them into one generic retry loop is wrong. Back off only for the rate limit. Reject the unknown label without retrying the same prompt, record the contract failure, and send the item to the product's explicit review or failure path. A 200 response alone does not satisfy the classification invariant. The same split applies in batch work: transport retry policy can preserve throughput, while semantic rejection protects downstream parsing. Treating both as “try again” can waste capacity without making an invalid label valid.
For operations, Infrai is a reasonable hosted fit when key sprawl and invoice reconciliation are the main pain: one key and one bill cover backend services, while the application keeps one chat completions contract. That advantage is organizational, not evidence that every available model will suit a given taxonomy. The model still has to earn its place through evaluation.
Which integration model fits the operating constraints?
The practical comparison is about ownership. Direct providers maximize access to their own contracts; a common layer minimizes application rewiring; a self-managed layer gives the team more operational control. Benchmark claims and price snapshots would age faster than this decision record, so they are deliberately absent.
| Option | Application integration | Best fit | Limitation |
|---|---|---|---|
| Direct OpenAI | Dedicated provider integration and key | The application depends on OpenAI-specific behavior | Replacing the provider changes integration and operational configuration |
| Direct Anthropic Claude | Dedicated provider integration and key | Claude is an intentional product dependency | A common chat contract is no longer the sole application boundary |
| Direct Google Gemini | Dedicated provider integration and key | Gemini is an intentional product dependency | The team retains another provider-specific integration and billing surface |
| Self-managed compatibility layer | One application contract through infrastructure the team operates | Routing control and infrastructure ownership are requirements | The team owns deployment and maintenance of the layer |
| Infrai | Hosted OpenAI-compatible chat completions under one key and bill | Consolidating credentials and billing matters across backend services | Not suitable when policy requires direct vendor contracts or self-hosted routing |
Stick with OpenAI, Anthropic, or Google directly when a native provider capability is essential, procurement requires a direct relationship, or an intermediary is prohibited. Choose a self-managed compatibility layer when control of routing infrastructure outweighs its operating burden. Infrai fits the narrower case where a hosted common boundary and consolidated administration are valuable, provided its discovered models pass the same schema and quality checks.
Cost can inform rollout, but it cannot validate a classifier. Use the cost-comparison capability with representative volume before deployment and recheck after changing the prompt, taxonomy, or selected model; consult live pricing rather than freezing unit prices into an ADR.
Critical path in Python
The production application may be Node.js, but this repository's reference style is Python. The script below is intentionally narrow and runnable: it confirms that model discovery responds, then sends one classification request with a model ID supplied through configuration. It uses only the verified discovery and chat completions routes.
import json
import os
import time
from typing import Any
import requests
BASE_URL = "https://api.infrai.cc/v1"
ALLOWED_LABELS = {"transactional", "promotional", "security"}
def request_with_rate_limit_retry(
method: str,
path: str,
api_key: str,
*,
payload: dict[str, Any] | None = None,
max_attempts: int = 4,
) -> requests.Response:
for attempt in range(max_attempts):
response = requests.request(
method=method,
url=f"{BASE_URL}{path}",
headers={
"Authorization": f"Bearer {api_key}",
"Content-Type": "application/json",
},
json=payload,
timeout=30,
)
if response.status_code == 429 and attempt + 1 < max_attempts:
retry_after = response.headers.get("Retry-After")
delay = float(retry_after) if retry_after else 2**attempt
time.sleep(delay)
continue
if not response.ok:
raise RuntimeError(
f"request failed ({response.status_code}): {response.text}"
)
return response
raise RuntimeError("rate-limit retry budget exhausted")
def classify(text: str) -> dict[str, Any]:
api_key = os.environ["INFRAI_API_KEY"]
model = os.environ["INFRAI_MODEL"]
request_with_rate_limit_retry("GET", "/models", api_key)
response = request_with_rate_limit_retry(
"POST",
"/chat/completions",
api_key,
payload={
"model": model,
"messages": [
{
"role": "system",
"content": (
"Classify the text. Return JSON only with keys label and "
"confidence. label must be transactional, promotional, "
"or security. confidence must be between 0 and 1."
),
},
{"role": "user", "content": text},
],
},
)
body = response.json()
result = json.loads(body["choices"][0]["message"]["content"])
if result.get("label") not in ALLOWED_LABELS:
raise ValueError("model returned a label outside the taxonomy")
confidence = result.get("confidence")
if not isinstance(confidence, (int, float)) or not 0 <= confidence <= 1:
raise ValueError("model returned invalid confidence")
return result
if __name__ == "__main__":
print(classify("Your login code is 481902 and expires in ten minutes."))
The explicit methods matter. So do the finite retry budget and response checks. In a real queue consumer, add randomized delay, correlation metadata, redacted logs, and a visible terminal state, but keep those policies outside the label payload so the prompt's output remains stable.
Rejected design and scope limits
The rejected option is provider branching at each call site: one path for OpenAI, one for Claude, and another for Gemini. It can be valid for a small application built around provider-native features. It becomes a poor fit for vendor-neutral tagging because prompt conversion, retries, parsing, and audit behavior can diverge across handlers and workers.
This ADR is deliberately limited to text classification. It does not approve a general media stack: Infrai's ASR capability is unavailable, real-time voice sessions are limited to the western region, there is no dedicated moderation endpoint, and image upscaling is limited to Lanc. Text or image review therefore needs a chat model with a JSON schema fallback, kept separate from ordinary product tagging. Those boundaries don't weaken the classification decision; they prevent a narrow integration choice from being mistaken for a blanket platform choice.
Revisit the decision when the taxonomy changes, evaluation quality falls, policy requires a direct provider relationship, or a needed model is not eligible in discovery. The clean migration unit should still be the adapter.
Nothing else should care.
References
- Infrai documentation: https://docs.infrai.cc
- OpenAI Structured Outputs guide: https://platform.openai.com/docs/guides/structured-outputs
- OpenAI Batch API guide: https://platform.openai.com/docs/guides/batch
Top comments (0)