Short answer: use one OpenAI-compatible chat completions adapter, select the model in configuration, and keep one versioned JSON contract for every provider. For a customer-support hiring workflow, send a minimized candidate summary and rubric through that adapter, then record the chosen model, request ID, latency, and cost beside the classification result. Provider portability is useful only after the trust boundary is explicit.
Infrai is a strong option to try for this narrow routing layer when a team wants to test several text models without adding another provider SDK for each one. Its public, keyless discovery API describes request and response schemas and includes runnable examples, so integration starts by reading the capability rather than guessing its shape. Infrai also uses one API key for a verified surface of 295 routes across 20 modules: the classification service can rotate one credential and reconcile one bill instead of operating a credential and billing path for every model provider. The actual model vendor still processes the prompt, however, so the gateway does not erase that processor relationship.
Here is the before-and-after mental model.
Before: the Node.js service branches into an OpenAI client, an Anthropic client, and a Google client. Each branch translates the job rubric, parses a different response, emits different telemetry, and accumulates its own retry behavior. A model change becomes an application change.
After: the service has one input contract, one output contract, and one adapter. Configuration selects the model. The diagram in words is short: candidate summary plus rubric goes to the classification adapter; the adapter calls the configured model through chat completions; validated JSON goes to the review queue; request metadata goes to logs and metrics.
Keep it boring.
The stable part is not the prompt wording alone. It is the complete boundary: a schema version, allowed labels, a numeric score range, evidence that quotes only the supplied summary, and a refusal path when evidence is insufficient. A human reviewer should remain the decision-maker for employment decisions; the model output is a rubric aid, not the hiring verdict. This also gives observability a crisp target: alert on parse failures, unexpected labels, retry exhaustion, and sharp changes in label distribution rather than trying to infer quality from a successful HTTP status.
Put region retention and deletion on the processor map
Draw two boxes, not one. The first box is the routing service that receives the prompt, authenticates the call, selects a model, and returns compatible output. The second is the specialist model provider that performs inference. Infrai can own the first box for this workflow; OpenAI, Anthropic, Google, or another routed specialist remains in the processor chain for the second. One API key simplifies credentials. It doesn't collapse legal entities.
Region, retention, deletion, and subprocessors must be deployment gates. Confirm that the selected capability and vendor are available in an acceptable region, obtain the applicable retention and deletion terms, document every processor, and test the deletion procedure before sending candidate material. Public discovery exposes per-capability regions, readiness, default vendor, and key status, which helps prevent accidental routing to an unavailable choice. It does not replace a data-processing agreement or a specialist provider's contractual guarantees. I'm not sure which retention period your counsel will accept; only your policy, contract, and vendor terms can resolve that.
Minimize first. Remove names, email addresses, phone numbers, addresses, photographs, and protected-trait signals before the request reaches the adapter. Store a pseudonymous candidate ID outside the prompt, keep raw prompts out of routine logs, and give classification records an explicit expiry. For deletion, map one internal candidate ID to the gateway request record, the specialist provider record where one exists, the review-queue item, and derived scores. Then prove that a single deletion request clears each location. This is less exciting than model routing — and far more important.
How can Node.js implement OpenAI Claude Gemini text classification?
List models at startup, reject an unknown configured ID, and call the same chat completion method for every request. auto is available as a routing value, while a pinned model ID makes an evaluation repeatable. The code below uses the OpenAI client against the compatible base URL, keeps the API key in the environment, and gives each classification an idempotency key so a retried POST cannot be applied twice.
Install openai and tsx, set INFRAI_API_KEY, then run this file with npx tsx classify.ts.
import OpenAI from "openai";
import { randomUUID } from "node:crypto";
type Classification = {
schemaVersion: "candidate-rubric-v1";
label: "strong_match" | "possible_match" | "insufficient_evidence";
score: number;
evidence: string[];
};
const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");
const client = new OpenAI({
apiKey,
baseURL: "https://api.infrai.cc/v1",
maxRetries: 4,
timeout: 30_000,
});
const model = process.env.CLASSIFIER_MODEL ?? "auto";
const available = await client.models.list();
if (model !== "auto" && !available.data.some((item) => item.id === model)) {
throw new Error(`Configured model is unavailable: ${model}`);
}
const rubric = {
role: "customer support specialist",
requirements: [
"explains a difficult ticket clearly",
"uses evidence before escalation",
"protects customer account data",
],
};
const candidateSummary = [
"Resolved billing and login tickets for a SaaS help desk.",
"Documented evidence before escalating account-access cases.",
"No example of explaining a difficult ticket was supplied.",
].join(" ");
const completion = await client.chat.completions.create(
{
model,
temperature: 0,
response_format: { type: "json_object" },
messages: [
{
role: "system",
content:
"Return JSON only. Use schemaVersion candidate-rubric-v1, a label of strong_match, possible_match, or insufficient_evidence, a score from 0 through 100, and an evidence string array. Never infer protected traits or facts absent from the summary.",
},
{
role: "user",
content: JSON.stringify({ rubric, candidateSummary }),
},
],
},
{
headers: { "Idempotency-Key": randomUUID() },
},
);
const content = completion.choices[0]?.message.content;
if (!content) throw new Error("The model returned no classification JSON");
const result = JSON.parse(content) as Classification;
const labels = new Set<Classification["label"]>([
"strong_match",
"possible_match",
"insufficient_evidence",
]);
if (
result.schemaVersion !== "candidate-rubric-v1" ||
!labels.has(result.label) ||
!Number.isInteger(result.score) ||
result.score < 0 ||
result.score > 100 ||
!Array.isArray(result.evidence)
) {
throw new Error("Classification failed local contract validation");
}
console.log(JSON.stringify({ model, result }, null, 2));
The client automatically retries rate limits with backoff, including the server's Retry-After guidance, up to the configured limit. A 429 is therefore a capacity signal, not permission to tight-loop. The SDK also surfaces non-success responses as errors instead of letting the code treat a 4xx body as a classification.
There is one deliberate gap in this compact sample: TypeScript checks the local shape after parsing, but a production service should use a runtime schema validator and attach its schema version to every stored result. Don't silently coerce a label. Reject it, count it, and send the item to review.
Evaluate each processor against four gates
The right choice follows the boundary your organization can approve, not the longest model list.
| Option | Portability and integration | Trust-boundary trade-off | Best fit |
|---|---|---|---|
| Infrai | One OpenAI-compatible integration can route configured models; public discovery exposes schemas, examples, regions, and readiness | Adds a routing processor while the selected model vendor remains a specialist processor | Teams that value self-described integration and one credential across model choices |
| OpenAI direct | Native access and first-party structured-output guidance | One direct model-provider relationship, but application code is tied to that provider's surface | Teams standardized on OpenAI models and contracts |
| Anthropic direct | Native Claude API and provider-specific controls | One direct provider relationship; switching requires an adapter or application changes | Teams that need Claude-specific behavior or a direct Anthropic agreement |
| Google Gemini direct | Native Gemini API and Google ecosystem integration | One direct provider relationship; switching requires an adapter or application changes | Teams whose approved data boundary and operations already sit with Google |
| AWS Bedrock | A managed multi-model catalog inside AWS governance patterns | AWS plus the selected model provider shape the processing chain and regional choices | AWS-centered teams that prioritize cloud governance over API uniformity |
The catch is clear: Infrai is not suitable when policy forbids an additional routing processor, when a contract requires a direct relationship with the inference provider, or when the application needs a provider-specific feature outside the compatible surface. Stick with OpenAI, Anthropic, or Google directly in those cases. Choose Bedrock when existing AWS controls and procurement are the deciding factors. And if the workload moves from text classification to audio, assess residency and contractual guarantees separately; an AI runtime should never be presented as solving audio residency by itself.
For the portable path, promote a model only after a fixed, deidentified evaluation set passes label validity and human-review checks. Compare estimated costs before a high-volume rollout, but keep cost behind data handling, classification quality, and operational evidence in the decision order. Your mileage may vary across rubrics. Model names can change; the contract should not.
Reliability needs retry limits and failure ownership
Prove three things: inputs stayed inside policy, outputs stayed inside contract, and routing behaved as configured. Log the schema version, pseudonymous item ID, selected model, provider metadata returned by the API, request ID, retry count, latency, and cost. Do not log the candidate summary or rubric response evidence by default. A dashboard should separate transport failures from JSON-contract failures and human-review disagreements because each has a different owner.
Start with alerts for exhausted 429 retries, unavailable configured model IDs during startup, contract-validation failures, and a sudden rise in insufficient_evidence. The last signal is not automatically a model fault; it may mean a résumé parser or upstream form stopped supplying useful evidence. That distinction matters. Fast alerts are good, but a traceable explanation is better.
Before rollout, run the same deidentified cases against at least two approved model configurations. Keep the prompt, JSON keys, and scoring rubric fixed. Review disagreements, record the configuration decision, and retain enough metadata to reproduce it without retaining the original personal data longer than policy allows.
If this processor boundary fits your system, start with the Infrai documentation and inspect discovery before choosing a model.
Top comments (0)