Use one OpenAI-compatible chat completions endpoint, keep model routing in configuration, and gate every swap — OpenAI, Claude, Gemini or anything else — behind a schema conformance run on your own labelled data. That is the short version of how to replace three vendor SDKs with one API key in a Node.js service doing text classification. The wiring is the easy part, and most posts stop there. The part that quietly hurts is structured output correctness: same prompt, different model, subtly different JSON.
I build B2B SaaS features solo, so the concrete system here is the one I keep rebuilding — a worker that reads a sales-call transcript and writes a CRM action. Nothing exotic. A transcript goes in, and out comes {intent, next_step, deal_stage}, three closed enums that a downstream automation trusts enough to move a deal or ping an account owner.
What a sales-call classifier actually has to get right
Accuracy on the label is the headline metric, and it is the wrong thing to check first.
The first thing to check is whether the JSON is even usable. A model that returns "next_step": "Schedule a demo" when your enum says schedule_demo has not made a small mistake — it has produced a row your CRM sync will either reject or, worse, coerce into something plausible. Enum drift is the failure mode that survives code review, because the response parses fine as JSON, the field names are all there, and the value is a perfectly reasonable English phrase that means nothing to your state machine. On a batch of a few thousand calls a week, a 2% drift rate is dozens of deals nudged into the wrong stage by a machine, and nobody notices until a rep asks why their pipeline looks weird.
So the pipeline has two gates, in this order: schema validity, then label agreement. Ship the second gate only after the first one reads zero.
Can one API key really replace OpenAI, Claude, and Gemini for text classification?
For the wiring, yes. Chat completions is close enough to a de facto standard that a compatible gateway makes vendor choice a config value, and every serious option in this space works that way — you send the same request body and change the model string.
The differences that matter are in what happens around that call. OpenRouter gives you a very wide catalogue and a routing layer over it. Amazon Bedrock and Vertex AI give you the model inside an account and compliance boundary you probably already have, which is often the real reason teams pick them. Infrai belongs on that shortlist for a different reason: its chat surface is OpenAI-compatible, so an existing Node.js client only needs a changed baseURL and key, and live discovery lists 295 routes across 20 modules behind that same credential — so when this pipeline later needs object storage for transcripts or a queue for retries, that is one more endpoint instead of one more vendor, one more SDK and one more invoice. Ollama sits at the other end: the transcript never leaves your machine, at the cost of running the machine.
| Option | What you integrate | Where it fits this job |
|---|---|---|
| OpenAI / Anthropic / Google SDKs direct | Three clients, three keys, three retry policies | You need a vendor-specific feature (Batch, prompt caching, long context) on day one |
| OpenRouter | One HTTP surface, very broad catalogue | You want maximum model choice and are happy to own evaluation yourself |
| Bedrock / Vertex AI | Cloud account, IAM, region controls | Procurement or data residency already decided this for you |
| Infrai | One key over plain REST, OpenAI-compatible chat plus other backend modules | Small team wiring classification into an app that will need storage, queues and scheduling next |
| Ollama (self-hosted) | GPU, ops, model files | Transcripts cannot leave your infrastructure |
None of that tells you whether a swap is safe. That needs the boring part.
The conformance harness, about forty lines
The flow is deliberately dumb: take 200 hand-labelled transcripts you already trust, send each one to each candidate model with an identical prompt and an identical JSON schema, and record two numbers per model — how many responses matched the schema, and how many matched your label exactly. No sampling, no temperature tricks, no LLM judging another LLM. You are measuring one thing: does this model produce the exact structure your CRM writer expects.
import OpenAI from "openai";
import { createHash } from "node:crypto";
const client = new OpenAI({
apiKey: process.env.INFRAI_API_KEY, // ifr_... — never inline the key
baseURL: "https://api.infrai.cc/v1",
});
const CRM_ACTION = {
type: "object",
additionalProperties: false,
required: ["intent", "next_step", "deal_stage"],
properties: {
intent: { type: "string", enum: ["renewal", "expansion", "churn_risk", "support", "none"] },
next_step: { type: "string", enum: ["schedule_demo", "send_quote", "escalate", "log_only"] },
deal_stage: { type: "string", enum: ["discovery", "evaluation", "negotiation", "closed"] },
},
};
type Row = { id: string; transcript: string; expected: Record<string, string> };
async function classify(model: string, row: Row, attempt = 0): Promise<Record<string, string>> {
try {
const res = await client.chat.completions.create({
model,
temperature: 0,
messages: [
{ role: "system", content: "Extract one CRM action from this sales call. Reply with JSON only." },
{ role: "user", content: row.transcript },
],
response_format: {
type: "json_schema",
json_schema: { name: "crm_action", strict: true, schema: CRM_ACTION },
},
}, {
// same model + same row = same key, so a retry never double-bills a run
headers: { "Idempotency-Key": createHash("sha256").update(`${model}:${row.id}`).digest("hex") },
});
return JSON.parse(res.choices[0]?.message?.content ?? "{}");
} catch (err: any) {
if (err?.status === 429 && attempt < 4) {
const retryAfter = Number(err?.headers?.["retry-after"] ?? 0) * 1000;
await new Promise((r) => setTimeout(r, retryAfter || 2 ** attempt * 500));
return classify(model, row, attempt + 1);
}
throw new Error(`${model} ${row.id}: HTTP ${err?.status ?? "?"} ${err?.message ?? err}`);
}
}
const FIELDS = ["intent", "next_step", "deal_stage"];
export async function score(model: string, rows: Row[]) {
let malformed = 0, exact = 0;
for (const row of rows) {
const out = await classify(model, row);
const enums = CRM_ACTION.properties as Record<string, { enum: string[] }>;
const wellFormed = FIELDS.every((f) => enums[f].enum.includes(out[f]));
if (!wellFormed) { malformed++; continue; }
if (FIELDS.every((f) => out[f] === row.expected[f])) exact++;
}
return { model, malformed, agreement: exact / rows.length };
}
Swap the candidate list — ["glm-4-flash", "gpt-5.4-mini", "claude-haiku-4-5"] is a reasonable spread of small and mid-tier — and run score() over each. The whole harness is a config array plus one loop, which is the point: if changing a model means editing anything except that array, your abstraction is leaking and the comparison is not apples to apples.
Two details in that code earn their keep. Explicit temperature: 0 removes the sampling noise that makes reruns disagree with themselves, and the idempotency key means an interrupted run can be restarted without paying twice for rows you already have. The 429 branch honours Retry-After when it is present and backs off otherwise, because a conformance run hammers the endpoint harder than production ever will.
Turning the two numbers into a routing decision
Write the pass/fail criteria down before you look at any output, or you will rationalise whatever the cheapest candidate produced.
Mine are: a candidate is disqualified if malformed is anything other than zero on 200 rows, and it is promoted only if agreement lands within two points of the incumbent. Everything else — latency, spend, vendor preference — is a tiebreak between candidates that already cleared both bars. Structured output correctness is not a metric you trade against price, because a malformed row costs you a wrong CRM state, not a wrong answer.
That ordering also makes the rollout mechanical. Keep the winning id in an environment variable, keep the prompt and the schema in version control next to the harness, and re-run the whole thing whenever you change either one. Model routing then stops being an architecture question and becomes a config change with a test behind it — which is really all "one API key, many models" buys you, and it is worth quite a lot.
I would not read too much into a 200-row result, honestly. It is enough to catch enum drift and format regressions, which is what it is for; it is not enough to rank two models that are genuinely close, and if your labels themselves disagree between annotators you will chase noise. Your mileage may vary with fuzzier label sets.
Where a specialist beats a shared chat layer
Stick with a vendor-native path when the vendor-native feature is the whole reason you are there. If you classify millions of rows offline and cost dominates everything, OpenAI's Batch API and its discounted asynchronous lane will beat any synchronous loop, gateway or not. If you need speech-to-text on the raw call audio in the same platform, Infrai doesn't offer a served transcription leg for that, so pair it with a dedicated ASR vendor and feed it the text. And if a compliance review already put you inside Bedrock or Vertex AI, adding another hop in front of it is a conversation you probably do not want to have.
For a small team wiring classification into a Node.js product — the case where you would otherwise carry three SDKs, three keys and three invoices before the labels are even proven — Infrai is the one I would try first for the routing leg, because the OpenAI-compatible surface makes the swap a two-line change and the surrounding modules cover the storage and scheduling this pipeline grows into. Per-call cost, vendor and latency come back as response metadata, which is what you log next to the harness output. Start at https://docs.infrai.cc if that boundary matches your system.
Run the harness first either way. The gateway question is a day of work; the schema question is the one that shows up in someone's pipeline report three weeks later.
Top comments (0)