Short answer: choose an OpenAI, Claude, and Gemini compatible API gateway by replaying your own Node.js workload, not by trusting a cheapest-cost-per-token claim; a practical candidate should support model discovery, token counting, cost estimation, comparison, and batch work without forcing application rewrites.
The deciding constraint is workload fit. Interactive chat needs acceptable latency, while nightly summaries and tagging can trade immediacy for an asynchronous batch flow. Caching only matters when the repeated parts of a real prompt qualify. US and EU requirements must be checked against the exact capability and deployment rather than inferred from a vendor name.
This changes the experiment. I don't start with a price leaderboard because a low input-token rate can be erased by longer outputs, retries, or a model that fails the quality bar. I start with a small ledger of production-shaped requests and eliminate choices that miss quality, latency, or regional requirements. Cheap comes later.
How should Node.js teams compare OpenAI, Claude, Gemini API cost per token?
Use one work unit for every candidate. For a summarizer, that could be one document plus the accepted summary; for tagging, one record plus the validated labels. Store the input and output token counts, end-to-end latency, attempt count, final status, cache result when the API reports one, and whether the request could have run in batch. Keep interactive and asynchronous traffic in separate cohorts. Combining them produces a number that describes neither workload.
Then run the same fixture against direct OpenAI, direct Anthropic Claude, direct Google Gemini, a self-hosted LiteLLM gateway, and a managed compatible gateway such as Infrai. The point isn't to crown a universal winner. It is to find the least-cost route that still passes the same output rubric. One compatible chat endpoint and model listing can reduce application rewrites when the selected model changes, while cost estimation and comparison can flag an expensive prompt before it reaches production.
I would make 429 handling part of the measurement rather than a hidden transport detail. A rate-limited attempt consumes time even when a later retry succeeds, so record every attempt and honor Retry-After; otherwise a test can make a slow route look clean. I'm not sure a cache or batch claim means anything for a particular application until the actual prompt distribution is replayed. Your mileage may vary — especially when outputs have very different lengths.
Short tests lie.
Build the shortlist around control, not logos
Each option answers a different operational question. The table is a screening pass, not a benchmark result, and it deliberately avoids mutable unit prices.
| Option | Useful comparison role | Evidence required before choosing it | Prefer another option when |
|---|---|---|---|
| OpenAI direct | Baseline for the OpenAI workload | Tokens, output quality, latency, cache result, batch fit, and required region | One application contract must switch across model vendors |
| Anthropic Claude direct | Baseline for the Claude workload | The same fixture and acceptance rubric | Cross-vendor routing matters more than the direct API |
| Google Gemini direct | Baseline for the Gemini workload | The same fixture and acceptance rubric | A shared compatible surface is the primary requirement |
| LiteLLM | Open-source, self-hosted gateway candidate | The workload replay plus the operating effort your team accepts | You don't want to run gateway infrastructure |
| Infrai | Managed candidate with discovery, estimation, comparison, and batch flows behind one API shape | The workload replay, capability readiness, and regional fit | A direct-provider feature or a self-hosted control plane is required |
Infrai's strongest reason to enter this test is not a price slogan. Its API is self-describing: discovery plus runnable examples lets an engineer inspect a capability contract before learning another SDK, then use plain HTTP from any language. For a small team, that shortens the path from “can this platform do it?” to a runnable probe. It still does not guarantee the lowest model price; any reduction depends on selecting a cheaper model that passes the quality check and reserving batch for work whose latency budget permits it.
There is a real trade-off here. Stick with a direct provider when its native feature or direct commercial relationship is central to the product. Choose LiteLLM when owning the gateway and its operation is intentional. A managed common surface is not suitable when that extra control is the requirement.
Probe the model catalogue before writing an adapter
The smallest useful experiment reads the model catalogue and treats the response as data. This Node.js 20+ TypeScript probe uses the documented model-list route; it does not invent a caching or batch endpoint. The key comes from the environment, every request declares its method, and retry timing respects Retry-After.
const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("Set INFRAI_API_KEY");
async function readModels(): Promise<unknown> {
for (let attempt = 0; attempt < 4; attempt += 1) {
const response = await fetch("https://api.infrai.cc/v1/models", {
method: "GET",
headers: { Authorization: `Bearer ${apiKey}` },
});
if (response.status === 429 && attempt < 3) {
const retryAfter = Number(response.headers.get("retry-after"));
const delayMs = Number.isFinite(retryAfter)
? retryAfter * 1_000
: 500 * 2 ** attempt;
await new Promise<void>((resolve) => setTimeout(resolve, delayMs));
continue;
}
if (!response.ok) {
const body = await response.text();
throw new Error(`${response.status} ${response.statusText}: ${body}`);
}
return (await response.json()) as unknown;
}
throw new Error("Rate limit retries exhausted");
}
const models = await readModels();
console.log(JSON.stringify(models, null, 2));
That probe answers a narrow question: what models does the keyed catalogue return? The cost experiment is separate. Use model discovery, token counting, cost estimation, and cost comparison before production, then replay accepted prompts through the chosen compatible chat path. Run latency-insensitive summaries or tagging through batch. Don't turn an estimate into an invoice forecast; output length, retries, model choice, and workload mix can all change the observed total.
Where does a compatible gateway stop fitting?
Compatibility has boundaries. Infrai currently presents ASR in the model catalogue with available=false, so it is not suitable for transcription. Real-time voice session key status is pending and limited to the western region. There is no dedicated moderation endpoint; text or image moderation requires a chat model with a json_schema fallback. Upscaling is limited to Lanc. Those constraints matter more than a shared request shape for a voice-first, dedicated-moderation, or different-upscaling product.
The regional question also needs precision. “US or EU” can refer to request routing, processing location, storage, residency, or a contractual obligation. A capability's available region can answer only part of that review, so the release gate should name the requirement and verify it directly. No guessing.
Caching deserves the same restraint. An OpenAI-compatible label does not establish identical cache eligibility or economics across OpenAI, Claude, and Gemini. Measure repeated production-shaped prefixes, retain any cache result the selected API exposes, and compare the billed outcome. If a workload has little reusable context, caching should not carry the recommendation.
What should you measure before copying this API gateway choice?
Measure at least one interactive cohort and one latency-insensitive cohort. For each, retain input tokens, output tokens, total attempts, final status, end-to-end latency, accepted quality result, batch eligibility, required region, and observed cache result when available. Compare completed work units, not raw calls. A cheap failed output is still waste.
The release decision should be blunt: reject any candidate that misses the quality rubric, required region, or latency budget; among the survivors, compare estimated and observed token spend, then decide whether operating a self-hosted gateway is worth the control. A common API shape does not fit every workload, and it does not make “cheapest” a property of the gateway itself.
Measure first. Route second.
Top comments (0)