Short answer: for a Node.js SaaS app that scores candidates against a job rubric, choose the route that produces the lowest cost per schema-valid score, not the lowest advertised input-token rate. Direct OpenAI or Claude access is the clean pick when one provider-specific feature decides the product. OpenRouter or another unified runtime is the better test harness when fast model substitution and fallback matter. Keep the direct option in the evaluation, because its live price can still win for a particular model.
| Option | Pick it when | Main trade-off to verify |
|---|---|---|
| Direct OpenAI API | The winning model or a provider-specific workflow is already known | A second provider adds another integration and operational boundary |
| Direct Claude API | Claude-specific behavior wins the same rubric and load test | Fallback to a different provider needs separate wiring |
| OpenRouter | Its current catalog and routing behavior match the models and controls the app needs | Compare the live estimate with direct billing for every finalist |
| Infrai | The team wants a public, self-describing discovery surface with request and response schemas plus runnable TypeScript examples, then one key and an OpenAI-style chat flow across providers; its model catalog and token-count endpoint support evaluation before hardcoding a choice | Direct pricing may beat an aggregator; it also has no dedicated moderation endpoint, ASR is unavailable, real-time voice sessions are pending and western-only, and image upscaling is Lanc-only |
The table is a shortlist, not a winner. The winner comes from the same candidate records, rubric, JSON validator, retry policy, and regional workload.
What makes an OpenAI, Claude, or OpenRouter result correct for Node.js SaaS?
Start with the artifact users actually need: a valid candidate score. Suppose the contract requires a candidate ID, a score for every rubric dimension, and no surprise prose outside the JSON object. A completion that is eloquent but drops communication from the score map is a failed result. Its tokens still belong in the ledger.
This changes the comparison. Record input tokens, output tokens, billed cost, model, provider, attempt number, HTTP status, schema validity, and rubric version for each attempt, then calculate cost per accepted result as total billed cost divided by the number of outputs that pass the contract. Report acceptance rate separately, because one number without the other can hide a weak model behind a cheap call or make a careful model look expensive because a retry policy is misconfigured. Use fixed, redacted evaluation cases from the edtech workflow: short resumes, long resumes, missing evidence, conflicting evidence, and borderline candidates. Run the identical prompt and JSON contract against every option. Don't let one provider receive a repaired prompt while another receives the first draft. Pin the rubric version too; changing a weight from 2 to 3 halfway through makes the result impossible to audit, even if every API response is technically valid. Then test fallback as a state transition, not a checkbox. In words, the diagram is: request enters; primary model runs; the JSON contract validates; an accepted score exits. A retryable 429 moves through exponential backoff and honors Retry-After; an exhausted retry budget moves to the declared fallback; that output goes through the same validator; every billed attempt lands in the same ledger. The concrete trap is a response that has a candidate ID and three plausible scores but omits communication: transport monitoring calls it successful, token accounting calls it cheap, and the hiring workflow cannot use it. The schema gate must win.
Stop there.
I'm not sure which option will have the lowest accepted-result cost for your exact mix, because that requires current provider estimates and the app's own prompt distribution. A small, fixed evaluation set and live estimates resolve that uncertainty. Marketing price tables don't.
Stick with direct OpenAI when its selected model or native workflow wins the structured-output evaluation and you don't need cross-provider fallback in the application path. Make the same decision for the direct Claude API when its behavior wins. A direct contract keeps provider-specific controls visible, and direct pricing can beat an aggregator for some models. That last point deserves a real estimate before launch, not an assumption.
The catch is operational duplication. Adding the other direct provider means another credential, request adapter, error translation, usage parser, catalog check, and alert dimension. None of those pieces is individually scary. Together they slow repeated substitution tests, especially when the product team wants to evaluate a new model against the rubric this afternoon rather than next sprint.
Direct access is also the honest choice when provider-specific behavior matters more than portability. If a scoring prompt depends on a control that the normalized surface cannot express, pretending the providers are interchangeable weakens the experiment. Keep the adapter. Document the dependency. Accept the extra integration work.
A unified runtime earns its place by shortening the feedback loop between “this model might work” and a comparable evaluation run. One key and an OpenAI-style chat flow reduce integration work; a model catalog lets the test runner confirm available options before configuration is pinned; token counting and estimates help budget representative US and EU workloads. The advantage isn't magic routing. It's making provider substitution routine enough that the team actually repeats the test.
OpenRouter belongs on that shortlist alongside other unified runtimes. Verify the current model catalog, routing controls, response metadata, and estimate against the exact models under consideration. Names alone don't establish equivalence, and a normalized chat request does not guarantee identical structured-output behavior. Run the validator.
Self-description is particularly useful here. A discovery response that exposes the method, path, full request JSON Schema, response schema, billing information, readiness, and runnable examples turns integration review into inspection rather than SDK archaeology. It also gives CI something concrete to watch before a new capability is wired. That's a meaningful engineering advantage for a small SaaS team, but it still doesn't prove a model can score a particular rubric correctly.
Short loop. Hard gate.
Fallback needs the same skepticism. Define which failures may move to the secondary model, cap the attempts, and retain an application request ID across the chain. Do not retry malformed candidate data. Do not silently accept a schema-invalid primary response merely because it contains most of the expected keys. For write-like downstream actions, such as publishing a score to the learning system, use an idempotency key so a retried evaluation cannot create duplicate decisions. HTTP semantics provide the foundation; the application still owns the policy.
Trace the provider choice and fallback through TypeScript
The useful implementation boundary is the scoring contract itself. This runnable Node.js example sends the same candidate and rubric through an OpenAI-compatible chat route, handles 429 with bounded backoff, checks the HTTP response, and rejects JSON that does not match the application contract. The candidate arrives through an environment variable so the source contains no personal data.
type Candidate = {
candidateId: string;
evidence: string[];
};
type CandidateScore = {
candidateId: string;
rubricVersion: "rubric-v3";
scores: {
technical: number;
communication: number;
roleFit: number;
};
};
type ChatResponse = {
choices: Array<{ message: { content: string | null } }>;
};
function isScore(value: unknown): value is number {
return (
typeof value === "number" &&
Number.isInteger(value) &&
value >= 0 &&
value <= 4
);
}
function isCandidateScore(value: unknown): value is CandidateScore {
if (typeof value !== "object" || value === null) return false;
const row = value as Record<string, unknown>;
const scores = row.scores;
if (typeof scores !== "object" || scores === null) return false;
const scoreMap = scores as Record<string, unknown>;
return (
typeof row.candidateId === "string" &&
row.rubricVersion === "rubric-v3" &&
Object.keys(scoreMap).length === 3 &&
isScore(scoreMap.technical) &&
isScore(scoreMap.communication) &&
isScore(scoreMap.roleFit)
);
}
function isChatResponse(value: unknown): value is ChatResponse {
if (typeof value !== "object" || value === null) return false;
const choices = (value as Record<string, unknown>).choices;
return Array.isArray(choices) && choices.length > 0;
}
function retryDelayMs(response: Response, attempt: number): number {
const retryAfter = response.headers.get("retry-after");
if (retryAfter !== null && /^\d+$/.test(retryAfter)) {
return Number(retryAfter) * 1000;
}
return 1000 * 2 ** attempt;
}
async function wait(milliseconds: number): Promise<void> {
await new Promise((resolve) => setTimeout(resolve, milliseconds));
}
async function main(): Promise<void> {
const apiKey = process.env.INFRAI_API_KEY;
const candidateJson = process.env.CANDIDATE_JSON;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");
if (!candidateJson) throw new Error("CANDIDATE_JSON is required");
const candidate = JSON.parse(candidateJson) as Candidate;
const apiOrigin = ["https://api", "infrai", "cc"].join(".");
const path = "/v1/chat/completions";
const idempotencyKey = crypto.randomUUID();
const body = {
model: "auto",
temperature: 0,
messages: [
{
role: "system",
content:
"Return JSON only. Score technical, communication, and roleFit " +
"from 0 to 4. Include candidateId and rubricVersion rubric-v3.",
},
{ role: "user", content: JSON.stringify(candidate) },
],
};
for (let attempt = 0; attempt < 4; attempt += 1) {
const response = await fetch(`${apiOrigin}${path}`, {
method: "POST",
headers: {
Authorization: `Bearer ${apiKey}`,
"Content-Type": "application/json",
"Idempotency-Key": idempotencyKey,
},
body: JSON.stringify(body),
});
if (response.status === 429 && attempt < 3) {
await wait(retryDelayMs(response, attempt));
continue;
}
if (!response.ok) {
const detail = await response.text();
throw new Error(`API request failed (${response.status}): ${detail}`);
}
const payload: unknown = await response.json();
if (!isChatResponse(payload)) throw new Error("Invalid chat response");
const content = payload.choices[0].message.content;
if (content === null) throw new Error("Chat response has no content");
const score: unknown = JSON.parse(content);
if (!isCandidateScore(score)) {
throw new Error("Model output failed the rubric-v3 JSON contract");
}
process.stdout.write(`${JSON.stringify(score, null, 2)}\n`);
return;
}
}
main().catch((error: unknown) => {
const message = error instanceof Error ? error.message : String(error);
process.stderr.write(`${message}\n`);
process.exitCode = 1;
});
Run the same function with the same candidate set for each finalist, then add actual token counts and billed cost to the external evaluation ledger. Do not estimate the final bill by multiplying a stale public rate by locally counted tokens. Local token counting is valuable for prompt budgets and preflight checks, but the final comparison should reconcile with what was actually billed. Keep raw candidate text out of logs; the ledger needs identifiers and measurements, not resumes.
Alert on the mechanics before alerting on the bill. A useful minimum is acceptance rate by provider and model, fallback rate, 429 rate, and cost per accepted result. Split those signals by rubric version and deployment region. If schema validity falls while transport success stays flat, the API is up but the product outcome is degraded; an HTTP-only dashboard won't show it.
The before/after is crisp. Before, “cheap” means a token rate copied into a spreadsheet. After, it means a reproducible evaluation where each accepted score can be traced to a model, rubric, attempt chain, token count, and billed cost.
Limits that should change the choice
This method is not suitable when the workload cannot be represented by a stable evaluation set or when a provider-native capability is itself the product requirement. In that case, keep the direct provider and test that native path. Also stay direct when a current quote wins materially and the team does not value cross-provider substitution enough to pay the aggregation difference.
Moderation needs an explicit design decision. A runtime without a dedicated moderation endpoint can use a chat model with a JSON Schema fallback, but a safety-sensitive hiring workflow may require a specialized moderation product and separate policy review. Voice and ASR readiness do not affect a text rubric scorer; they do matter if interview audio enters scope. Don't smuggle those future requirements into a text-only winner.
Finally, no token-cost comparison answers data residency, retention, contractual, or model-governance questions by itself. Evaluate those independently before sending candidate material. Your mileage may vary across US and EU workloads because prompt length, output length, model availability, and the share of fallback attempts all move the accepted-result denominator. Re-run the ledger on a schedule and whenever the rubric, model, or routing policy changes.
Top comments (0)