Short answer: no public price table can establish whether OpenRouter, direct OpenAI, or Claude API access is cheapest for a SaaS app; compare each route by cost per accepted supplier invoice within the same latency budget. Keep one tested fallback behind your own TypeScript interface; a cheap token is irrelevant when a malformed total, a missed currency, or a slow retry forces another call.
For invoice extraction, I wouldn't select between OpenRouter, direct OpenAI, and the Claude API from a public price table alone. Run the same redacted invoice set through each route, validate the same fields, and count every attempt. A unified key can reduce integration work; direct accounts can keep provider-specific controls visible. Neither property settles the quality-versus-latency decision.
The practical unit is an accepted invoice.
Should a Node.js SaaS app compare direct API routes with a unified key?
Start with a frozen evaluation corpus that resembles production, including the documents that make extraction awkward: multi-page invoices, credit notes, repeated tax rows, comma decimal separators, missing purchase-order numbers, and scans with rotated pages. Keep tenant data out of a shared benchmark unless its handling and retention are explicitly approved. The point is to compare routes against identical inputs, not to reward whichever route received the easy documents that week.
Define acceptance before sending a request. For a supplier invoice, a useful contract might require supplierName, invoiceNumber, invoiceDate, currency, subtotal, tax, and total, plus line-level evidence for fields that drive payment. Syntax is the shallow gate. Semantic checks should reject impossible dates, currencies outside the tenant's policy, duplicate invoice numbers, and totals that do not reconcile within a declared rounding tolerance. Human review is still appropriate for ambiguous documents; the evaluator should record that outcome rather than quietly treating it as model failure.
Then measure three clocks separately: time to first usable response, time through validation, and time to final acceptance after any fallback. A route can look quick at the HTTP boundary yet lose once invalid output triggers a second attempt. Conversely, a slower first pass can win for an interactive workflow if it avoids review often enough. Your mileage may vary because invoice mix, prompt shape, output schema, and model revisions all move the result.
OpenRouter, direct OpenAI, and the Claude API belong in this experiment as routes, not as conclusions. Capture a dated model identifier and a dated pricing snapshot for each candidate. Don't assume models with similar labels have interchangeable behavior, and don't combine results from different prompts into one league table. The cheapest route is the one with the lowest observed accepted-result cost under the same contract; without that workload evidence, I'm not sure a universal cheapest claim means anything.
Use a compact ledger for each attempt:
| Field | Why it belongs in the ledger |
|---|---|
routeId and modelId
|
Reproduce the exact candidate without turning the domain layer into vendor code |
| input and output tokens | Apply the dated rate card outside the request path |
| validation outcome | Separate valid JSON from a payable invoice result |
| elapsed milliseconds | Test the actual product latency budget |
| attempt number and reason | Expose retry amplification and fallback use |
| corpus item hash | Compare identical redacted inputs without logging invoice contents |
Don't average away the tail. Report median and high-percentile acceptance latency, accepted-result rate, review rate, and cost per accepted invoice. Slice those numbers by document class. Ten clean digital PDFs can hide one dense scanned credit note, yet that one note may be exactly where an automatic payment system needs the stricter gate.
Invoice quality is the first gate
The simple approach is to parse JSON, check that keys exist, and route failures to the next model. It fails because plausible strings are not necessarily correct accounting data. "total": "1,234.50" might be valid for one locale, while "1.234,50" needs another parser; accepting both as arbitrary strings only postpones the error. The chosen approach should normalize deterministically, validate arithmetic, and return a small set of outcomes that the router understands.
This focused TypeScript example scores already-normalized model output. It makes no network request, so the same gate can sit behind a direct integration or a unified-key integration.
type InvoiceCandidate = {
invoiceNumber?: string;
currency?: string;
subtotalMinor?: number;
taxMinor?: number;
totalMinor?: number;
};
type Verdict =
| { accepted: true }
| { accepted: false; reason: "missing_field" | "unsupported_currency" | "total_mismatch" };
const allowedCurrencies = new Set(["USD", "EUR", "GBP"]);
function validateInvoice(candidate: InvoiceCandidate): Verdict {
const { invoiceNumber, currency, subtotalMinor, taxMinor, totalMinor } = candidate;
if (
!invoiceNumber ||
!currency ||
subtotalMinor === undefined ||
taxMinor === undefined ||
totalMinor === undefined
) {
return { accepted: false, reason: "missing_field" };
}
if (!allowedCurrencies.has(currency)) {
return { accepted: false, reason: "unsupported_currency" };
}
if (Math.abs(subtotalMinor + taxMinor - totalMinor) > 1) {
return { accepted: false, reason: "total_mismatch" };
}
return { accepted: true };
}
Minor currency units avoid a floating-point comparison in the gate, while the one-unit tolerance makes the rounding policy explicit. Real invoices need more rules: negative totals for credit notes, jurisdiction-specific tax treatment, and line-item reconciliation. Those rules belong in versioned domain code. They shouldn't be buried in a prompt where a copy edit can change payment behavior.
A failed validation is not automatically permission to call another model. Classify it. A truncated response may justify a controlled retry; an unsupported currency is a policy decision; unreadable source pixels may require document review. Blind fallback can pay twice for an input that no model should approve. Keep the original candidate, validation reason, and subsequent decision linked by one correlation ID, but do not put raw supplier details into routine logs.
This is also where the quality-versus-latency policy becomes concrete. An upload workflow may return processing and finish asynchronously, while an operator waiting to approve a bill may need a hard deadline. Set those product budgets first. Then let the router enforce them.
Retry semantics define the latency ceiling
Retries need HTTP semantics, not optimism. RFC 9110 distinguishes idempotent methods because an automatic retry can repeat an intended effect when the client does not know whether the first request succeeded. Model inference is often invoked with POST; even if generating text has no physical side effect, your surrounding code may reserve quota, append an attempt, or enqueue review. Give each logical extraction an idempotency key inside your application and make ledger writes conditional on that key.
A 429 should not become an instant spray across every configured route. Respect an applicable retry delay, add bounded jitter, and stop when the remaining latency budget cannot fit another attempt. A connection timeout is ambiguous — the provider may have processed the request even though the client did not receive the response — so deduplicate your own accounting and downstream actions. Never let a late primary response and a fallback response create two payable invoice records.
The state machine can stay small: pending, attempting, validating, accepted, review, and failed. Store route policy separately from extraction logic. The policy can say that a high-priority tenant gets one alternate route after a retryable transport outcome, while a batch import goes to review after a semantic mismatch. This keeps provider churn out of the invoice domain and makes a unified key an implementation option rather than an architectural dependency.
There is a catch. Hedged requests can reduce tail latency by starting a second route before the first finishes, but they increase token use and complicate cancellation and accounting. They are not suitable when the job is asynchronous or the value of a few saved seconds is lower than a second inference attempt. Stick with one bounded attempt plus queued fallback for bulk imports. For a genuinely interactive approval screen, test hedging against a strict delay and count both calls, including responses you discard.
Batch processing is another distinct lane. The OpenAI Batch API guide describes asynchronous processing with a completion window, so it can fit nightly backfills or non-interactive imports rather than a user waiting on an approval screen. Treat that as a workload split, not a blanket recommendation: the latency contract changes, and the benchmark must label batch results separately from synchronous ones.
Keep it boring.
Operationally, alert on changes in accepted-result rate, review rate, fallback frequency, and latency by corpus class. Token totals alone won't tell you that a prompt revision started dropping tax identifiers. Run a small frozen canary set before changing a model snapshot or validation prompt, and preserve the old policy long enough to compare. The route with the attractive first-call result may stop looking attractive once its review queue is visible.
Read the evaluation as a deployment experiment
Measure at least one full billing cycle if invoice seasonality matters, but don't wait for a perfect study before shipping a guarded version. Begin with a redacted stratified sample, a schema version, an explicit review path, and a hard cap on attempts. Record provider-reported token counts when available and keep the raw rate card version used for cost calculation. Recompute historical ledger rows when you want to test a new price assumption; don't rewrite the original usage facts.
The decision record should state the latency budget, minimum accepted-result rate, permitted review rate, maximum attempts, data-handling constraints, and exit condition. It should also identify the switching cost. A provider-neutral TypeScript interface reduces code changes, but prompts, model behavior, safety controls, regional availability, and contract terms can still create lock-in. A unified key reduces credential handling inside the app, yet it adds another policy and trust boundary. Direct integrations expose separate credentials and operational surfaces. Those are engineering costs even when they don't appear in a token column.
For a solo team, the sensible first release is narrow: one primary route, one fallback condition, one deterministic validator, and one dashboard based on accepted invoices. Expand only when the ledger shows a real failure mode. More routes create more combinations to test, more model changes to watch, and more ways to spend money twice.
The final choice should be reversible. Keep invoice validation and the attempt ledger under your control, pin candidate identities, and rerun the frozen corpus when quality, latency, or terms change. Use the measured frontier rather than a universal ranking: retain any route only while it meets the product's acceptance threshold and latency limit at an acceptable complete cost, including retries and human review.
Top comments (0)