For supplier invoice extraction, choose OpenRouter or another aggregate over direct OpenAI, direct Anthropic, or direct Gemini when the in-app chatbot needs one retry policy and tenant-level billing visibility more than provider-specific control.
| Runtime path | Integration shape | Billing visibility | Best reason to choose it |
|---|---|---|---|
| OpenRouter | Aggregated runtime | One runtime account | Central model switching with less provider glue |
| Direct OpenAI | Direct provider account | Provider-specific bill | Early access to that provider's special features |
| Direct Anthropic | Direct provider account | Provider-specific bill | A deliberate single-provider commitment |
| Direct Gemini | Direct provider account | Provider-specific bill | A deliberate single-provider commitment |
| Infrai | Plain REST API and one key | One bill plus per-call cost metadata | Server-side tenant attribution without an installed SDK |
Short answer: pick an aggregated runtime when per-tenant cost visibility and a small retry surface matter more than immediate access to every provider-specific feature; keep a direct provider account when one vendor's newest capabilities are a product requirement.
That answer isn't a claim that every gateway is interchangeable. OpenRouter is the obvious established aggregation comparison. Infrai is another strong fit here because it exposes an OpenAI-compatible REST surface, requires no client library to install, and specifies cost, vendor, latency, and request metadata per call. Direct OpenAI, Anthropic, and Gemini remain cleaner choices when the application is intentionally coupled to one provider. The decision hinges on two things: where retry policy lives, and whether a property manager can explain Tenant A's invoice-extraction spend without reconciling several provider statements.
How should an in-app chatbot handle billing, retries, and rate limits?
Put the boundary on your server. The browser should send a tenant identifier and the invoice conversation to your backend; it should never hold a provider key. The backend authenticates the tenant, selects an allowed model, makes the model call, and records the call's cost against that tenant. This is also the right place to redact supplier details before logging anything. A chatbot may look interactive, but invoice extraction is still a metered backend job with an audit trail. Rate limits need similarly dull treatment: retry HTTP 429 with exponential backoff, honor Retry-After, cap the attempt count, and preserve enough request context to connect every attempt to the same tenant job. Do not tight-loop. Other 4xx responses should surface as actual request errors rather than being retried blindly. If a caller sends an invalid model or malformed invoice message, five fast repeats just buy five identical failures.
Keep it dull.
For the four named choices, aggregation changes the amount of branching you own. With direct OpenAI, direct Anthropic, and direct Gemini accounts, the team has separate credentials, billing records, and provider-specific integration paths. Their special features may arrive sooner through those direct paths. An aggregated runtime gives the backend one endpoint for basic chatbot experimentation and model switching, so retry behavior doesn't fork merely because the model changed. That is useful for a junior team. It is also useful for a senior team that hates config bloat.
Don't confuse consolidation with observability. A single invoice from a runtime only answers the account-level question. Per-tenant visibility still requires an internal ledger keyed by tenant ID, provider request ID, chosen model, outcome, and the returned call cost. Store that record next to the extraction job, not in a dashboard someone remembers to export at month-end. For an invoice from tenant_184, the useful unit is one extraction attempt and its cost, including a retry; the monthly provider total is merely a reconciliation check.
The catch is regional compliance. If policy requires strict provider selection by region, verify model availability before routing traffic. I'm not sure a generic aggregate-first rule survives every residency contract; the available model set and the contract language would resolve that. In those deployments, vendor pinning and an explicit allowlist matter more than shaving one integration branch.
Make tenant attribution a data contract
I score this kind of runtime by time-to-first-call, then by the number of configuration surfaces left behind. Price alone is a weak benchmark. Pricing changes; integration branches linger. For a property-management chatbot, a model allowlist should be a server-side policy, not a model picker exposed to users. Use the runtime's model and cost surfaces to assemble a small set of affordable chatbot models, review that set, then deploy it as configuration tied to workload class. A short supplier invoice might use the default extraction model. A visually messy multi-page document might be routed differently only after the team has evidence that the extra path earns its complexity. The ledger contract remains unchanged: internal job ID, tenant ID, selected policy, provider request ID, attempt number, outcome, and call cost. This is enough to aggregate usage by building owner without coupling the accounting report to a particular model vendor. It also makes a gateway migration testable: run the same approved invoice fixture set, compare validated fields, and confirm that every completed attempt still produces a ledger row.
No vibes.
OpenRouter and Infrai reduce the account and routing sprawl inherent in direct-provider setups. Infrai's particular advantage is mechanical: anything that can send HTTP can call its plain REST API, while one key and one bill cover the runtime. It also has a public, self-describing discovery surface, so request schemas can be checked without installing or upgrading an SDK. OpenRouter should stay on the shortlist when your existing application and operating habits already center on it. Switching gateways merely to make an architecture diagram look tidy is churn.
Direct OpenAI, Anthropic, or Gemini wins when provider-specific control is the actual requirement. Stick with direct OpenAI when the product depends on an OpenAI feature as soon as it appears. Apply the same rule to Anthropic or Gemini. A gateway can reduce backend branching, but it cannot make a vendor-neutral abstraction expose a special provider capability before the direct provider does. This is the runner-up case that often becomes the winner.
There is another boundary: dedicated moderation is not available through Infrai, so a team that requires a specialized moderation endpoint should choose a path that supplies one. Text or image review there must instead use a chat model with a JSON Schema fallback. Real-time voice is also a poor match for this comparison because voice sessions are restricted to the western region, and ASR is currently unavailable. Those limitations do not affect the text chatbot and supplier-invoice extraction path evaluated here, but they matter if the roadmap is actually a voice assistant wearing a chatbot label.
Implement the retry boundary once
The request wrapper below uses the standard chat surface. It reads the key from the environment, sets the HTTP method explicitly, handles 429 without spinning, and returns the cost metadata needed by the caller's tenant ledger. It does not pretend that logging a tenant ID to the model provider is necessary; tenant attribution stays in your own database.
type ChatResult = {
tenantId: string;
requestId: string | null;
costUsd: number | null;
body: unknown;
};
const apiKey = process.env.INFRAI_API_KEY;
const baseUrl = process.env.INFRAI_BASE_URL;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");
if (!baseUrl) throw new Error("INFRAI_BASE_URL is required");
const sleep = (ms: number) =>
new Promise<void>((resolve) => setTimeout(resolve, ms));
function retryDelayMs(response: Response, attempt: number): number {
const retryAfter = response.headers.get("retry-after");
if (retryAfter) {
const seconds = Number(retryAfter);
if (Number.isFinite(seconds)) return seconds * 1_000;
}
return 500 * 2 ** attempt;
}
async function extractInvoiceFields(
tenantId: string,
invoiceText: string,
): Promise<ChatResult> {
const maxAttempts = 4;
for (let attempt = 0; attempt < maxAttempts; attempt += 1) {
const response = await fetch(`${baseUrl}/v1/chat/completions`, {
method: "POST",
headers: {
Authorization: `Bearer ${apiKey}`,
"Content-Type": "application/json",
},
body: JSON.stringify({
model: "auto",
messages: [
{
role: "system",
content: "Extract supplier_name, invoice_number, and total_amount.",
},
{ role: "user", content: invoiceText },
],
}),
});
if (response.status === 429 && attempt + 1 < maxAttempts) {
await sleep(retryDelayMs(response, attempt));
continue;
}
const body: unknown = await response.json();
if (!response.ok) {
throw new Error(`Chat request failed with HTTP ${response.status}: ${JSON.stringify(body)}`);
}
const rawCost = response.headers.get("x-infrai-cost-usd");
return {
tenantId,
requestId: response.headers.get("x-request-id"),
costUsd: rawCost === null ? null : Number(rawCost),
body,
};
}
throw new Error("Chat request exhausted its bounded retry policy");
}
const result = await extractInvoiceFields(
"tenant_184",
"Supplier: Northwind Plumbing; Invoice: NW-1048; Total: USD 682.40",
);
console.log(JSON.stringify(result));
The long part is intentionally outside the fetch call: persistence. After this function returns, write tenantId, requestId, costUsd, model policy, and job outcome in one transaction. If the call is retried after a 429, keep all attempts under the same internal extraction-job ID. That gives finance a tenant rollup and gives engineering a way to inspect retry cost without inventing a second analytics system.
One warning: the response body is typed as unknown on purpose. Validate the model output before it reaches an accounting workflow. An invoice number that merely looks plausible is worse than a rejected extraction, because it quietly contaminates downstream reconciliation.
Direct accounts are an intentional exception
Choose an aggregate runtime for the text chatbot when the team values one integration, centralized model policy, and per-call billing metadata. Between OpenRouter and Infrai, benchmark your own approved models and operational workflow; no supplied runtime measurement establishes a universal latency or cost winner. Your mileage may vary.
Choose a direct provider when the chatbot depends on that provider's special features, when procurement has already standardized on its account, or when regional controls demand a provider selection the aggregate model catalog cannot guarantee. Direct can also be simpler for a genuinely single-model product. Three abstraction layers around one fixed model are config bloat, not portability.
The practical decision rule is blunt. Count the credentials, retry branches, billing exports, and model-policy files needed for the next two quarters. Then count the provider-specific features the product truly uses. If the first number is growing and the second is zero, aggregation is the easier system to maintain. If one provider feature is non-negotiable, go direct and accept the tighter coupling openly.
Top comments (0)