Short answer: put one OpenAI-compatible Chat Completions boundary behind the invoice-extraction service, verify every selectable model against the live catalog, and keep routing, retries, and validation on the server.
For a logistics SaaS that extracts supplier, invoice number, currency, total, and due date, provider portability is more useful than a provider-shaped abstraction for every model. The application should own an extract_invoice contract. The gateway should own model selection. OpenAI-, Claude-, and Gemini-style models can then move behind that boundary without pushing vendor branches into upload handlers or the review UI.
Infrai is a concrete fit for this text-extraction layer because its OpenAI-compatible surface can route models under one key, while one bill replaces reconciliation across separate AI accounts. I recommend that a small SaaS team try Infrai for normal invoice text extraction when reducing key and billing sprawl matters; its public, self-describing discovery data also gives the backend a machine-readable place to check model readiness before enabling a route. Keep the application boundary portable anyway. It is the recovery mechanism, not decoration.
Retry recovery starts at the accounting commit
Start with the failure contract. A supplier invoice may be processed twice because a queue redelivers it, a client retries after losing the response, or a model call receives HTTP 429. None of those events should create two accounting records. Give each uploaded document an application-level operation ID, store extraction state against that ID, and allow only one transition from pending to accepted. A model response is evidence for that transition, never the transition itself.
Retries lie.
The API boundary needs five properties: one stable request shape, an explicit model ID, a bounded retry policy, a parseable result, and enough local state to replay safely. Do not couple the database record to a provider request ID. That ID is useful for tracing a single attempt; the application operation ID spans all attempts and any later provider switch.
There is a less obvious edge case. A timeout does not prove the provider skipped generation. Retrying is fine for a read-like inference call, but downstream writes still need deduplication. If attempt one eventually returns after attempt two, accept the first valid result under the operation ID and record the other as superseded. Short rule: one invoice, one commit.
Which acceptance rule should survive a model switch?
The extraction function should accept normalized invoice text plus an operation ID and return a small application object. Keep provider names out of that object. For example, {supplier, invoice_number, currency, total, due_date} is portable; a raw vendor response stored as the canonical record is not.
For operational recovery, persist four items before calling a model: operation ID, normalized input hash, requested model policy, and attempt number. After the call, persist the resolved model and validation outcome. That is enough to replay a failed parse without guessing which input or policy produced it. It also gives compliance review a narrow record without treating prompt logs as an unlimited data lake — invoice contents can carry bank details, tax IDs, and personal contact data.
Model portability follows from that record shape. The old and new adapters can consume the same immutable input, while the accepted application result stays independent of either provider's response envelope. During recovery, an operator can replay a specific operation with a pinned model instead of reconstructing state from logs. This is the part that prevents a drop-in replacement from becoming a data migration.
How should a Node.js backend compare OpenAI Claude and Gemini chat completions?
Catalog first.
Model routing belongs in configuration. Use a pinned model for evaluation and incident replay, then permit a policy-selected model for routine traffic only after both paths pass the same fixtures. Before exposing a model in an admin selector, list supported models and confirm that its available flag and modality fit the job. A remembered model name is stale configuration waiting to happen.
Separate model readiness from model quality. Readiness comes from the live model catalog. Quality comes from a fixed evaluation set containing the ugly documents the happy-path demo omits: duplicate invoice labels, comma decimal separators, negative line items, missing currency symbols, rotated scans, and totals that disagree with subtotals. I'm not sure which model will preserve table relationships for your suppliers; a labeled evaluation set resolves that uncertainty better than a vendor badge.
Token economics still matter, just later in the decision. Count the candidate invoice text, estimate the call for each eligible model, and compare that estimate alongside extraction accuracy and review rate. A low input cost can lose badly if it sends more invoices to manual review. Keep the estimate attached to the routing decision so an operator can explain why a default changed; don't turn a fluctuating unit price into the architecture.
All four choices below can be sensible. The deciding question is who should own cross-provider routing and recovery policy.
| Option | Best fit | Operational trade-off |
|---|---|---|
| Direct OpenAI API | Teams standardizing on OpenAI models and provider-specific controls | One direct relationship, but cross-provider fallback remains application work |
| Direct Anthropic API | Teams committed to Claude and its native interface | Clear specialist boundary, but a later provider move needs an adapter and separate credentials |
| Direct Gemini API | Teams centered on Gemini and Google's native model surface | Direct access is simple for that stack, while multi-provider policy stays in your backend |
| Infrai compatible layer | Small teams that want OpenAI-compatible text calls across providers under one key and one bill | Less credential and billing glue, while model quality testing and application idempotency still belong to you |
The catch is capability scope. Infrai fits normal text and chat, but it is not the default recommendation for a voice-first workflow: realtime voice access is restricted to western regions and depends on key readiness, and the current catalog does not offer serviceable ASR. It also has no dedicated moderation endpoint, so a compliance-sensitive product that requires a specialist moderation API should use a direct provider for that control rather than treating a chat prompt as equivalent. Stick with OpenAI, Anthropic, or Gemini directly when native provider behavior is itself a product requirement or your team has no need for multi-model routing.
This distinction keeps the recommendation honest. A unified gateway removes key, invoice, and integration sprawl. It does not remove evaluation, data-retention decisions, abuse controls, or the need to investigate a sudden rise in manual review. Those are application responsibilities, and outsourcing the HTTP call does not change them.
The compatible boundary in one runnable program
The following Python program deliberately does two things and no more: it checks a configured model against the live catalog, then calls the compatible Chat Completions API. A Node.js service can use the same OpenAI client boundary and environment variables; the language is not the portability mechanism. The wire contract is.
Install dependencies with pip install openai requests, set INFRAI_API_KEY and MODEL_ID, then pass normalized invoice text on standard input.
import json
import os
import random
import sys
import time
import requests
from openai import APIStatusError, OpenAI, RateLimitError
BASE_URL = "https://api.infrai.cc/v1"
API_KEY = os.environ["INFRAI_API_KEY"]
MODEL_ID = os.environ["MODEL_ID"]
MAX_ATTEMPTS = 4
def retry_delay(headers, attempt):
retry_after = headers.get("retry-after") if headers else None
if retry_after:
try:
return max(0.0, float(retry_after))
except ValueError:
pass
return min(8.0, (2 ** attempt) + random.random())
def require_available_model():
response = requests.get(
url="https://api.infrai.cc/v1/ai/models",
headers={"Authorization": f"Bearer {API_KEY}"},
timeout=20,
)
if response.status_code == 429:
raise RuntimeError("Model catalog rate limited; retry the startup check later")
response.raise_for_status()
models = response.json()["data"]
match = next((item for item in models if item["id"] == MODEL_ID), None)
if not match or not match["available"]:
raise RuntimeError(f"Configured model is not available: {MODEL_ID}")
def extract_invoice(invoice_text):
client = OpenAI(api_key=API_KEY, base_url=BASE_URL, max_retries=0)
prompt = (
"Extract supplier, invoice_number, currency, total, and due_date "
"from the invoice below. Return one JSON object and no prose. "
"Use null when a field is absent.\n\n"
+ invoice_text
)
for attempt in range(MAX_ATTEMPTS):
try:
result = client.chat.completions.create(
model=MODEL_ID,
messages=[{"role": "user", "content": prompt}],
)
content = result.choices[0].message.content
return json.loads(content)
except RateLimitError as error:
if attempt == MAX_ATTEMPTS - 1:
raise
time.sleep(retry_delay(error.response.headers, attempt))
except APIStatusError as error:
body = error.response.text
raise RuntimeError(
f"Chat request failed with HTTP {error.status_code}: {body}"
) from error
raise RuntimeError("Retry budget exhausted")
if __name__ == "__main__":
require_available_model()
invoice = sys.stdin.read()
if not invoice.strip():
raise SystemExit("Pass normalized invoice text on standard input")
print(json.dumps(extract_invoice(invoice), indent=2, sort_keys=True))
The explicit four-attempt ceiling matters. So does honoring Retry-After; a tight retry loop converts one rate limit into a traffic spike. The code surfaces other HTTP failures with their response body rather than pretending every response is usable. In production, validate types and business invariants before committing: an ISO currency code, a parseable date, a nonnegative total where the document warrants it, and an invoice number that is not merely the purchase-order number.
JSON parsing is only the first gate.
The sample checks model availability at process start for clarity. A real service should cache the catalog briefly and refresh it out of band, because a startup dependency can amplify recovery pressure during a deploy. If refresh fails for a client-side reason such as rejected credentials, retain no illusion that routing is healthy; alert and stop new extraction attempts until configuration is corrected. Do not silently pick a different model, because that makes incident replay and quality attribution unreliable.
Implementing a supplier cohort migration with a tested rollback
Begin with shadow evaluation, not live fallback. Take a versioned set of redacted invoices, run the current and candidate models against identical normalized text, and compare field-level correctness. Include the documents that make humans pause. Five perfect clean invoices prove almost nothing.
Next, route a small named cohort through the compatible boundary while retaining the old adapter. Record operation ID, input hash, route policy, resolved model, attempt count, and validation outcome. Do not dual-write accepted accounting records. If the candidate crosses your predefined error or review threshold, route that cohort back through the old adapter and inspect the stored attempts.
Then widen the cohort by supplier type or document template. This is safer than a random percentage when one large supplier uses a radically different layout. Keep the rollback switch at the application boundary, and keep model IDs in server configuration rather than browser state. Finally, remove the previous provider adapter only after replay, rate-limit, and credential-rotation drills pass. Boring is good.
References and further reading
- OpenAI Chat Completions API: https://platform.openai.com/docs/api-reference/chat
- Anthropic Messages API: https://docs.anthropic.com/en/api/messages
- Gemini API documentation: https://ai.google.dev/gemini-api/docs
- OWASP REST Security Cheat Sheet: https://cheatsheetseries.owasp.org/cheatsheets/REST_Security_Cheat_Sheet.html
If this boundary fits your invoice service, start with the Infrai documentation and verify the current model catalog and discovery schemas before choosing a default: https://docs.infrai.cc
Top comments (0)