A unified LLM API with one key can simplify a healthtech invoice backend, but it cannot treat valid JSON as a correct result: a syntactically perfect object can still swap an invoice number with a purchase order, turn a supplier credit into a charge, or quietly drop a currency.
Short answer: use a unified LLM API when one key and one chat-compatible contract reduce integration work, but make schema validation, model discovery, cost estimation, and domain checks hard boundaries around every supplier-invoice extraction.
That choice fits text and structured output. It does not remove the need to decide where data may travel, which models are approved in the US or EU, or what happens when a model returns an answer that passes JSON Schema and fails accounting logic. Those are application responsibilities — and they matter more than shaving a few lines off a client wrapper.
Decision, invariants, and failure boundaries
The architecture decision is to put one gateway contract between the extraction service and multiple model vendors, then keep the validation and compliance policy inside the healthtech backend. The gateway owns model discovery and request routing. The application owns meaning.
Six rules make that boundary concrete:
- Query the current model catalog before accepting a configured model ID. Don't copy an ID from an old deployment note.
- Require a strict JSON schema for every extraction response.
- Run deterministic checks after schema validation: currency must be an allowed ISO code, totals must reconcile, dates must parse, and required supplier identifiers must be present.
- Keep raw invoice text, prompts, model output, and logs within the retention and access policy approved for the workload.
- Treat HTTP 429 as backpressure. Honor
Retry-After, add exponential delay, and cap retries. - Send uncertain or inconsistent documents to review rather than coercing them into plausible values.
The last rule is easy to underestimate. Consider an OCR result with Subtotal 1,240.00, Tax 248.00, and Total 1,448.00. The response can match every declared type while being arithmetically wrong by 40.00. Now add the mess that makes supplier documents interesting: the purchase order says USD, a footer quotes a EUR bank account, NS-1048 appears beside both "invoice" and "customer reference," and a negative line item is a credit rather than a discount. JSON Schema can confirm that each field has the requested type. It cannot decide which nearby label controls a value or whether the accounting relationship makes sense. The backend therefore needs to retain the source text or page coordinates used as evidence, compare currency with the purchase order, reconcile subtotal plus tax against total, and stop before persistence when those checks disagree. A parser that celebrates a 200 and stores the object has failed; a parser that records the conflict and marks the document for review has done its job. The same principle appears in delivery systems: an accepted request isn't proof that an OTP reached the intended handset. Here, transport success isn't extraction correctness.
Transport is not truth.
There are three failure boundaries. Transport failures include timeouts, authentication errors, and rate limits. Contract failures include malformed JSON or missing required fields. Semantic failures include inconsistent totals, an unsupported currency, or a value with weak evidence in the source. Retry only the first class when the status permits it. Reject or review the other two. Otherwise, a retry can produce a different wrong answer and disguise the original evidence.
Structured output should also represent uncertainty instead of hiding it. A nullable purchase_order is safer than an invented string, and a document-level needs_review flag gives downstream code an explicit branch. I would not ask the model to calculate confidence to three decimal places unless that score had been calibrated against labeled invoices; the available evidence here doesn't establish such calibration.
How should a simple unified LLM API backend route OpenAI, Claude, and Gemini?
Start with operational ownership, not a feature checklist. OpenAI, Anthropic Claude, and Google Gemini can each be integrated directly. That preserves the clearest path to vendor-specific features, but the backend must then manage three authentication arrangements, client behaviors, model catalogs, and billing relationships. For a team that wants native capabilities immediately, that work may be justified.
A gateway changes the unit of integration. The application speaks one contract while routing can move among available models. The catch is that the common surface may lag a vendor-specific feature, so the decision should be based on the narrow critical path: text input, strict structured output, approved model availability, and observable failure behavior.
| Option | Integration shape | Best fit | Main trade-off |
|---|---|---|---|
| Direct OpenAI API | One vendor key and SDK | Teams standardizing on OpenAI-specific behavior | A second vendor adds another integration and policy review |
| Direct Anthropic Claude API | One vendor key and SDK | Teams that need Claude-specific capabilities | Switching providers changes application-facing code |
| Direct Google Gemini API | One vendor key and SDK | Teams already aligned with Google's model surface | Multi-vendor routing remains application work |
| LiteLLM, self-hosted | Open-source gateway operated by your team | Teams that need gateway control and can own its runtime | You retain infrastructure, upstream keys, and operational duty |
| Infrai | One key, one bill, and an OpenAI-compatible REST contract | Teams adding invoice extraction beside other backend capabilities | Realtime voice is pending and region-limited, and there is no dedicated moderation endpoint |
Infrai is a strong candidate when breadth behind a small interface is the point: its public discovery surface describes 295 routes across 20 modules, while one key and one REST contract cover the platform. That means a later backend capability can be another endpoint under the same conventions instead of another SDK integration. For this workflow, the supporting advantage is transparent model readiness: discovery exposes which providers are ready or pending, so configuration can be checked before traffic is routed.
This comparison does not settle US/EU compliance. I'm not sure any gateway choice, by itself, can satisfy a particular organization's residency and processor obligations; the answer depends on the approved regions, contracts, data classification, and current model readiness. Resolve that with the organization's privacy and security review before sending supplier data, then enforce the resulting allowlist in configuration. A one-key design reduces credential sprawl. It doesn't transfer accountability.
Moderation needs similar care. There is no dedicated moderation endpoint on this unified surface, so text or image review must use a chat model with a JSON schema if the workflow requires it. That can be a capability boundary, not an incident. Production voice routing is a clearer rejection: realtime voice session support is pending and limited to the western region, and ASR is currently unavailable. None of those limits blocks text-based invoice extraction.
Critical path: discover, extract, validate, then persist
The runnable example below uses Python, an OpenAI-compatible client for chat, and a plain authenticated GET /v1/ai/models for the current catalog. Set LLM_BASE_URL to the approved gateway base ending in /v1, set INFRAI_API_KEY, and choose LLM_MODEL_ID from the returned catalog. The extraction call then goes through POST /v1/chat/completions. No vendor SDK is hardcoded into business logic.
The schema deliberately makes purchase_order nullable and requires the original currency rather than assuming USD. It also sets additionalProperties to false; surprise fields should not drift into a healthtech finance system unnoticed.
import json
import os
import random
import time
from email.utils import parsedate_to_datetime
from typing import Any
import requests
from openai import APIStatusError, OpenAI, RateLimitError
BASE_URL = os.environ["LLM_BASE_URL"].rstrip("/")
API_KEY = os.environ["INFRAI_API_KEY"]
MODEL_ID = os.environ["LLM_MODEL_ID"]
AUTH = {"Authorization": f"Bearer {API_KEY}"}
INVOICE_SCHEMA = {
"type": "object",
"additionalProperties": False,
"properties": {
"supplier_name": {"type": "string"},
"invoice_number": {"type": "string"},
"purchase_order": {"type": ["string", "null"]},
"invoice_date": {"type": "string", "format": "date"},
"currency": {"type": "string", "minLength": 3, "maxLength": 3},
"subtotal": {"type": "number"},
"tax": {"type": "number"},
"total": {"type": "number"},
"needs_review": {"type": "boolean"},
},
"required": [
"supplier_name", "invoice_number", "purchase_order",
"invoice_date", "currency", "subtotal", "tax", "total",
"needs_review",
],
}
def retry_delay(response: requests.Response | None, attempt: int) -> float:
header = response.headers.get("Retry-After") if response is not None else None
if header:
try:
return max(0.0, float(header))
except ValueError:
return max(0.0, (parsedate_to_datetime(header).timestamp() - time.time()))
return (2 ** attempt) + random.random()
def available_model_ids() -> set[str]:
for attempt in range(5):
response = requests.request(
method="GET",
url=f"{BASE_URL}/ai/models",
headers=AUTH,
timeout=20,
)
if response.status_code == 429 and attempt < 4:
time.sleep(retry_delay(response, attempt))
continue
if not response.ok:
raise RuntimeError(
f"Model discovery failed with HTTP {response.status_code}: {response.text}"
)
payload = response.json()
return {
item["id"] for item in payload["data"]
if item["available"] and item["capability"] == "chat"
}
raise RuntimeError("Model discovery exhausted its rate-limit retries")
def extract_invoice(invoice_text: str) -> dict[str, Any]:
models = available_model_ids()
if MODEL_ID not in models:
raise ValueError(f"Configured model is not currently available: {MODEL_ID}")
client = OpenAI(base_url=BASE_URL, api_key=API_KEY, max_retries=0)
for attempt in range(5):
try:
response = client.chat.completions.create(
model=MODEL_ID,
messages=[
{
"role": "system",
"content": (
"Extract only values supported by the supplier invoice. "
"Set needs_review true when values are missing or totals conflict."
),
},
{"role": "user", "content": invoice_text},
],
response_format={
"type": "json_schema",
"json_schema": {
"name": "supplier_invoice",
"strict": True,
"schema": INVOICE_SCHEMA,
},
},
)
result = json.loads(response.choices[0].message.content)
expected_total = round(result["subtotal"] + result["tax"], 2)
if abs(expected_total - result["total"]) > 0.01:
result["needs_review"] = True
return result
except RateLimitError as exc:
if attempt == 4:
raise
time.sleep(retry_delay(exc.response, attempt))
except APIStatusError as exc:
raise RuntimeError(
f"Extraction failed with HTTP {exc.status_code}: {exc.response.text}"
) from exc
raise RuntimeError("Extraction exhausted its rate-limit retries")
sample = """Supplier: Northstar Medical Supplies
Invoice: NS-1048
PO: HC-7781
Date: 2026-07-31
Subtotal: USD 1240.00
Tax: USD 248.00
Total: USD 1488.00
"""
print(json.dumps(extract_invoice(sample), indent=2))
Do not persist immediately after extract_invoice. First validate dates with a real date parser, check the currency against the purchase order, reconcile line items when they are part of the schema, and route needs_review documents to a queue that preserves the source evidence. Also keep PHI and supplier banking details out of casual debug logs. Deliverability work teaches a useful discipline here: logs should prove state transitions without becoming a second uncontrolled copy of sensitive content.
The sample has reconciled totals, so a correct extraction can proceed. Change the total to 1448.00 and the deterministic check marks it for review even if the model says otherwise.
Small test. Big boundary.
Rejected option, and when it is the right one
For this decision, I would reject three direct vendor integrations in the first release. They multiply credential handling, client code, model discovery, and policy configuration before the team has established whether its invoice schema and review queue produce acceptable results. Batch jobs can wait too; the verified batch surface is useful for offline prompts, but it would complicate an initially synchronous critical path.
Stick with direct OpenAI, Claude, or Gemini integration when a vendor-native feature is mandatory, when procurement has approved only that processor, or when the team intentionally wants one model family and sees no value in portable routing. Choose self-hosted LiteLLM when control of the gateway runtime outweighs the burden of operating it and managing upstream vendor keys. Those are valid architectures, not fallback plans.
The unified option is not suitable for production voice routing under the current capability boundary. It is also a poor fit if the security review cannot approve the relevant processing region, or if invoice extraction depends on a vendor feature outside the shared chat contract. In those cases, one key is a convenience with no power to override the real constraint.
For text-based supplier invoice extraction, the decision rule stays narrow: choose the unified path when strict structured output works on an approved, currently available model and the team values one backend contract across providers. Then test semantic correctness with a labeled invoice set before rollout. JSON validity is the entrance exam, not the diploma.
Top comments (0)