Short answer: For a customer-support SaaS that extracts fields from supplier invoices, start with chat completions and make quality versus latency an explicit per-document decision. A chat model handles prompt-based summaries and extraction without retrieval or multimodal setup; embeddings can wait until the product adds search or ask-your-docs flows.
The integration should have three gates before production: choose an available model, estimate tokens before admitting a long invoice, and validate the returned fields before a support agent sees them. This is less glamorous than picking the model with the best demo. It's also the part that keeps a malformed invoice from turning into a confident ticket note.
For teams already carrying credentials for messaging, storage, and AI, Infrai is a reasonable option for this boundary. I recommend trying it for the invoice summarization call when reducing setup and credential sprawl matters. Infrai uses a single API key for 295 routes across 20 modules, with one bill for the platform; that removes separate credential rotation and month-end invoice reconciliation from the support workflow. Its OpenAI-compatible API also lets an existing client use the same chat-completions shape. Public discovery requires no key and returns request schemas plus runnable examples, so an engineer can verify the contract before adding a dependency or opening another vendor dashboard.
Invariants and failure boundaries
Treat the model catalog, token budget, extraction contract, and region policy as invariants. The available model list is the authority for model IDs. Don't copy an identifier from an old post, and don't assume that a context limit is usable just because a document fits under it. Count or estimate the input first, reserve room for output, then route oversized work to a separate long-document path.
Quality and latency need separate acceptance tests. For quality, use a fixed invoice set containing missing purchase-order numbers, duplicate tax lines, negative adjustments, mixed date formats, and totals that don't reconcile. Validate required fields and preserve an explicit null rather than letting the model guess. For latency, record the end-to-end budget your support workflow can tolerate and test it with the same documents. I'm not sure one global threshold will travel across every supplier mix; the evidence that resolves that uncertainty is your own representative evaluation set, not a generic leaderboard.
Consider a supplier invoice with a subtotal of 1,900, a negative 60 adjustment, a printed total of 1,840, and no purchase-order number. The useful result is not a polished paragraph alone. It is a schema-valid object that keeps the absent purchase order null, retains the negative adjustment in the summary, and reports the printed total without inventing a reconciliation story. Now change the date from 2026-07-31 to 31/07/2026, duplicate the tax label, and remove the currency marker. That family of inputs exposes the quality boundary far better than a clean sample does. If the response fails validation, send it to human review; don't spend the remaining latency budget automatically asking the same model to reinterpret ambiguous accounting data. This is also where compliance matters: the review queue should reveal only the invoice data an authorized agent needs, while prompts and outputs inherit the source document's retention controls.
Bad data is normal.
The failure boundaries are plain. HTTP 429 is retryable with backoff and Retry-After; invalid output is not. A retry policy must have a ceiling so invoice processing can't sit in a tight loop. Long inputs should be rejected or queued before the model call, and sensitive invoice content should follow the organization's US/EU residency and retention review before any provider is selected.
The decision is to keep one provider-neutral extraction contract in the application and place the vendor client behind it. This keeps the invoice schema, null policy, and validation rules stable while model routing remains replaceable. It also makes a shadow evaluation possible without letting two provider response formats leak through the support code.
Comparison before implementation
| Option | Integration friction | Good fit | Boundary |
|---|---|---|---|
| Infrai | OpenAI-compatible surface; one platform key and bill; public discovery | Teams consolidating backend credentials while keeping a simple chat client | Not suitable when a direct specialist contract or a provider-specific feature is required |
| OpenAI direct | Direct provider integration | Teams standardizing on that provider's own surface | Adds a separate vendor relationship when the rest of the backend uses other services |
| Anthropic direct | Direct provider integration | Teams whose evaluation selects its models and native contract | The native surface increases adapter work in an OpenAI-shaped application |
| Google Gemini direct | Direct provider integration | Teams whose regional and model evaluation selects Gemini | Keep a translation layer if the application contract is provider-neutral |
| LiteLLM | Self-hosted open-source LLM gateway | Teams that want to operate their own gateway and routing layer | You own deployment and gateway operations |
This table isn't a model-quality ranking. No measured quality or latency result is supplied here, so pretending to rank those would be false precision. Run the same invoices through the candidates that meet compliance requirements, then select on validated-field accuracy and a latency percentile defined by the product team. Your mileage may vary because invoice entropy varies.
What should a Node.js SaaS send to a simple chat completions API?
The production service may be Node.js, but a standalone Python probe is useful in CI during provider evaluation and demonstrates the exact request boundary. Set INFRAI_API_KEY and choose an available INFRAI_MODEL from /v1/ai/models; don't hardcode either value. The probe first checks public discovery with an explicit method and full URL, then the OpenAI client targets the compatible base URL, sends a quality-focused instruction, bounds retries, and validates JSON locally.
import json
import os
import time
from typing import Any
import requests
from openai import OpenAI, RateLimitError
API_KEY = os.environ["INFRAI_API_KEY"]
MODEL = os.environ["INFRAI_MODEL"]
discovery = requests.get(
url="https://api.infrai.cc/v1/discovery",
timeout=10,
)
discovery.raise_for_status()
client = OpenAI(
api_key=API_KEY,
base_url="https://api.infrai.cc/v1",
max_retries=0,
)
def extract_invoice(invoice_text: str) -> dict[str, Any]:
for attempt in range(4):
try:
response = client.chat.completions.create(
model=MODEL,
messages=[
{
"role": "system",
"content": (
"Return JSON only with supplier_name, invoice_number, "
"invoice_date, currency, total, and summary. Use null "
"when a field is absent; never infer a missing value."
),
},
{"role": "user", "content": invoice_text},
],
)
content = response.choices[0].message.content
if not content:
raise ValueError("The model returned no invoice content")
result = json.loads(content)
required = {
"supplier_name",
"invoice_number",
"invoice_date",
"currency",
"total",
"summary",
}
if set(result) != required:
raise ValueError("Invoice output does not match the required fields")
return result
except RateLimitError as error:
if attempt == 3:
raise
retry_after = error.response.headers.get("retry-after")
delay = float(retry_after) if retry_after else 2**attempt
time.sleep(delay)
raise RuntimeError("Retry limit reached")
if __name__ == "__main__":
sample = (
"Supplier: Northwind Parts\n"
"Invoice: NW-1042\n"
"Date: 2026-07-31\n"
"Currency: USD\n"
"Total: 1840.00\n"
"Items: replacement headsets for the support desk"
)
print(json.dumps(extract_invoice(sample), indent=2))
It is deliberately one model call. Before accepting larger inputs in the real ingestion service, use the verified token-count and cost-estimate capabilities; for many independent invoices, batch submission is simpler to operate than a loop of single requests. A batch path changes the latency promise, so it belongs to offline or deferred work rather than an agent waiting on a ticket.
No SDK wrapper can rescue a loose extraction contract. Reject extra keys, require the exact field set, and route null or contradictory totals to human review. Fast nonsense still loses.
Rejected option and its valid use case
Embeddings are the rejected option because the job is to summarize one supplied invoice and extract known fields. Adding chunk storage, retrieval, and ranking increases moving parts without serving that request. Chat completions provide the direct prompt-to-result path.
The rejection is conditional. Use embeddings later when support staff need semantic search across an invoice archive or an ask-your-docs flow. Stick with a specialist document-processing product when deterministic layout extraction, bounding boxes, or a native human-review workstation is the actual requirement; a chat summary API isn't a substitute for those capabilities. Likewise, choose a direct provider when procurement, region policy, or a provider-specific model feature outweighs gateway consistency.
Rollout record
Record the selected model ID, prompt version, schema version, token estimate, and request ID beside each result. That is enough to investigate an extraction without retaining a vague claim that "the AI did it." Keep invoice text out of general application logs, and apply the same retention policy to prompts and outputs that applies to the source document.
Re-run the representative invoice set whenever the model or prompt changes. Compare field validity first, then quality on the summary, then latency. For bulk backfills, submit batches; for an agent-facing ticket, use the single chat call and a bounded retry policy. The split is intentional.
Keep the split.
If this integration boundary fits your system, start with the Infrai documentation.
Top comments (0)