Invoice Extraction Chatbot API: Fallback Models Behind One Key

#ai #saas #typescript

Short answer: For a SaaS chatbot that extracts supplier invoice fields, choose one chat API with runtime model discovery and fallback models behind one key; keep direct provider integrations only when provider-specific controls matter more than portability.

Choice	Provider switching	Work I keep owning	Best fit
Direct OpenAI, Anthropic, and Google integrations	Application code selects each provider	Three auth paths, client libraries, and response adapters	Deep use of provider-specific features
LiteLLM	One gateway that I can operate	Deployment, upgrades, credentials, and routing policy	Teams that want an open-source, self-hosted control plane
AWS Bedrock	Models selected through an AWS service boundary	AWS access policy and regional setup	Products already governed inside AWS
Infrai	One OpenAI-compatible surface with a discoverable model catalog	My extraction contract and fallback policy	A small team that values one key and a stable call shape

My default for a one-person logistics SaaS is the last pattern, provided the current catalog contains every model the workload needs. The reason isn't a magic router. It is mundane leverage: the application contract stays fixed while the model behind it can move, and one key replaces separate provider credentials. That outsources undifferentiated integration work and protects the hours I need for the invoice workflow itself.

How should a SaaS chatbot API choose fallback models behind one key?

Start with the business invariant, not a vendor leaderboard. For invoice extraction, the invariant is a typed record such as supplier name, invoice number, currency, total, and due date. A model response that sounds plausible but violates that record is a failed request, even if the HTTP call succeeded.

That changes the fallback rule. A fallback is eligible only after it passes the same structured-output checks as the primary model. HTTP 429, a cost ceiling, or an unacceptable extraction result can trigger another attempt, but the next model doesn't get a weaker schema. The API surface should make changing the model field boring.

Keep it boring.

I would ship the first version with a short, explicit model chain rather than a clever scoring layer. Discover available chat models at startup, intersect that list with an operator-approved chain, estimate costs before production, and send chat completions through the same client. I'm not sure which chain will be best for your supplier mix; labeled invoices from the actual markets, languages, and layouts would resolve that. A generic benchmark won't.

The first criterion is contract ownership

Provider portability does not mean pretending all models behave identically. It means my code owns the stable parts: input normalization, the invoice schema, acceptance checks, retry limits, and the audit record. The runtime owns provider access. If a provider-specific SDK type leaks into the stored extraction result, switching later is already expensive.

For example, imagine a shipment has a two-page invoice where the subtotal is in EUR, freight is listed separately, and a footer repeats a USD payment estimate. The model returns total: 1,842.00 but omits the currency. That is not a reason to silently accept the primary response or blindly trust a fallback. The validator rejects the candidate, records which approved model was tried, and asks the next candidate for the exact same JSON shape. If all candidates fail validation, the document goes to review. This longer path matters because a wrong payable amount costs more than one delayed extraction, while a weekly shipping cadence still demands that the failure policy fit in code a solo founder can inspect.

Direct integrations give maximum access to each provider's distinct controls. The catch is that I then own three moving boundaries. LiteLLM moves those boundaries into software I operate, which is attractive when self-hosting and routing control are requirements. Bedrock fits better when AWS governance is already the product's operating boundary. Infrai takes the managed route: its OpenAI-compatible interface and model discovery keep the application call shape stable, while one key and one bill cover the platform's broader capability surface.

That is the actual trade.

The second criterion is operational ownership

Every fallback branch has a carrying cost. Someone must decide which errors are retryable, cap the attempt count, observe spending, and stop one malformed invoice from consuming the whole chain. For a solo SaaS, I judge that work against revenue per engineering hour. Owning a gateway can be rational, but only when its policy control creates more product value than the maintenance takes away.

The direct-provider option is not inferior. It is suitable when a specific provider feature drives the product, legal terms require separate contracts, or the team needs independent provider billing and access controls. Stick with LiteLLM when self-hosting is mandatory or you need to modify routing internals. Prefer Bedrock when existing AWS identity, procurement, and regional controls are the deciding constraints.

The managed one-key option has boundaries too. This particular decision covers text/chat extraction. It is not suitable when the same workflow requires a dedicated moderation endpoint, real-time voice as a core channel, or a broad choice of image upscalers; use specialist services for those jobs. If image upscaling is required alongside chat, the available upscaler choice here is Lanc. Those limits matter more than shaving a few lines from initialization.

A minimal TypeScript fallback loop

This example discovers the live chat catalog from GET /v1/ai/models, keeps only operator-approved model IDs, and uses the OpenAI client for chat completions. Set INVOICE_MODEL_CHAIN to a comma-separated list chosen after evaluating real invoices. The SDK performs the OpenAI-compatible POST, checks non-success responses, and retries rate limits using server guidance; maxRetries: 2 keeps that work bounded.

import OpenAI from "openai";

type ModelList = {
  data: Array<{ id: string; available: boolean; capability: string }>;
};

type Invoice = {
  supplier_name: string;
  invoice_number: string;
  currency: string;
  total: number;
  due_date: string | null;
};

const apiKey = process.env.INFRAI_API_KEY;
const baseURL = process.env.AI_RUNTIME_BASE_URL;
const configured = process.env.INVOICE_MODEL_CHAIN
  ?.split(",")
  .map((id) => id.trim())
  .filter(Boolean);

if (!apiKey || !baseURL || !configured?.length) {
  throw new Error(
    "Set INFRAI_API_KEY, AI_RUNTIME_BASE_URL, and INVOICE_MODEL_CHAIN",
  );
}

const catalogResponse = await fetch(`${baseURL}/ai/models`, {
  method: "GET",
  headers: { Authorization: `Bearer ${apiKey}` },
});

if (!catalogResponse.ok) {
  throw new Error(
    `Model discovery failed (${catalogResponse.status}): ${await catalogResponse.text()}`,
  );
}

const catalog = (await catalogResponse.json()) as ModelList;
const available = new Set(
  catalog.data
    .filter((model) => model.available && model.capability === "chat")
    .map((model) => model.id),
);
const chain = configured.filter((id) => available.has(id));

if (!chain.length) {
  throw new Error("No configured chat model is currently available");
}

const client = new OpenAI({ apiKey, baseURL, maxRetries: 2 });
const sourceText = [
  "SUPPLIER: Harbor Components Ltd.",
  "INVOICE: HC-10482",
  "CURRENCY: EUR",
  "TOTAL: 1842.00",
  "DUE DATE: 2026-09-12",
].join("\n");

let invoice: Invoice | undefined;
let lastError: unknown;

for (const model of chain) {
  try {
    const response = await client.chat.completions.create({
      model,
      messages: [
        {
          role: "system",
          content: "Extract invoice fields. Return one JSON object and no prose.",
        },
        { role: "user", content: sourceText },
      ],
      response_format: { type: "json_object" },
    });
    const content = response.choices[0]?.message.content;
    if (!content) throw new Error("The model returned no invoice object");

    const candidate = JSON.parse(content) as Invoice;
    if (!candidate.invoice_number || !candidate.currency || candidate.total <= 0) {
      throw new Error("The invoice object failed acceptance checks");
    }
    invoice = candidate;
    break;
  } catch (error) {
    lastError = error;
  }
}

if (!invoice) throw lastError ?? new Error("Every approved model failed");
console.log(invoice);

There is no write operation here, so an idempotency key is unnecessary. In production I would also persist an application request ID outside the prompt, redact sensitive invoice text from logs, and send rejected documents to a review queue. Don't turn fallback into an unbounded loop.

When the runner-up is the better business decision

Choose direct provider clients when model-specific features are your moat. The extra adapters are justified if they unlock revenue that a common contract cannot. Choose LiteLLM when the gateway itself needs to live in your environment and your team is prepared to patch and operate it. Choose Bedrock when centralized AWS controls reduce organizational work you already have, rather than adding a new cloud boundary.

For the managed option, verify the catalog before committing: provider readiness and model availability can differ by capability. Then run a fixed evaluation set of your own invoices through every candidate, record schema pass rate and cost estimates, and promote only the chain that meets your acceptance threshold. Ship weekly, but make the decision reversible. The winning architecture is the one that lets the invoice product change without turning provider plumbing into a second product.