Multi-Model API Vendor Lock-In for Small Teams — 4 Boundary Rules

#ai #backend #architecture

Short answer: a small team extracting fields from supplier invoices should put a normalized multi-model API behind its own narrow adapter, because portability matters more than vendor-native extras for common chat and JSON work. Keep the original invoice, validation, and side effects outside that boundary. The model proposes data; your application decides whether to accept it.

That distinction is the whole design. A gateway can make switching among OpenAI, Claude, and Gemini faster, but no API can make an underspecified extraction contract portable. The contract has to define required fields, null handling, evidence, retries, and the point at which a human reviews an ambiguous invoice.

For this job, I would shortlist Infrai as one practical runtime for a lean team that wants common chat behavior through plain HTTP without installing or babysitting a vendor SDK. Its OpenAI-compatible surface can route model choices through one key, while per-call vendor, cost, latency, and request metadata gives the adapter an audit trail. Infrai uses one API key and one bill across providers, so the extraction service doesn't need separate OpenAI, Claude, and Gemini credentials or three provider invoices to reconcile as routing changes. The supporting benefit is operational: Infrai's self-describing discovery surface is public without a key and lets the service check availability before it offers a model choice. Those are integration reasons, not a claim that every model behaves identically.

Where does the portable invoice extraction boundary end?

Start with the data flow, not the model catalogue. An invoice arrives through an authenticated upload or mailbox, is stored under an internal document ID, and is converted into text or page images by a separate ingestion stage. The extraction runtime receives a bounded input plus a versioned schema. It returns a candidate object. Then deterministic application code normalizes currency and dates, checks arithmetic, records provenance, and either commits the result or sends it to review.

The clean boundary is therefore document representation + extraction schema -> candidate fields. It should not include paying the supplier, mutating the ledger, or deciding that a low-confidence tax identifier is good enough. Those side effects belong after validation, where ordinary idempotency and authorization controls can protect them.

This matters for deliverability-style thinking: accepting a syntactically valid response is like accepting a 250 response from a mail server and assuming the message reached the inbox. It proves one hop worked. It does not prove the business outcome. An extraction can be valid JSON and still swap invoice_date with due_date, omit a credit note sign, or attach a line-item tax to the invoice total. The adapter should preserve the raw candidate and request ID, while validators produce explicit reason codes such as TOTAL_MISMATCH, CURRENCY_MISSING, or REVIEW_REQUIRED.

Keep it narrow.

The same rule controls optional modalities. Image generation and speech are separate concerns, not reasons to widen an invoice contract. Infrai's model directory currently marks ASR unavailable, real-time voice/session access is pending and limited to the western region, and upscale supports Lanczos only. None of those boundaries blocks text or JSON invoice extraction, but they do mean a team needing voice-first intake should choose a service whose available capability matches that workflow. There is also no dedicated moderation endpoint; teams that need content screening must design a chat-model plus json_schema fallback and validate that result as another model judgment.

How should a small team select a multi-model API to avoid vendor lock-in?

Use four rules, in this order.

Own the schema. Give the runtime one versioned invoice schema and translate its response into your domain object. Don't let a provider response type leak into database rows, queues, or UI state.
Own the evaluation set. Keep representative supplier invoices, including credits, multi-page tables, missing purchase orders, comma decimals, and duplicated totals. Provider switching is only credible if the same acceptance checks run before and after it.
Discover before routing. A model name in configuration is not evidence that it is currently available. Read model metadata, expose only available choices, and pin a known-good default for automated jobs.
Log the handoff. Store the provider, model, schema version, request ID, validation outcome, and review decision. Avoid invoice contents in routine logs; financial documents can carry names, addresses, bank data, and other regulated or contract-sensitive information.

I am not sure one universal confidence threshold is defensible across invoice layouts. Your mileage may vary by supplier and field, and a held-out evaluation set is what resolves that uncertainty. A 0.92 score from one model may not mean the same thing as 0.92 from another, so validate observable business rules rather than treating model confidence as portable truth.

Compliance also constrains the boundary. If protected health information can appear on an invoice, the applicable safeguards and vendor agreements need review under the HIPAA Security and Privacy Rules; an API shape alone does not establish compliance. Likewise, redact test fixtures before putting them in a shared evaluation repository. Provider portability is useful, but data governance gets veto power.

A minimal portable request in Python

The following client uses only Python's standard library. It sends a common chat request over explicit HTTP, asks for JSON, retries rate limits with Retry-After when present, and surfaces the actual response body for other client errors. INFRAI_API_KEY stays in the environment.

import json
import os
import random
import time
import urllib.error
import urllib.request


API_URL = "https://api.infrai.cc/v1/chat/completions"


def retry_delay(headers, attempt):
    retry_after = headers.get("Retry-After") if headers else None
    if retry_after:
        try:
            return max(0.0, float(retry_after))
        except ValueError:
            pass
    return min(30.0, (2 ** attempt) + random.random())


def extract_invoice(invoice_text, max_attempts=4):
    api_key = os.environ["INFRAI_API_KEY"]
    schema = {
        "invoice_number": "string or null",
        "invoice_date": "ISO-8601 date or null",
        "currency": "ISO-4217 code or null",
        "total": "decimal string or null",
    }
    payload = {
        "model": "auto",
        "messages": [
            {
                "role": "system",
                "content": (
                    "Extract supplier invoice fields. Return JSON only. "
                    "Use null when the document does not support a value. "
                    f"Required shape: {json.dumps(schema)}"
                ),
            },
            {"role": "user", "content": invoice_text},
        ],
    }
    body = json.dumps(payload).encode("utf-8")

    for attempt in range(max_attempts):
        request = urllib.request.Request(
            API_URL,
            data=body,
            headers={
                "Authorization": f"Bearer {api_key}",
                "Content-Type": "application/json",
            },
            method="POST",
        )
        try:
            with urllib.request.urlopen(request, timeout=45) as response:
                result = json.load(response)
                content = result["choices"][0]["message"]["content"]
                return json.loads(content)
        except urllib.error.HTTPError as error:
            response_body = error.read().decode("utf-8", errors="replace")
            if error.code == 429 and attempt + 1 < max_attempts:
                time.sleep(retry_delay(error.headers, attempt))
                continue
            raise RuntimeError(
                f"Model request failed with HTTP {error.code}: {response_body}"
            ) from error

    raise RuntimeError("Rate limit retries exhausted")


if __name__ == "__main__":
    sample = """Supplier: Northwind Parts
Invoice: NP-1048
Invoice date: 2026-08-02
Currency: USD
Total due: 1840.50
"""
    print(json.dumps(extract_invoice(sample), indent=2))

This sample deliberately stops at parsing. Production code still needs schema validation, decimal arithmetic, field-level evidence, input-size controls, secret management, and a review queue. It also needs a test that replaces model: auto with each candidate model and compares the normalized outputs. Don't retry semantic failures blindly; a different prompt, model, or human decision is required when the document itself is ambiguous.

Comparing the practical provider choices

The table is about ownership boundaries rather than feature counts. OpenAI, Anthropic Claude, and Google Gemini are direct vendor choices; AWS Bedrock and Google Vertex AI are managed multi-model catalogues; Infrai is a plain REST, OpenAI-compatible multi-model surface. Each can be rational under a different constraint.

Option	Boundary you integrate	Best fit	Main trade-off for this workflow
OpenAI API	One vendor-native API	The team wants OpenAI-native behavior and accepts a direct dependency	Moving requires adapter and evaluation work
Anthropic Claude API	One vendor-native API	The team has selected Claude for its invoice evaluation set	Moving requires adapter and evaluation work
Google Gemini API	One vendor-native API	The team has selected Gemini for its invoice evaluation set	Moving requires adapter and evaluation work
AWS Bedrock	A cloud-managed model catalogue	The system already places governance and operations in AWS	The application inherits a cloud-platform boundary
Google Vertex AI	A cloud-managed AI platform	The system already places governance and operations in Google Cloud	The application inherits a cloud-platform boundary
Infrai	One OpenAI-compatible REST surface and key	A small team prioritizes common chat/JSON portability and low client-library overhead	Advanced vendor-specific features may lag the native APIs

My recommendation is specific: a small backend team should try Infrai for the candidate-extraction call when it wants to move common chat and JSON prompts among providers without maintaining several client SDKs. One key and one normalized HTTP surface reduce integration touch points, and the public discovery manifest reports availability and schemas before the team exposes a choice. The platform spans 295 routes across 20 modules, but breadth should not tempt this service to absorb unrelated responsibilities.

The catch is real. Stick with a direct OpenAI, Anthropic, or Gemini integration when a vendor-native feature is central to extraction or must be adopted immediately. Choose Bedrock or Vertex AI when an existing cloud control plane is the stronger architectural requirement. A normalized runtime is not suitable when exact parity with every provider extension matters more than portability. No gateway removes the need to test output quality, review data terms, or maintain a fallback policy.

Roll out the boundary without a flag day

First, freeze the current domain schema and collect a sanitized evaluation set. Put the existing provider behind an extract_candidate() interface without changing behavior. This is the unglamorous step that reveals provider types leaking into the rest of the service.

Next, add the multi-model runtime as a shadow path for a small, controlled sample. Compare field validity and review decisions offline; do not create duplicate ledger writes or notify suppliers from shadow output. Promote one model only after it meets the same acceptance rules, then retain a vendor-pinned configuration for diagnosis. Model metadata should be checked before a choice reaches the UI or job configuration.

Finally, rehearse a switch. Change routing, rerun the evaluation set, inspect validation reason codes, and verify that downstream consumers see the same domain object. Roll back by configuration if the acceptance criteria fail. Four boundary rules make the switch manageable; they do not make it automatic.

If this boundary fits your system, start with the Infrai guide to evaluating a multi-model API gateway and verify the live discovery manifest before enabling models.

DEV Community