DEV Community

Cover image for I Wired 4 LLM Providers Into One Interface So I'd Never Have to Beg a Single Vendor for Mercy
Suyash Vashishtha
Suyash Vashishtha

Posted on Originally published at myvitals.co.in

I Wired 4 LLM Providers Into One Interface So I'd Never Have to Beg a Single Vendor for Mercy

#ai #node #architecture #healthtech

MyVitals reads your lab reports with AI. "AI" here is actually two different jobs wearing a trenchcoat:

  1. Extraction — read a PDF, pull structured data out, needs strong vision + reliable structured output.
  2. Chat — answer "is my TSH bad" fast and cheap, at conversational volume.

Bolting both onto one model is how you either overpay for reasoning nobody needed on a "thanks!" message, or starve extraction of the quality it actually needs. So they run on independently swappable providers:

LLM_PROVIDER=mistral        # extraction + metric matching
CHAT_LLM_PROVIDER=openai    # chat
Enter fullscreen mode Exit fullscreen mode

Swapping either is a .env edit and a restart. Not a deploy. Not a refactor.

One interface, four adapters

Every provider — Gemini, OpenAI, Anthropic, Mistral — implements the same three methods: extractReport, matchUnmatchedMetrics, chat. No controller anywhere talks to a provider SDK directly. Four vendors also means four dialects of "give me JSON back" (responseSchema, json_schema strict mode, output_config.format, OCR annotation format) — normalized once behind a shared schema instead of four copies of prompt text slowly drifting apart.

Two billing models, one surprising economics fact

Mistral's OCR extraction is billed per page. Everything else is billed per token. Which means: a report upload costs roughly the same no matter how chatty your history is, but a chat message gets more expensive the longer you've used the product — because grounded chat means stuffing your entire metric history into the prompt every turn (no vector DB, deliberately — per-user data is small enough that RAG would solve a problem that doesn't exist yet). Your most retained users are mechanically your most expensive ones. Nobody puts that in the pitch deck.

The bug: 400 Invalid Model, for a month, silently

MISTRAL_MODEL (goes to Chat Completions, used for metric matching) got set to an OCR-only model name. Every matching call failed with 400 Invalid Model for roughly a month — and nobody noticed, because the pipeline is designed to degrade: a failed matching call just falls through to auto-creating a new dictionary entry instead of finding the existing one. Every upload still reported success. The only symptom was a canonical-metrics dictionary quietly fragmenting in the background.

Lesson: if a stage is designed to degrade instead of fail, its failure rate needs its own visible metric — "the operation succeeded" tells you nothing about whether the expensive, accurate path ran at all.

Aliases in, concrete models logged

Env vars hold aliases (mistral-small-latest) so vendor upgrades land free. But logging the alias into a cost table is useless six months later. So the call goes out with the alias, and a resolver (cached 6h, never throws) logs what it actually resolved to — mistral-small-2603 — alongside the alias. Upgrade for free, keep history that still means something.

👉 Try MyVitals now — upload a PDF and watch it get read, matched, and charted.

Top comments (0)