Routing your LLM traffic through a gateway is handing a stranger two things at once: your prompts and your billing. Discovery is easy — there are dozens of "one API for every model" services now. Knowing which one won't silently swap models, pad invoices, or vanish with your balance is the hard part.
I've written before about vetting agent skills and MCP servers before you trust them. Same energy, bigger blast radius: a bad skill wastes an afternoon; a bad gateway sits in the middle of every production request you make.
Full disclosure up front: we run one of these — ApiFlux. So instead of a neutral survey, this post does something more useful: here are the five checks I'd run against any gateway, and then I run them against our own product in public, including the ones we currently fail.
Why bother: the gateway failure modes nobody puts on the landing page
Every gateway sells the same three lines: one key, lower prices, automatic failover. The actual risks live one layer down:
Markup opacity. "Up to 50% off" against an unstated baseline is not a price. It's a vibe.
Model substitution. You pay for a frontier model; the request quietly lands on something cheaper. Unless you're fingerprinting outputs, you'll never notice — your evals just get mysteriously worse.
Prompt logging. Your traffic is their data unless a policy says otherwise.
Operator risk. Prepaid balance + anonymous operator + no legal entity = the oldest exit scam in the business.
None of these are hypothetical. They're the standard failure modes of an industry where anyone can put an OpenAI-compatible proxy behind a landing page in a weekend.
Check 1 — Demand per-token numbers, not discounts
A trustworthy gateway shows you the exact per-1M-token price per model, per route, with a "last updated" date — so you can diff it against the provider's own list price and see the margin with your own eyes.
Red flags: prices only visible after signup, "credits" with fuzzy exchange rates, discounts quoted against nothing.
Quick test: pick one model you know the official price of, and find it on the gateway's public pricing page in under a minute. If you can't, that's the answer.
Check 2 — Verify the model is what it says
Model substitution is the quiet killer. Three cheap fingerprints, no tooling required:
1. Ask for the model's knowledge cutoff and a fact just inside it.
2. Run a prompt your target model reliably formats a specific way
(system-prompt adherence differs sharply between model families).
3. Compare token counts: same prompt, gateway vs. official API.
Tokenizers differ between families — a big mismatch means a swap.
Run them on day one, then randomly. A gateway that welcomes this kind of testing is structurally different from one that gets cagey about it.
Check 3 — Read the logging and data policy like it's code
You're looking for three specific answers, not vibes: Is prompt content stored, or only metadata? For how long? Is anything used for training? "We take privacy seriously" answers none of these. A dated, versioned policy page answers all three.
Check 4 — Failover claims need a visible status history
Everyone advertises 99.9%. The question is whether you can see it: a public status page, per-model uptime history, incident postmortems. If reliability is the pitch, the receipts should be public. "Trust us" is not an SLA.
Check 5 — Know who you're prepaying
Prepaid balance is a loan to the operator. Before you top up, find: the legal entity and jurisdiction, the refund window for unused balance, what happens to your balance if they shut down. If the answer to "who do I even complain to" is a Telegram handle, size your deposit accordingly.
Grading ourselves: where ApiFlux passes and where it doesn't (yet)
The same five checks, run against ApiFlux — verifiable from public pages, not from my word:
Three passes, one partial, one honest fail. I'd rather publish the fail than pretend the checklist only applies to other people — an AI gateway asking for your production traffic should be gradeable by its own standards, in public.
The honest caveat
No checklist catches intent. A gateway can pass all five today and degrade tomorrow; the only durable defense is that these checks are cheap enough to re-run — pricing diff, fingerprint prompts, status page glance — quarterly, like dependency updates. Five minutes per quarter against the service that sits in the middle of every AI request you make is the best ratio in your stack.
If you want to run the pricing check against us right now: apiflux.ai — every price is public, and check 4 should be off the fail list shortly. Hold us to it.
Top comments (0)