DEV Community: yansen zhu

Pointing Claude Code at an AI gateway: what actually breaks (and the 4 env vars that fix it)

yansen zhu — Wed, 19 Aug 2026 01:29:24 +0000

Disclosure: I work on ApiFlux, an AI router with native Anthropic, OpenAI and Gemini endpoints. Everything below applies to any gateway that speaks the Anthropic Messages protocol — the config is not vendor-specific.

If you use Claude Code daily, at some point you'll want to route it through a gateway: for failover when api.anthropic.com returns 529s, for one bill across Claude/GPT/Gemini, or just for a usage log you can actually grep. It should be one environment variable. In practice there are four, and two of them aren't documented anywhere obvious. Here's what broke for us and the config that's been stable for two months.

Why "OpenAI-compatible" is not enough
Most gateways advertise "works with Claude" because they proxy /v1/chat/completions. Claude Code doesn't speak that. It uses the Anthropic Messages API (POST /v1/messages, x-api-key header, anthropic-version header), and the tool-use payload — tool_use / tool_result content blocks, cache_control markers, streaming event types — has no 1:1 mapping onto OpenAI function calling. Translation layers get simple chats right and then quietly drop fields on multi-turn agentic loops. Symptoms: tools that "run" but return empty results, prompt caching silently disabled, or a 400 on the third turn.

Rule of thumb: before pointing Claude Code at a gateway, confirm it exposes a native /v1/messages endpoint. If the docs only mention chat completions, don't.

The four variables
# ~/.zshrc or ~/.bashrc export ANTHROPIC_BASE_URL="https://apiflux.ai" # your gateway; try ".../v1" if you get path errors export ANTHROPIC_API_KEY="sk-your-gateway-key" # the GATEWAY key, not your Anthropic key export DISABLE_INTERLEAVED_THINKING=1 export CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS=1

The first two are the obvious ones. The last two are the ones that cost us an afternoon.

Claude Code sends anthropic-beta headers by default — interleaved thinking, plus whatever experimental betas the current build enables. Some upstreams behind a gateway (Bedrock and Vertex in particular) don't accept every beta flag, and the failure mode is not a helpful error: you get an empty assistant turn or a generic 400. Setting those two variables strips the beta headers and everything becomes deterministic. You lose interleaved thinking, which for coding tasks we haven't missed.

PowerShell users:

$env:ANTHROPIC_BASE_URL="https://apiflux.ai" $env:ANTHROPIC_API_KEY="YOUR_GATEWAY_KEY" $env:DISABLE_INTERLEAVED_THINKING="1" $env:CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS="1"

Verify with claude doctor, then send one cheap request and check it shows up in the gateway's log with the same API key. If the request succeeds but no log appears, you're on a different key/account than you think.

What failover actually looks like
The reason to bother with all this: a single model ID can sit in front of several upstream channels. On our side claude-opus-5 resolves to Anthropic → Amazon Bedrock → Vertex AI; if the primary returns 529/overloaded the router retries the next channel and Claude Code never sees it. We tried maintaining this ourselves with LiteLLM first; rotating three providers' keys and quotas was more ops than we wanted for a side concern.

Two habits that make gateway debugging painless:

Keep the X-Request-Id response header. It's the same ID in the gateway log — paste it into a support ticket instead of a screenshot.
Use a non-production key while evaluating. Zero-data-retention gateways still log metadata (model, tokens, status); you want that log to be yours to read.
Same key, other CLIs
Once the gateway is set up, the other coding CLIs come free.

Codex CLI (~/.codex/config.toml) — OpenAI protocol, so it's the easy one:

`model_provider = "apiflux"
model = "gpt-5"

[model_providers.apiflux]
name = "ApiFlux"
base_url = "https://apiflux.ai/v1"
env_key = "APIFLUX_API_KEY"`

OpenCode — add a custom OpenAI-compatible provider pointing at https://apiflux.ai/v1 and pick a model ID from the gateway's model list.

Three tools, one key, one invoice.

Cost, since someone will ask
Per 1M tokens, August 2026, gateway price vs. official list:

If you've got Claude Code running through a different gateway, I'd genuinely like to know whether you needed the two beta-disabling variables too, or whether that's specific to Bedrock/Vertex-backed routes. Comments open.

Jason Zhu — building ApiFlux. @ApiFluxAI on X.

How to vet an AI gateway before you route production traffic through it (we run one — here's the checklist we grade ourselves against)

yansen zhu — Mon, 10 Aug 2026 15:16:21 +0000

Routing your LLM traffic through a gateway is handing a stranger two things at once: your prompts and your billing. Discovery is easy — there are dozens of "one API for every model" services now. Knowing which one won't silently swap models, pad invoices, or vanish with your balance is the hard part.

I've written before about vetting agent skills and MCP servers before you trust them. Same energy, bigger blast radius: a bad skill wastes an afternoon; a bad gateway sits in the middle of every production request you make.

Full disclosure up front: we run one of these — ApiFlux. So instead of a neutral survey, this post does something more useful: here are the five checks I'd run against any gateway, and then I run them against our own product in public, including the ones we currently fail.

Why bother: the gateway failure modes nobody puts on the landing page
Every gateway sells the same three lines: one key, lower prices, automatic failover. The actual risks live one layer down:

Markup opacity. "Up to 50% off" against an unstated baseline is not a price. It's a vibe.
Model substitution. You pay for a frontier model; the request quietly lands on something cheaper. Unless you're fingerprinting outputs, you'll never notice — your evals just get mysteriously worse.
Prompt logging. Your traffic is their data unless a policy says otherwise.
Operator risk. Prepaid balance + anonymous operator + no legal entity = the oldest exit scam in the business.
None of these are hypothetical. They're the standard failure modes of an industry where anyone can put an OpenAI-compatible proxy behind a landing page in a weekend.

Check 1 — Demand per-token numbers, not discounts
A trustworthy gateway shows you the exact per-1M-token price per model, per route, with a "last updated" date — so you can diff it against the provider's own list price and see the margin with your own eyes.

Red flags: prices only visible after signup, "credits" with fuzzy exchange rates, discounts quoted against nothing.

Quick test: pick one model you know the official price of, and find it on the gateway's public pricing page in under a minute. If you can't, that's the answer.

Check 2 — Verify the model is what it says
Model substitution is the quiet killer. Three cheap fingerprints, no tooling required:

1. Ask for the model's knowledge cutoff and a fact just inside it.
2. Run a prompt your target model reliably formats a specific way
   (system-prompt adherence differs sharply between model families).
3. Compare token counts: same prompt, gateway vs. official API.
   Tokenizers differ between families — a big mismatch means a swap.
Run them on day one, then randomly. A gateway that welcomes this kind of testing is structurally different from one that gets cagey about it.

Check 3 — Read the logging and data policy like it's code
You're looking for three specific answers, not vibes: Is prompt content stored, or only metadata? For how long? Is anything used for training? "We take privacy seriously" answers none of these. A dated, versioned policy page answers all three.

Check 4 — Failover claims need a visible status history
Everyone advertises 99.9%. The question is whether you can see it: a public status page, per-model uptime history, incident postmortems. If reliability is the pitch, the receipts should be public. "Trust us" is not an SLA.

Check 5 — Know who you're prepaying
Prepaid balance is a loan to the operator. Before you top up, find: the legal entity and jurisdiction, the refund window for unused balance, what happens to your balance if they shut down. If the answer to "who do I even complain to" is a Telegram handle, size your deposit accordingly.

Grading ourselves: where ApiFlux passes and where it doesn't (yet)
The same five checks, run against ApiFlux — verifiable from public pages, not from my word:

Three passes, one partial, one honest fail. I'd rather publish the fail than pretend the checklist only applies to other people — an AI gateway asking for your production traffic should be gradeable by its own standards, in public.

The honest caveat
No checklist catches intent. A gateway can pass all five today and degrade tomorrow; the only durable defense is that these checks are cheap enough to re-run — pricing diff, fingerprint prompts, status page glance — quarterly, like dependency updates. Five minutes per quarter against the service that sits in the middle of every AI request you make is the best ratio in your stack.

If you want to run the pricing check against us right now: apiflux.ai — every price is public, and check 4 should be off the fail list shortly. Hold us to it.

How to vet an MCP server before you install it (I graded 130,000 to find out)

yansen zhu — Fri, 10 Jul 2026 09:04:30 +0000

Installing an MCP server is running a stranger's code with your agent's permissions. Your shell, your environment variables, your filesystem — and increasingly your agent's own config and memory.

Discovery is the easy part; there are tens of thousands of servers to pick from. Knowing whether the one you just found is safe to run is not. I spent the last few months building a rule-based scanner and grading the entire public catalog — 130,000+ open-source agent skills and MCP servers — to figure out what "vetting" actually looks like at scale.

Here's the short version: four manual checks that catch most bad ones, and the data on why you need them.

Why bother: the risk lives exactly where you least expect it

Three numbers from grading the catalog:

83% of the public catalog has never been audited by anyone. No marketplace flag, no review, nothing.
Among the ~21,500 repos popular enough to grade, 3.3% are unsafe or worse — credential harvesting, data exfiltration, curl | sh installers.
The unsafe rate is ~9× higher in the long tail (3.8% at 5–20 stars) than among popular repos (0.4% at 1,000+ stars).

That last one is the trap. The server you've heard of is almost certainly fine. The danger is the obscure 7-star repo you grab from a search for some niche task — which is exactly the moment a directory is supposed to help you, and usually doesn't.

Check 1 — Read the tool definitions, not the README

The README tells you what the author wants you to think the server does. The tool definitions tell you what it can actually do.

Open the source and find where tools are registered (in TypeScript servers, look for server.tool(...) or the tools array; in Python, the @mcp.tool() decorators). Ask one question per tool: does this tool's capability match the server's stated purpose?

A weather server that registers a run_shell_command tool is not a weather server. A "read-only" database inspector that registers execute_query with no statement filtering is not read-only. This check takes five minutes and catches the single most common failure mode: capability creep.

Check 2 — Follow the environment variables

Grep the source for process.env (or os.environ):

grep -rn "process.env\|os.environ" src/ | grep -iv "NODE_ENV\|LOG"

Two things to look for:

What does it read? An API key for the service it wraps: fine. AWS_SECRET_ACCESS_KEY, SSH_AUTH_SOCK, or a loop over all of process.env: walk away.
Where does it send them? Trace every outbound request. If an env var ends up in a request body to a domain that isn't the service the server claims to integrate with, that's exfiltration — I flagged hundreds of these.

Check 3 — Check the install path for `curl | sh`

If the README's install instructions pipe a remote script into your shell, the author is asking you to run unreviewable code before you've even run their reviewable code. Same red flag for postinstall scripts in package.json that fetch remote payloads.

Prefer servers you can run with a plain npx <package> or uvx <package> — the code that executes is at least the code that was published.

Check 4 — Maintenance beats stars

Stars measure marketing. Maintenance measures whether someone will fix the CVE.

Thirty seconds on the repo page: when was the last commit? Do issues get responses? Is there exactly one giant "initial commit"? In the graded data, abandonment correlates with risk far better than star count does — a 40-star repo with weekly commits is a better bet than a 900-star repo untouched for a year.

Or: let the grading be done before you search

The four checks above are what my scanner automates, at catalog scale. Every entry on Agent Skills Hub — all 130,000+ — carries a SAFE / CAUTION / UNSAFE grade plus a quality score, computed from 35 rule-based flags (the SlowMist agent-security taxonomy, extended), refreshed every 8 hours.

From the terminal, the same grades are one command away:

npx @agentskillshub/cli search "browser automation" --safe

It's free, no login, and the full graded dataset is open (CC-BY-4.0) on Hugging Face if you want to run your own analysis or challenge the methodology — it's rule-based, not perfect, and false-positive reports genuinely improve it.

The honest caveat

No static scanner catches intent. A grade of SAFE means "no known bad pattern," not "audited line by line." For anything touching production credentials, do checks 1–2 yourself regardless of what any directory says — including mine.

But the baseline matters. Right now the ecosystem's default is that nobody has looked at 83% of what you can install. Five minutes of vetting — yours or automated — beats that default by a lot.

We security-graded 117,854 AI agent skills. Here's what we found.

yansen zhu — Tue, 23 Jun 2026 01:21:45 +0000

Only 17.7% of the catalog is popular enough to be graded, 1 in 32 graded skills is unsafe, and the risk lives in the long tail — plus a new agent-native attack surface.

We Security-Graded 117,854 AI Agent Skills. Here's What We Found. | Agent Skills Hub

Only 17.7% are popular enough to be graded. Among graded skills, 1 in 32 is unsafe. The risk lives in the long tail.

agentskillshub.top

The uncomfortable part isn't the skills that are unsafe. It's how few have been checked at all.

Installing an AI agent skill or MCP server means handing untrusted code your shell, your environment variables, and increasingly your agent's own config and memory. Discovery is easy — there are tens of thousands to pick from. Knowing whether the one you found is safe to run is not.

So we scanned the whole catalog. Here's the honest picture.

📄 This is a cross-post. Canonical version (with charts): agentskillshub.top/blog/securing-117k-ai-skills

How we scanned

A rule-based scanner, modeled on SlowMist's Agent Security Framework and its 11 red-flag categories. It runs static checks over each skill's README and code, looking for concrete patterns: outbound data exfiltration (curl -d $(...)), credential harvesting (env | grep -i token), reading .env / .ssh / .aws, curl | sh install scripts, privilege escalation, persistence, and secret-exfil combos. Each skill gets a grade — safe / caution / unsafe / reject — plus the specific flags it tripped. Skills with no README or too new to fetch stay unknown.

This is deliberately a first layer: it catches patterns, not intent. At 117K scale, the pattern layer is what makes the catalog auditable at all.

Finding 1 — 82% of the catalog has never been graded

Of 117,854 indexed skills, only 20,853 (17.7%) clear 5 stars — the threshold where a skill is popular enough to be worth grading. The other ~97,000 are effectively unaudited.

"We have 117K skills" is not a feature. The number that matters is how many you can actually trust, and for the long tail the honest answer is: nobody has looked.

Finding 2 — Among graded skills, 1 in 32 is unsafe or worse

Grade	Share
🟢 safe	85.5%
🟡 caution	5.3%
🔴 unsafe	3.0%
⛔ reject	0.1%
⚪ unknown	6.1%

8.4% carry a security concern. 3.1% — about 1 in 32 — are unsafe or reject. At this catalog's size that's ~650 graded skills you genuinely should not run blind, sitting in the same search results as everything else.

Finding 3 — Popularity predicts safety. The risk lives in the long tail.

Stars	Unsafe / reject
5–20★	4.1%
20–100★	3.7%
100–1,000★	0.9%
1,000★+	0.4%

The skill you've heard of is almost certainly fine. The danger is the obscure 7-star repo you'd grab from a search for a niche task — exactly the moment a directory is supposed to help, and usually doesn't.

Finding 4 — The red flags include a new, agent-native attack surface

Most common flags among a sample of 1,000 flagged skills:

Flag	Count
sudo usage	483
background service install	152
curl \| shell	99
agent config theft	87
tunnel service	66
eval()	52
sensitive env vars	34
agent memory theft	23
backdoor install	11

The classic shell risks dominate. But look at agent config theft (87) and agent memory theft (23): skills that read your agent's configuration and memory files. That's not a server exploit — it's a new attack surface that only exists because you're running an agent. Your Claude/MCP config, your stored context, your credentials-by-proxy. The threat model moved, and most directories haven't noticed.

What to do about it

Check the trust signal before you install, from where you already work:

npx @agentskillshub/cli search "postgres mcp" --safe
npx @agentskillshub/cli audit owner/repo

Every result carries its grade and the specific flags it tripped. --safe hides anything unaudited or worse.

The honest caveats (because that's the whole point)

Our 3% is a floor, not a ceiling. Academic deep-analysis (Liu et al., 2026, arXiv:2601.10338) puts the agent-skill vulnerability rate at 26.1%, because they analyze semantics, not just patterns. Our rule-based first pass deliberately under-claims. Read 3% as the lower bound of a bigger problem.
⚪ unknown is not "probably fine." It means no one has checked. 97K of the catalog is unknown. We label it gray and don't dress it up.
All numbers are reproducible. Every grade is visible on the site and via the CLI. Re-derive them yourself.

A trust layer that only told you the good news wouldn't be one. The most useful thing we can say about 97,000 skills is that we don't yet know — and we'll tell you that to your face.

Full writeup with charts: We security-graded 117,854 AI agent skills. Check any skill before you install: npx @agentskillshub/cli audit owner/repo.