DEV Community: Seven

OpenAI cut prices 80%. Here's why I'm not celebrating.

Seven — Fri, 31 Jul 2026 16:23:40 +0000

This week, OpenAI slashed Luna (GPT-5.6) pricing by 80% — from $7 to $1.40 per million tokens. Anthropic shipped Opus 5 at half the price of its predecessor. The LLM API market is in a genuine price war, and for once, developers are winning.

I should be celebrating. I'm not. Here's why.

The hidden math of cheaper models

When official API prices drop, the economics of model-swapping change in a way nobody talks about.

A bad relay has a simple business model: bill you for Model X, serve you Model Y, pocket the difference. When Model X costs $15/M tokens and Model Y costs $0.15/M, the margin is $14.85. That's already attractive.

When Model X drops to $1.40? The absolute margin shrinks — and that's what most people see. But the relative margin — the ratio of what you're charged vs what's actually served — can actually grow if budget models get cheaper still. And they have.

The point isn't the exact math. The point is: price competition between legitimate providers doesn't eliminate the incentive to swap models. It often masks it. When everyone's prices are dropping, a relay that's 30% cheaper than official looks like aggressive discounting, not fraud.

"What model are you?" doesn't work

If you're still asking an API endpoint to identify itself, stop. That's a system prompt — trivially spoofed. Every relay operator knows to make the model say "I am Claude, created by Anthropic."

What works: behavioral fingerprinting at temp=0.

Give the model a fixed set of tasks that different models handle differently — reasoning problems, long-context recall, strict JSON formatting, refusal boundaries. Run them at temperature 0 to remove sampling noise. Diff the results against the official API.

This isn't theoretical. Here's the actual tool:

git clone https://github.com/seven7763/llm-honesty-probe
cd llm-honesty-probe
python3 -m llm_honesty_probe --self-test --card

That runs a battery of deterministic probes and produces a verdict card: PASS, SUSPICIOUS, or DEGRADED — with the specific probes that triggered each finding. No API key leaves your machine. No judgment calls required. The self-test runs without any API key and takes about 30 seconds.

To test your actual endpoint:

export LLM_ENDPOINT="https://your-endpoint/v1"
export LLM_API_KEY="sk-your-key"
python3 -m llm_honesty_probe --compare

Why this matters more now, not less

The price war is genuinely good news. Lower costs mean more experiments, more products, more people who can afford to build with LLMs. I want all of that.

But cheaper official APIs also mean more users signing up for the first time — users who don't know what "Claude" is supposed to sound like, what GPT-5.6's reasoning depth should feel like, or what latency to expect. These are exactly the users a model-swapping relay wants: no baseline for comparison.

The defense isn't trust. It's verification that costs less than being wrong.

Test your endpoint right now

If you're using any third-party API endpoint — mine included — run the probe. The self-test takes 30 seconds. The compare against your endpoint takes about 2 minutes.

If everything passes, you've got evidence — not proof, but signals — that you're getting what you pay for. If something flags, you know exactly which probe to investigate.

Either way, you know more than you did before. That's the whole point.

Disclosure: I work on daoxe, an OpenAI-compatible LLM gateway. I also maintain llm-honesty-probe, the open-source tool mentioned in this post. The probe treats every endpoint identically — including daoxe. Point it at anyone. If it flags something on my own service, I want to know before you do.

The price war is real. The trust problem is real. They're related, and fixing one doesn't automatically fix the other.

Deploy a real AI chat app in one click — key stays server-side (Vercel/Render/Railway)

Seven — Tue, 21 Jul 2026 14:40:49 +0000

Most "deploy an AI app in 5 minutes" tutorials quietly do the one thing you must never do: put an API key where a browser can see it. This post is three one-click starters that are correct by construction — the key is a deploy-time environment variable, server-side only, never in client code, never committed to git.

Each template is provider-neutral: it defaults to an OpenAI-compatible endpoint (https://daoxe.com/v1) but points at any compatible endpoint by changing one variable. Pick the one that matches what you're building.

Disclosure: I work on daoxe, the OpenAI-compatible gateway these default to. Everything below works against any compatible endpoint — the default and the "powered by" link are one-line changes.

The rule that makes these safe

An LLM API key is a billing credential. Anything shipped to a browser is public. So every template here follows the same discipline:

DAOXE_API_KEY (and any tokens) are deploy-time env vars, declared in each platform's config with no baked-in value.
.env.example files ship empty placeholders.
The key is used server-side only — an Edge function, a proxy server, or a bot process holds it; the browser never sees it.

If a tutorial ever tells you to paste your key into front-end JavaScript, close the tab.

Template 1 — Streaming chat UI on Vercel

A minimal chat UI backed by an Edge function that holds the key and streams tokens back to the browser. Good for: a demo, an internal tool, a landing-page assistant.

Repo: seven7763/daoxe-vercel-chat

Deploy: click the Vercel button in the repo README; Vercel prompts you for two env vars:

DAOXE_API_KEY — your key.
DAOXE_MODEL — an exact id from GET /v1/models.

That's it — you get a URL with a working streaming chat. The browser calls your Edge function; the Edge function calls the gateway with the key. The key never leaves the server.

Browser ──▶ /api/chat (Edge, holds key) ──▶ https://daoxe.com/v1/chat/completions
        ◀── streamed tokens ◀──────────────

Optional DAOXE_ALLOWED_MODELS lets you restrict which models the function will serve.

Template 2 — OpenAI-compatible proxy on Render

Sometimes you don't want a UI — you want to share one key with a team or a script without handing out the real credential. This template is a tiny OpenAI-compatible proxy: it exposes /v1/chat/completions, injects the real key server-side, and gates access with a token you hand out instead.

Repo: seven7763/daoxe-render-proxy

Deploy: click the Render button in the repo README; set DAOXE_API_KEY (the real key) and a share token. Now teammates point their OpenAI-compatible tools at your proxy URL with the share token — they get model access, never your billing credential. It also serves a live /healthz page.

Template 3 — Discord bot on Railway

A Discord bot that answers /ask with an LLM. Good for: team assistants, support bots, community helpers.

Repo: seven7763/daoxe-railway-discord-bot

Deploy: from the public repo, open Railway → New Project → Deploy from GitHub (or create a Template once), then set DAOXE_API_KEY, DAOXE_MODEL, and your Discord bot token. The bot process holds the key; Discord users just type /ask. (This one has a single runtime dependency, discord.js.)

Env vars cheat sheet

Variable	Where	Notes
`DAOXE_API_KEY`	all three	Billing credential. Server-side only.
`DAOXE_MODEL`	vercel-chat, railway bot	Exact id from `GET /v1/models`.
`DAOXE_ALLOWED_MODELS`	vercel-chat (optional)	Comma-separated allowlist.
Share / Discord tokens	render proxy / railway bot	Separate from the billing key.

(Repos are public. Vercel/Render one-click buttons live in each README; Railway is "Deploy from GitHub" from the public repo — no private template id required.)

What this buys you

Three one-click AI starters: Vercel (streaming chat), Render (OpenAI-compatible proxy), Railway (Discord bot).
Key never in the browser — the failure mode most tutorials ship by accident.
One key → many models behind a single OpenAI-compatible base URL, with fallback and one bill if you want it.

Don't trust the pitch — deploy one template, open the network tab, and confirm the key is not in any client request. Then point the same templates at any OpenAI-compatible endpoint and compare.

One key, two ecosystems: routing between DeepSeek/Kimi/Qwen and Claude/GPT/Gemini by task

Seven — Sun, 19 Jul 2026 22:40:57 +0000

Picking a single model family for an app is a false choice. Some Chinese-origin models are strong and cost-efficient at certain tasks; some Western models lead at others. If you're building outside mainland China, the practical move isn't to bet on one — it's to route each request to the right one, and make switching a one-line change instead of a new integration.

This guide shows how to do that through a single OpenAI-compatible endpoint, so DeepSeek, Kimi, Qwen, Claude, GPT and Gemini all sit behind one key and one bill — and how to choose between them per task without guessing.

Disclosure & scope: I work on daoxe, an OpenAI-compatible gateway. It is not available in mainland China — this article is written for developers outside it (overseas devs, overseas Chinese, and users in Russia, Brazil, Vietnam, Turkey and elsewhere) who want access to both ecosystems in one place. Everything here is standard OpenAI-compatible config; point it at any endpoint that carries the models you want.

Why route at all

Three reasons a single-model app leaves value on the table:

Task fit. Models have uneven strengths — reasoning, long-context, code, multilingual, structured output, vision. The best model for summarizing a 200-page PDF isn't necessarily the best for a tricky refactor.
Cost. For high-volume, low-stakes calls (classification, extraction, routing itself), a cheaper model is often good enough, and the savings compound. Reserve the expensive model for the calls that need it.
Resilience. If one provider is rate-limited or degraded, being able to fail over to another family keeps you shipping.

The blocker has always been integration cost: different SDKs, keys, billing, and response quirks per vendor. An OpenAI-compatible endpoint removes it — model choice becomes a string.

The mechanism: model choice is a string

Every call uses the same shape; only the model field changes:

curl https://daoxe.com/v1/chat/completions \
  -H "Authorization: Bearer $DAOXE_API_KEY" -H "Content-Type: application/json" \
  -d '{"model":"<MODEL_ID>","messages":[{"role":"user","content":"..."}]}'

List what your key can actually call — the authoritative, account-scoped source is:

curl https://daoxe.com/v1/models -H "Authorization: Bearer $DAOXE_API_KEY"

Use those exact ids. Families you'll typically see include Chinese-origin ones (DeepSeek, Kimi, Qwen, and others) and Western ones (Claude, GPT, Gemini). Don't hardcode ids from this post — they drift; read them from /v1/models.

A simple task-based router (Python)

You don't need a framework. A dict from task → model id gets you 90% of the value:

import os, httpx

BASE = "https://daoxe.com/v1"
KEY = os.environ["DAOXE_API_KEY"]

# Map task types to EXACT ids from GET /v1/models (examples — replace with yours).
ROUTES = {
    "cheap_bulk":   "deepseek-...",     # high-volume extraction/classification
    "long_context": "kimi-k2.5",        # very long inputs
    "reasoning":    "claude-sonnet-4-6",# hard multi-step problems
    "code":         "gpt-5.5",          # tricky refactors
    "default":      "claude-haiku-...", # fast, inexpensive fallback
}

def ask(task: str, prompt: str) -> str:
    model = ROUTES.get(task, ROUTES["default"])
    r = httpx.post(f"{BASE}/chat/completions",
        headers={"Authorization": f"Bearer {KEY}"},
        json={"model": model, "temperature": 0,
              "messages": [{"role": "user", "content": prompt}]}, timeout=60)
    r.raise_for_status()
    return r.json()["choices"][0]["message"]["content"]

print(ask("cheap_bulk", "Extract the invoice total as JSON."))
print(ask("reasoning",  "Prove or disprove: ..."))

Because it's one endpoint and one key, adding a route is adding a dict entry — no new SDK, no new credential. Add a try/except that falls back to another family on error and you've got cheap resilience too.

How to choose the route (without vibes)

Don't route on reputation — route on evidence for your workload:

Build a tiny eval set of real prompts per task type with known-good answers.
Run each candidate model through it, and record accuracy alongside latency (p50/p95) and token cost. (A small harness does this; I wrote up a reproducible gateway benchmark separately.)
Pick the cheapest model that clears your quality bar for each task — not the most expensive one you can afford.
Re-check periodically. New model versions ship constantly; a route that was optimal last quarter may not be now.

On price specifically: I'm not going to quote numbers here because they change. Check the provider's pricing page and put real $/1M figures next to your latency and accuracy — the right route is a three-way trade-off, not a single-axis choice.

Don't forget to verify what you're getting

Routing across families makes verification more important, not less: with several models behind one endpoint, you want to confirm each id actually serves what it claims and isn't being silently swapped or truncated. Test behavior, not the model's self-report — a tokenizer fingerprint, a capability-floor check, and a long-context needle test catch the common cheats. (There are open-source probes for this; I covered the method in a separate post.) A good endpoint is fine with you running these against it.

Honest positioning

A few things to be straight about:

This is for outside mainland China. daoxe doesn't serve mainland China; the audience here is overseas developers and users in regions where getting both ecosystems in one place — and paying for it — is otherwise fiddly.
"Chinese vs Western" is about model origin, not a quality ranking. Route on measured task fit and cost, not on where a model came from.
One key is a convenience, not magic. It removes integration and billing friction; it doesn't make a small model into a big one. Match the model to the task.

daoxe is the endpoint I use for this — OpenAI-compatible at https://daoxe.com/v1, native Anthropic Messages for Claude-native tools, one key across many models (Chinese-origin and Western), one bill, and a design meant to be verified. See the pricing page for current models and prices; use GET /v1/models for what your key can call.

TL;DR

Don't pick one family — route each task to the best/cheapest model.
One OpenAI-compatible endpoint makes model choice a string: one key, one bill, no per-vendor SDKs.
Choose routes on a small eval set (accuracy + p50/p95 + $/1M), not reputation; re-check as versions ship.
Verify each model behaviorally, and remember this is for use outside mainland China.

Add an AI chat widget to any website — without leaking your API key

Seven — Sun, 19 Jul 2026 22:31:19 +0000

Adding an AI chat box to a website is a five-minute job — and most five-minute tutorials get the one important thing wrong: they put an API key in the front-end. This walkthrough uses a tiny, dependency-free widget that refuses to let you do that, and shows the two correct ways to handle the key depending on whether the page is yours or public.

The widget is ~11 KB of vanilla JS — no build step, no framework — and it talks to any OpenAI-compatible Chat Completions endpoint. I'll use daoxe as the default (disclosure below), but it's one line to point it anywhere.

Disclosure: I work on daoxe, the widget's default endpoint. Nothing here is locked to it — change one line and it talks to any compatible endpoint.

The uncomfortable truth: anything in the browser is public

An LLM API key is a billing credential. Anything you ship to a browser — including a key in your JS bundle — is readable by every visitor. So the widget deliberately has no apiKey option in its config. Instead you pick one of two modes:

Mode	Who holds the key	Use it for	Trade-off
`byok` (default)	the visitor pastes their own key; kept only in their browser	a personal/local page only you use; dashboards; internal tools	needs the endpoint to allow browser CORS
`proxy` (recommended for public sites)	your server injects the key; the browser never sees it	any public website	you run a tiny proxy (examples included)

That single design choice is the whole point. If a widget lets you drop your own key into a public page's source, it's shipping your credential to the world.

A) BYOK mode — for a page that is yours

For a personal dashboard or an internal tool, the visitor supplies their own key, stored only in their browser's localStorage and sent straight to the endpoint:

<script src="daoxe-widget.js"></script>
<script>
  DaoxeChat.init({
    baseUrl: "https://daoxe.com/v1",
    model: "YOUR_DAOXE_MODEL_ID",   // exact id from GET /v1/models
    title: "Ask me anything"
  });
</script>

This only works if the endpoint returns permissive CORS headers (many gateways don't enable CORS on /chat/completions — that's what proxy mode is for). If you see a "network/CORS" error, switch to proxy.

B) Proxy mode — for public sites

For anything public, the browser talks to your route, and your server adds the key:

<script src="daoxe-widget.js"></script>
<script>
  DaoxeChat.init({
    mode: "proxy",
    proxyUrl: "/api/chat",          // a route you control that adds the key
    model: "YOUR_DAOXE_MODEL_ID",
    title: "Ask us anything"
  });
</script>

Ready-to-run proxies ship in examples/:

Cloudflare Worker — deploy in minutes; key stored as a Worker secret.
Node/Express — reads DAOXE_API_KEY from the environment and streams straight through.

Both keep the key server-side and can optionally allow-list which models the proxy will forward. In production, restrict Access-Control-Allow-Origin to your domain.

Visitor ──▶ /api/chat (your proxy, holds key) ──▶ https://daoxe.com/v1/chat/completions
        ◀── streamed tokens ◀───────────────────

Security details worth knowing

A chat widget renders untrusted model output into your page, so two things matter beyond the key:

The widget escapes all model output before rendering — it doesn't innerHTML raw model text. A tiny markdown-ish renderer handles code/bold/newlines on already-escaped text, and it does not execute HTML returned by the model. (If you build your own widget, do this — model output is untrusted input.)
Lock down the proxy. Restrict CORS to your domain and optionally allow-list models so a leaked proxy URL can't be used to run arbitrary expensive models on your bill.

Configuration highlights

DaoxeChat.init({ ... }) takes the usual knobs — title, subtitle, greeting, accent, position, startOpen, systemPrompt, and (BYOK only) temperature / maxTokens. Two worth calling out:

model is required — an exact id available to the key/account. Don't hardcode a list from a blog post; the authoritative list is GET {baseUrl}/models.
storageKey controls the localStorage key for BYOK; set it to null to never persist the visitor's key.

The widget is minimal by design: no file uploads, no image input, no tool calls, no cross-page persistence. Streaming uses the standard OpenAI SSE format and falls back to a single JSON read if an endpoint doesn't stream.

Why an OpenAI-compatible endpoint helps here

Because the widget speaks the standard Chat Completions shape, the same embed can serve GPT, Claude, Gemini or DeepSeek by changing a model id — one key, one bill, and you can repoint baseUrl to any provider you trust and verify. daoxe is the default (https://daoxe.com/v1, one key across many models, native Anthropic Messages too) and is not available in mainland China — but the widget privileges nothing; baseUrl and the "Powered by" link are one-line changes.

TL;DR

One <script> tag, ~11 KB, no build, any OpenAI-compatible endpoint.
Never put your key in front-end code. Use BYOK (your pages) or proxy (public sites).
Proxy examples included (Cloudflare Worker, Node/Express); lock CORS to your domain in prod.
Escape model output; require an exact model id from /v1/models.

One API key, three terminals: driving many models from the shell with llm, mods and aichat

Seven — Sun, 19 Jul 2026 22:24:18 +0000

The fastest way to use an LLM is often not a chat window — it's a pipe. git diff | ... "write a commit message", ... "explain this error" < stderr.log, cat main.py | ... "find the bug". Three of the best command-line tools for this — llm, mods, and aichat — all accept a custom OpenAI-compatible endpoint, which means one key can back all three and give you Claude, GPT, Gemini and more from the shell.

This is the terminal setup, end to end. It's genuinely tool-agnostic: every config below works against any OpenAI-compatible endpoint. I use daoxe as the example endpoint (disclosure below); swap the base URL for your own.

Disclosure: I work on daoxe, an OpenAI-compatible gateway (it also speaks Anthropic Messages natively). Everything here is standard OpenAI-compatible config — point these tools at whatever endpoint you trust.

One convention across all three: the key lives in an environment variable, DAOXE_API_KEY, never on the command line (so it doesn't land in your shell history). And model ids are account-scoped — list yours with curl https://daoxe.com/v1/models -H "Authorization: Bearer $DAOXE_API_KEY".

1. `llm` (Simon Willison) — the extensible one

llm is the most extensible of the three: a pluggy-based plugin system, a local SQLite log of everything, templates, embeddings. There are two ways to add an OpenAI-compatible gateway.

Option A — zero-YAML via a plugin

There's a small plugin, llm-daoxe, that registers the gateway's models and adds a command to pull your account's real model list:

llm install llm-daoxe           # (coming soon on PyPI; until then: clone + `pip install -e .`)
llm keys set daoxe              # store the key (or export DAOXE_API_KEY=...)
llm daoxe models --refresh     # pull your account's real models from GET /v1/models, and cache them
llm -m daoxe/claude-sonnet-4-6 "In one line, what's an OpenAI-compatible gateway?"
git diff | llm -m daoxe/gpt-5.5 -s "Write a concise commit message."

Model ids are namespaced as daoxe/<model> so they don't collide with other plugins' ids. The key is read only from llm's key store or DAOXE_API_KEY, and it's never printed.

Option B — no plugin, native config

Don't want a plugin? llm natively reads custom OpenAI-compatible models from extra-openai-models.yaml in its config dir:

- model_id: daoxe/claude-sonnet-4-6
  model_name: claude-sonnet-4-6
  api_base: https://daoxe.com/v1
  api_key_name: daoxe          # matches `llm keys set daoxe`
  can_stream: true
- model_id: daoxe/gpt-5.5
  model_name: gpt-5.5
  api_base: https://daoxe.com/v1
  api_key_name: daoxe

The plugin's advantage is just convenience: zero YAML plus llm daoxe models --refresh to auto-discover your account's models. Both paths use llm's built-in OpenAI transport, so streaming and tools behave the same as with official models.

2. `mods` (charmbracelet) — the pipe-native one

mods is built for Unix pipes and looks great doing it. Open its config with mods --settings and add a provider under apis::

default-api: daoxe            # optional: make it the default so you can skip -a
apis:
  daoxe:
    base-url: https://daoxe.com/v1
    api-key-env: DAOXE_API_KEY
    models:                  # use real ids from GET /v1/models; max-input-chars is a client-side value
      claude-sonnet-4-6:
        aliases: ["daoxe-sonnet"]
        max-input-chars: 600000
      gpt-5.5:
        aliases: ["daoxe-gpt"]
        max-input-chars: 600000

export DAOXE_API_KEY=your-daoxe-key
mods -a daoxe -m claude-sonnet-4-6 "explain this diff" < patch.diff
mods --model daoxe-sonnet "summarize" < README.md      # via alias

Gotcha: mods requires api-key-env to point at an environment variable that already exists, or it complains about a missing key. max-input-chars is a client-side truncation setting — tune it to your model, it's not a claim about the gateway.

3. `aichat` (sigoden) — the all-in-one one

aichat is a single binary that's a CLI, a REPL, and a local server. It uses an openai-compatible client type. Edit ~/.config/aichat/config.yaml:

clients:
  - type: openai-compatible
    name: daoxe
    api_base: https://daoxe.com/v1
    # omit api_key and aichat auto-reads DAOXE_API_KEY (client name → {NAME}_API_KEY)
    models:                     # real ids from GET /v1/models; tune capabilities per model
      - name: claude-sonnet-4-6
        max_input_tokens: 200000
        supports_function_calling: true
      - name: gpt-5.5
        max_input_tokens: 272000
        supports_function_calling: true

export DAOXE_API_KEY=your-daoxe-key
aichat -m daoxe:claude-sonnet-4-6 "explain this code" < main.py
aichat --list-models | grep daoxe

Nice touch: because the client is named daoxe, aichat automatically looks for DAOXE_API_KEY — you can omit the api_key field entirely. Model selection is client:model, e.g. daoxe:claude-sonnet-4-6.

Which one should you use?

Tool	Best for	Superpower
`llm`	scripting, logging, plugins	SQLite log of every call; huge plugin ecosystem
`mods`	quick pipes	gorgeous TUI output, alias/fallback config
`aichat`	interactive + server	REPL, roles, and a built-in local API server

They're not mutually exclusive — I keep all three configured against the same key. The same mental model applies to other CLIs too: anything that takes an OpenAI-compatible base_url + key + model id works the same way (point the base URL at the /v1 root, use a real id from /v1/models, keep the key in an env var). Verify each tool's current config fields before you rely on them — these tools move fast.

The reason one key matters here

CLI users tend to script things and forget them: a cron job that summarizes logs, a git hook that drafts commit messages, a shell function that explains errors. When each of those needs a different vendor key, it's a mess of secrets and bills. One OpenAI-compatible key across llm, mods, and aichat — and across every script that calls them — means one secret to rotate, one bill to read, and model choice as a string.

daoxe is the endpoint I point them at: https://daoxe.com/v1, one key across many models, plus native Anthropic Messages if a tool wants Claude's protocol. It's not available in mainland China. As always, none of the config above is daoxe-specific — and because it's a standard endpoint, you can benchmark and verify it rather than trust it.

TL;DR

llm, mods, and aichat all take a custom OpenAI-compatible endpoint — one key backs all three.
llm: a plugin (llm-daoxe, coming soon) or extra-openai-models.yaml.
mods: an apis: block; api-key-env must point at an existing env var.
aichat: an openai-compatible client; name it daoxe and it auto-reads DAOXE_API_KEY.
Keep the key in an env var, use exact ids from /v1/models, and pipe away.

Bring your own key to GitHub Copilot: Chat, CLI, App and SDK (with the honest caveat)

Seven — Sun, 19 Jul 2026 22:13:54 +0000

Most people don't know Copilot can do this yet: as of 2026, GitHub Copilot has GA'd Bring Your Own Key (BYOK). You can point Copilot Chat (in VS Code), the Copilot CLI, the Copilot app, and the Copilot SDK at any OpenAI-compatible endpoint, Anthropic, or Azure — including local runtimes and third-party gateways.

That's a big deal if you already pay for Copilot but want to run its chat and agent features on a model or a gateway of your choice. This guide covers all four surfaces, with the exact steps and one honest caveat that trips people up.

Disclosure: I work on daoxe, an OpenAI-compatible gateway that also speaks the Anthropic Messages protocol. It happens to line up neatly with Copilot's three wire protocols, so I use it as the example — but everything here works with any compatible endpoint. Substitute your own base URL.

The caveat, first (so nobody's misled)

BYOK affects Copilot Chat and agent sessions. It does not change inline completions. Those gray "Tab to accept" suggestions still run on Copilot's own infrastructure and are unaffected by your base URL. If your goal is to change what powers chat/agent, BYOK is for you. If you wanted to swap the model behind inline completions, BYOK won't do it. That's the honest scope.

Why a gateway fits Copilot's three protocols

Copilot's BYOK supports three provider "types", and a good multi-protocol gateway hits all three:

openai → the gateway's /v1/chat/completions (default).
openai with wireApi: "responses" → the gateway's /v1/responses.
anthropic → the gateway's /v1/messages (Claude's native protocol).

So one key can serve GPT-shaped, Responses-shaped, and Claude-native requests, depending on which model you pick. Now the four ways to wire it.

1. Copilot CLI (easiest — just env vars)

export COPILOT_PROVIDER_BASE_URL=https://daoxe.com/v1
export COPILOT_PROVIDER_API_KEY=$DAOXE_API_KEY     # your gateway key, from an env var
export COPILOT_MODEL=gpt-5.5                        # any exact id from GET /v1/models
# optional: COPILOT_PROVIDER_TYPE=openai (default) | anthropic | azure
copilot

For Claude's native protocol, set COPILOT_PROVIDER_TYPE=anthropic and use the gateway's Anthropic entry point. Confirm the exact path with curl first — many gateways take the host root (https://daoxe.com) and let the client append /v1/messages; some want /v1. Don't assume; check your account.

2. Copilot Chat (VS Code)

Command Palette → Chat: Manage Language Models.
Choose OpenAI Compatible / Custom Endpoint.
Base URL: https://daoxe.com/v1 (it must serve /chat/completions). VS Code probes GET /models to fill the model dropdown — if your gateway supports it, models auto-populate; otherwise type the Model ID by hand.
Paste your API Key, pick a Model ID (e.g. claude-sonnet-4-6) → Add Model.

The model now shows up in the Copilot Chat model picker alongside the built-ins.

3. Copilot App (desktop)

Settings → Model Providers → Add provider. Fill in the endpoint https://daoxe.com/v1 and your API key (or choose the anthropic type for Claude-native). Your models then appear in the model picker next to Copilot's hosted ones; the key is stored in your OS keychain.

4. Copilot SDK (for integrators)

provider: {
  type: "openai",                    // or "anthropic"
  baseUrl: "https://daoxe.com/v1",   // full path; for anthropic, use your account's path
  apiKey: process.env.DAOXE_API_KEY,
  wireApi: "completions",            // use "responses" when you need Responses-API behavior
}

Getting the model ids right

Copilot only calls what you tell it to, so use the exact ids your endpoint exposes:

curl https://daoxe.com/v1/models -H "Authorization: Bearer $DAOXE_API_KEY"

Guessed names fail silently or fall back. A gateway with an account-scoped /v1/models is ideal here, because the list you see is exactly the list you can call.

Troubleshooting

Model dropdown empty in VS Code? Your endpoint may not expose GET /models, or the key lacks access. Type the id manually and confirm the key with the curl above.
401 / auth errors? Check whether the provider expects Authorization: Bearer (OpenAI type) vs x-api-key (some Anthropic setups). Match the provider type to the protocol.
Anthropic type 404s? It's almost always the base URL shape — root vs /v1. Verify against your gateway's docs.
Chat works but inline completions "didn't change"? Expected — see the caveat. Inline stays on Copilot's models.

What to look for in a BYOK target

Since you're choosing the backend now, pick one that is:

Multi-protocol. OpenAI Chat Completions and Responses and Anthropic Messages, so all of Copilot's provider types work and Claude runs on its native protocol.
One key, many models, with an account-scoped /v1/models.
Verifiable. Fine with you benchmarking it and checking it isn't silently downgrading you (there are open-source tools for exactly this).
Reachable where official billing isn't. If your card keeps getting declined at an official checkout, a gateway that accepts other payment paths is the difference between shipping and not.

daoxe checks those boxes — OpenAI-compatible base URL (https://daoxe.com/v1), native Anthropic Messages, one key across many models — and it's not available in mainland China. But BYOK is the point: you're not locked in, so point Copilot at whatever endpoint you trust and can verify.

TL;DR

Copilot BYOK works in Chat, CLI, App, and SDK — set a base URL + key.
A multi-protocol gateway covers all three Copilot provider types (openai / responses / anthropic).
Use exact model ids from /v1/models.
Inline completions don't change — BYOK is for chat/agent only.

One key for CrewAI, AutoGen, LlamaIndex and 8 more: the base_url trick for Python agents

Seven — Sun, 19 Jul 2026 22:11:15 +0000

Here's a fact that quietly makes multi-model agent development a lot less painful: in 2026, virtually every Python agent framework natively supports pointing its underlying LLM at a custom OpenAI-compatible base_url. No new package, no fork, no framework change. You set one URL and one key, and the framework talks to whatever backend you choose.

That means a single OpenAI-compatible gateway — one key that fronts Claude, GPT, Gemini, DeepSeek, Kimi and more, with fallback and one bill — is a natural backend for all of these frameworks at once. The orchestration stays with the framework; the models come from one endpoint.

This post is the map: the exact one-liner for each framework, plus the one honest gotcha each has, because those gotchas are what actually cost you an afternoon.

Disclosure: I work on daoxe, one such gateway. But nothing below is daoxe-specific — every snippet works against any OpenAI-compatible endpoint. Swap the base URL for your own; the frameworks don't care who's behind it.

Why this works at all

Almost every framework builds on the OpenAI Chat Completions shape:

POST {base_url}/chat/completions
Authorization: Bearer <key>

The only two things that route this call to OpenAI are the base URL and the key. Change the base URL to a compatible gateway and the same orchestration code now runs against a different backend. Frameworks expose this under slightly different names — base_url, api_base, api_base_url — but it's the same idea.

Two universal rules before the snippets:

Model ids are account-scoped. Don't copy an id from a blog post. List yours with curl {base_url}/models -H "Authorization: Bearer $KEY" and use an exact id.
Keys go in env vars, never hardcoded. Every snippet below reads from the environment.

The 11 frameworks, grouped by mechanism

A) Frameworks with a direct `base_url` parameter

LangChain (+ LangGraph) — the biggest ecosystem, so start here:

from langchain_openai import ChatOpenAI
llm = ChatOpenAI(model="MODEL_ID", base_url="https://daoxe.com/v1", api_key="...")  # key via env
print(llm.invoke("ping").content)

# LangGraph: feed that llm straight into a prebuilt agent
from langgraph.prebuilt import create_react_agent
agent = create_react_agent(llm, tools=[...])

Gotcha: ChatOpenAI targets the official OpenAI spec; non-standard fields some proxies add aren't preserved. Fine for standard responses.

AutoGen (Microsoft AgentChat):

from autogen_agentchat.agents import AssistantAgent
from autogen_core.models import ModelInfo
from autogen_ext.models.openai import OpenAIChatCompletionClient

client = OpenAIChatCompletionClient(
    model="MODEL_ID", base_url="https://daoxe.com/v1", api_key="...",
    model_info=ModelInfo(vision=False, function_calling=True, json_output=True,
                         family="unknown", structured_output=True),
)
agent = AssistantAgent("assistant", model_client=client)

Gotcha: a non-OpenAI model id needs both base_url and model_info (capability flags). Omit model_info and it errors. Set the flags to match the model behind your id.

smolagents (Hugging Face):

from smolagents import CodeAgent, OpenAIModel
model = OpenAIModel(model_id="MODEL_ID", api_base="https://daoxe.com/v1", api_key="...")
print(CodeAgent(tools=[], model=model).run("ping"))

Gotcha: pre-1.9 the class was OpenAIServerModel (same params). Uses api_base, not base_url.

Haystack (deepset):

from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
from haystack.utils import Secret

gen = OpenAIChatGenerator(model="MODEL_ID", api_base_url="https://daoxe.com/v1",
                          api_key=Secret.from_env_var("DAOXE_API_KEY"))
print(gen.run([ChatMessage.from_user("ping")])["replies"][0].text)

Gotcha: the param is api_base_url (note the extra word), and the key goes through Secret.from_env_var(...).

Agno (formerly phidata):

from agno.agent import Agent
from agno.models.openai.like import OpenAILike
Agent(model=OpenAILike(id="MODEL_ID", base_url="https://daoxe.com/v1", api_key="...")).print_response("ping")

Gotcha: use OpenAILike (from agno.models.openai.like), not the plain OpenAIChat, for a third-party endpoint.

B) Frameworks with an explicit provider object

PydanticAI — type-safe agents:

from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.openai import OpenAIProvider

model = OpenAIChatModel("MODEL_ID",
    provider=OpenAIProvider(base_url="https://daoxe.com/v1", api_key="..."))
agent = Agent(model)
print(agent.run_sync("ping").output)

Gotcha: build a generic OpenAIProvider and pass it to OpenAIChatModel — this is the same pattern the docs use for any OpenAI-compatible provider.

LlamaIndex:

from llama_index.llms.openai_like import OpenAILike  # pip install llama-index-llms-openai-like

llm = OpenAILike(model="MODEL_ID", api_base="https://daoxe.com/v1", api_key="...",
                 is_chat_model=True, is_function_calling_model=True, context_window=128000)
print(llm.complete("ping"))

Gotcha: set is_chat_model=True or it calls the /completions endpoint instead of /chat/completions. Also set a real context_window.

C) Frameworks that route through LiteLLM (need the `openai/` prefix)

CrewAI:

from crewai import LLM, Agent, Crew, Task
llm = LLM(model="openai/MODEL_ID", base_url="https://daoxe.com/v1", api_key="...")
agent = Agent(role="Greeter", goal="Greet briefly.", backstory="Concise.", llm=llm)
task = Task(description="ping", expected_output="A short greeting.", agent=agent)
print(Crew(agents=[agent], tasks=[task]).kickoff())

Gotcha: keep the openai/ prefix on the model id. Without it, CrewAI (via LiteLLM) may match a native provider client for a familiar model name and ignore base_url — you'll get a misleading "API key not valid". Use the API root (.../v1), not .../chat/completions.

DSPy:

import dspy
lm = dspy.LM("openai/MODEL_ID", api_base="https://daoxe.com/v1", api_key="...")
dspy.configure(lm=lm)
print(dspy.Predict("question -> answer")(question="ping").answer)

Gotcha: same openai/ prefix, and keep /v1 in api_base with no trailing slash — a trailing slash can double the path and 404.

D) Wrap the OpenAI client directly

Instructor — structured output validated into Pydantic models:

import instructor
from openai import OpenAI
from pydantic import BaseModel

class Out(BaseModel):
    text: str

client = instructor.from_openai(OpenAI(base_url="https://daoxe.com/v1", api_key="..."))
print(client.chat.completions.create(model="MODEL_ID", response_model=Out,
        messages=[{"role": "user", "content": "ping"}]))

Gotcha: the default (tool-calling) mode works for tool-capable models; if a model lacks tool calling, pass mode=instructor.Mode.JSON.

The gotchas, in one table

Framework	Param name	The one gotcha
LangChain	`base_url`	non-standard proxy fields not preserved
AutoGen	`base_url`	non-OpenAI id needs `model_info` too
smolagents	`api_base`	class renamed from `OpenAIServerModel`
Haystack	`api_base_url`	key via `Secret.from_env_var`
Agno	`base_url`	use `OpenAILike`, not `OpenAIChat`
PydanticAI	`OpenAIProvider(base_url=)`	generic provider, not a vendor class
LlamaIndex	`api_base`	`is_chat_model=True` or it hits `/completions`
CrewAI	`base_url`	model must be `openai/<id>`
DSPy	`api_base`	`openai/` prefix + no trailing slash
Instructor	wrap `OpenAI(base_url=)`	use `Mode.JSON` if no tool calling

Why one endpoint beats juggling keys

Multi-agent orchestration is where model sprawl hurts most: a planner on a strong model, workers on a cheap one, a critic on a third — that's three vendors, three keys, three bills, three failure modes. Pointing all of them at one OpenAI-compatible endpoint means:

One key, one bill across every framework and every agent role.
Model choice is a string, not a new integration. Swap MODEL_ID and the same agent runs on Claude, GPT, or Gemini.
You're not locked to a framework. The same key runs the same task across all 11.

And because it's a standard endpoint, you can verify it rather than trust it — run a probe against it and against the official API and compare (I wrote a separate piece on catching silent model swaps).

daoxe is the endpoint I use for this: OpenAI-compatible at https://daoxe.com/v1, plus native Anthropic Messages for Claude-native tools, one key across many models. It's not available in mainland China. But the whole point of this article is that none of the code above is daoxe-specific — point it at any compatible endpoint you trust.

TL;DR

Every framework here takes a custom OpenAI-compatible endpoint natively — it's a one-liner.
Watch the param name (base_url vs api_base vs api_base_url) and the per-framework gotcha.
CrewAI/DSPy need the openai/ prefix; AutoGen needs model_info; LlamaIndex needs is_chat_model=True.
One key, one bill, model choice as a string — and you can verify the backend instead of trusting it.

Your cheap LLM relay might be swapping the model. Here's how to catch it.

Seven — Sun, 19 Jul 2026 22:07:09 +0000

If you buy model access through a cheap "GPT / Claude / DeepSeek" API relay, you have a trust problem nobody puts on the pricing page: some relays quietly swap in a smaller, quantized, or context-truncated model and bill you for the flagship. The nasty part is the timing — it often works fine on day one and degrades weeks later, right after you've stopped checking.

This article is the verification playbook I wish existed when I started evaluating gateways. It works against any OpenAI- or Anthropic-compatible endpoint, and at the end there's a small open-source tool that automates the whole thing.

Disclosure up front: I work on daoxe, one such gateway. I'm writing the verification method so you can test any provider — including us. A verification tool is only useful if it will just as happily flag the people who wrote it.

The one check that doesn't work

The obvious move is to ask the endpoint: "What model are you?"

Don't bother. That answer comes from a system prompt or a fine-tune and is trivially spoofable in both directions. A relay serving you a 7B model can print "I am GPT-4-class" all day; a genuine flagship can be told to call itself something else. Self-report is theatre.

You have to test capability and behavior, not what the model claims about itself. And you have to measure — one request tells you nothing, because temperature, sampling, and routing all add noise.

The four cheats you're actually guarding against

When margins are thin, a dishonest relay can:

Substitute the model. You ask for a flagship; they serve a cheaper one and bill you for the flagship.
Serve heavy quantization. Same model name, aggressively quantized weights, visibly worse reasoning.
Truncate your context. They silently cap your context window to save tokens, so long-context tasks quietly fail.
Fall back without telling you. When the upstream is rate-limited, they route you to whatever's available and never surface it.

Each of these leaves a behavioral fingerprint. Here's how to read them.

Five signals that actually catch it

None of these individually proves anything. Together, they either reassure you or give you a reason to look closer.

1. Tokenizer fingerprint

Different model families use different tokenizers, and the tokenizer leaks through the usage.prompt_tokens the server reports. Send a battery of crafted strings — digit runs, whitespace, CJK, emoji, code, URLs, mixed unicode — and record how many tokens each costs. To cancel the fixed per-request chat-template overhead, measure a delta: tokens(anchor + probe) − tokens(anchor).

Best case (reference-free): diff that token vector against an endpoint you trust serving the same model id. A different vector means a different tokenizer, which means a different model family — a high-confidence signal.
Single endpoint: classify the vector against known tokenizer tables (e.g. OpenAI's cl100k_base / o200k_base). If the claimed family has no public reference (Claude, Gemini…), the honest answer is inconclusive — use a diff, not a guess.

2. Capability floor

A handful of tasks with objectively checkable answers you already know: multi-step arithmetic, string reversal, character counting, plus a strict-JSON format test. These are trivial for any full-tier model; a downgraded or heavily quantized substitute is more likely to trip. Weight failure more than success — passing an easy task proves little, but a "flagship" that fails a floor task is worth surfacing.

3. Long-context recall (the needle)

Generate deterministic filler text, hide a unique passphrase in the middle, and ask the endpoint to read it back — across several context lengths. Because you placed the needle, you always know the right answer (no external key needed). Recall that works at short context but breaks at longer context is the classic signature of silent context truncation.

4. Stability & performance

Repeat one fixed prompt N times at temperature=0 and look at output determinism, the stability of the server-reported model field and system_fingerprint, plus p50/p95 latency and error rate. A server that reports different model values or wildly different outputs for identical calls may be routing you across different backends.

5. Self-report (low weight, on purpose)

Ask the model to name itself and compare to the claim — but cap its confidence at low, because it's spoofable both ways. Include it only because its absence would be conspicuous; never let it drive a verdict.

The design rules that make the signals trustworthy

temperature=0, fixed max_tokens, fixed system prompt — remove randomness.
Measure, don't eyeball — repeat and record percentiles.
Diff against a reference you trust whenever you can. This is the single strongest move.
Re-run on a schedule. Silent degradation is a time-series problem, not a launch-day one.

Do it by hand (the 60-second version)

You can start without any tooling. Pin the parameters and fire the same probe at two endpoints:

# Your relay
curl -s -w '\n%{time_total}s\n' https://your-relay.example/v1/chat/completions \
  -H "Authorization: Bearer $RELAY_KEY" -H "Content-Type: application/json" \
  -d '{"model":"MODEL_ID","temperature":0,"max_tokens":64,
       "messages":[{"role":"user","content":"Reverse this exactly: abcdefghij. Then count the letters."}]}'

# The official API, same model, as your reference
curl -s -w '\n%{time_total}s\n' https://api.openai.com/v1/chat/completions \
  -H "Authorization: Bearer $OFFICIAL_KEY" -H "Content-Type: application/json" \
  -d '{"model":"MODEL_ID","temperature":0,"max_tokens":64,
       "messages":[{"role":"user","content":"Reverse this exactly: abcdefghij. Then count the letters."}]}'

Watch the usage.prompt_tokens, the answer quality, and the latency. Do it a few times. Divergence is your first thread to pull.

Automate it: an open-source probe

Running that battery by hand every week is tedious, so there's a small open-source CLI, llm-honesty-probe, that does it for you. Design goals worth calling out:

Zero runtime dependencies. Pure Python standard library — nothing to pip install, nothing in a lockfile to audit. Clone it and read it before you run it.
Your key is never printed, logged, or stored. It's read only from an environment variable (there's deliberately no --key flag, so it can't land in shell history), and every line of output passes through a redaction layer.
Provider-neutral. No allow-list, no "preferred" endpoint, no telemetry. It treats the official API and any relay identically.

git clone https://github.com/seven7763/llm-honesty-probe
cd llm-honesty-probe
python3 -m llm_honesty_probe --self-test        # runs against a built-in mock; no key needed

# Single endpoint: does it behave like the model it claims?
export OPENAI_API_KEY="sk-...your key..."        # never appears in argv or history
python3 -m llm_honesty_probe \
  --base-url https://any-provider.example/v1 --claimed-model MODEL_ID

# Differential (strongest): diff the relay against the official API
export OPENAI_API_KEY_OFFICIAL="sk-...official..."
python3 -m llm_honesty_probe \
  --base-url https://cheap-relay.example/v1 --claimed-model MODEL_ID \
  --compare-base-url https://api.openai.com/v1 --compare-model MODEL_ID \
  --compare-api-key-env OPENAI_API_KEY_OFFICIAL

# Claude / Anthropic-shaped endpoints:
export ANTHROPIC_API_KEY="sk-ant-..."
python3 -m llm_honesty_probe \
  --base-url https://any-provider.example --protocol anthropic --claimed-model claude-sonnet-...

The output marks each signal [OK] (consistent), [!!] (suspicious), or [--] (inconclusive), each with a confidence level. Use --json --out report.json to diff results in cron/CI over time — which is the whole point, since degradation shows up as drift, not on launch day.

Read the limitations, seriously

Overstating an honesty tool defeats its purpose. So, plainly:

These are signals, not proof. Nothing here cryptographically establishes which weights a provider runs. A provider that mirrors the tokenizer and matches capability and holds context and stays stable is, for practical purposes, giving you the model — but you can't prove intent from the outside.
The probe set is small and known. A provider that special-cases these exact probes could game a single-endpoint run. That's why the strongest mode is a differential diff you control.
"Suspicious" has innocent explanations. A smaller model can be the correct answer if that's what you asked for; a short context limit can be the model's real limit; different infra changes latency. Use --compare to disambiguate.
temperature=0 isn't perfectly deterministic on real hardware, so the determinism signal is intentionally low confidence.

Treat a clean run as reassuring, not a guarantee. Treat a flag as a reason to look closer, not a conviction.

What "honest" looks like from a provider

The reason I can publish this without flinching is that verifiability is the whole pitch. A gateway worth using should be:

Benchmarkable. Fine with you running this against it. (If a provider gets cagey when you ask to benchmark, that's your answer.)
Native on the protocols you use. OpenAI Chat Completions and Anthropic Messages, so Claude Code works properly — not just the OpenAI shape.
One key, many models, with an account-scoped /v1/models so what you see is what you can call.
Transparent on cost and status — itemized usage, nothing about billing hidden.

daoxe is built around exactly that: an OpenAI-compatible base URL (https://daoxe.com/v1), native Anthropic Messages for Claude Code, one key across many models, and a design meant to be verified rather than trusted. It's not available in mainland China. Please don't take my word for any of it — point the probe at it and at whatever you use today, and compare. If daoxe ever fails these checks, that's a bug report we want.

Complementary tool: llm-gateway-benchmark answers "is this endpoint fast and cheap?" (success rate, p50/p95, $/1M tokens). This one answers "is it actually the model it claims?" They're complementary, not competing.

TL;DR

Don't ask the model what it is. Do test capability and behavior.
Five signals: tokenizer fingerprint, capability floor, long-context needle, stability, and (low-weight) self-report.
Pin temperature=0, measure percentiles, diff against a reference, and re-run weekly.
Signals, not proof — but signals are exactly what you're missing right now.

Your cheap LLM relay might be swapping the model. Here's how to catch it.

Seven — Sun, 19 Jul 2026 21:54:07 +0000

Disclosure up front: I work on daoxe, one such gateway. I'm writing the verification method so you can test any provider — including us. A verification tool is only useful if it will just as happily flag the people who wrote it.

The one check that doesn't work

The obvious move is to ask the endpoint: "What model are you?"

The four cheats you're actually guarding against

When margins are thin, a dishonest relay can:

Substitute the model. You ask for a flagship; they serve a cheaper one and bill you for the flagship.
Serve heavy quantization. Same model name, aggressively quantized weights, visibly worse reasoning.
Truncate your context. They silently cap your context window to save tokens, so long-context tasks quietly fail.
Fall back without telling you. When the upstream is rate-limited, they route you to whatever's available and never surface it.

Each of these leaves a behavioral fingerprint. Here's how to read them.

Five signals that actually catch it

None of these individually proves anything. Together, they either reassure you or give you a reason to look closer.

1. Tokenizer fingerprint

Best case (reference-free): diff that token vector against an endpoint you trust serving the same model id. A different vector means a different tokenizer, which means a different model family — a high-confidence signal.
Single endpoint: classify the vector against known tokenizer tables (e.g. OpenAI's cl100k_base / o200k_base). If the claimed family has no public reference (Claude, Gemini…), the honest answer is inconclusive — use a diff, not a guess.

2. Capability floor

3. Long-context recall (the needle)

4. Stability & performance

5. Self-report (low weight, on purpose)

The design rules that make the signals trustworthy

temperature=0, fixed max_tokens, fixed system prompt — remove randomness.
Measure, don't eyeball — repeat and record percentiles.
Diff against a reference you trust whenever you can. This is the single strongest move.
Re-run on a schedule. Silent degradation is a time-series problem, not a launch-day one.

Do it by hand (the 60-second version)

You can start without any tooling. Pin the parameters and fire the same probe at two endpoints:

# Your relay
curl -s -w '\n%{time_total}s\n' https://your-relay.example/v1/chat/completions \
  -H "Authorization: Bearer $RELAY_KEY" -H "Content-Type: application/json" \
  -d '{"model":"MODEL_ID","temperature":0,"max_tokens":64,
       "messages":[{"role":"user","content":"Reverse this exactly: abcdefghij. Then count the letters."}]}'

# The official API, same model, as your reference
curl -s -w '\n%{time_total}s\n' https://api.openai.com/v1/chat/completions \
  -H "Authorization: Bearer $OFFICIAL_KEY" -H "Content-Type: application/json" \
  -d '{"model":"MODEL_ID","temperature":0,"max_tokens":64,
       "messages":[{"role":"user","content":"Reverse this exactly: abcdefghij. Then count the letters."}]}'

Watch the usage.prompt_tokens, the answer quality, and the latency. Do it a few times. Divergence is your first thread to pull.

Automate it: an open-source probe

Running that battery by hand every week is tedious, so there's a small open-source CLI, llm-honesty-probe, that does it for you. Design goals worth calling out:

Zero runtime dependencies. Pure Python standard library — nothing to pip install, nothing in a lockfile to audit. Clone it and read it before you run it.
Your key is never printed, logged, or stored. It's read only from an environment variable (there's deliberately no --key flag, so it can't land in shell history), and every line of output passes through a redaction layer.
Provider-neutral. No allow-list, no "preferred" endpoint, no telemetry. It treats the official API and any relay identically.

git clone https://github.com/seven7763/llm-honesty-probe
cd llm-honesty-probe
python3 -m llm_honesty_probe --self-test        # runs against a built-in mock; no key needed

# Single endpoint: does it behave like the model it claims?
export OPENAI_API_KEY="sk-...your key..."        # never appears in argv or history
python3 -m llm_honesty_probe \
  --base-url https://any-provider.example/v1 --claimed-model MODEL_ID

# Differential (strongest): diff the relay against the official API
export OPENAI_API_KEY_OFFICIAL="sk-...official..."
python3 -m llm_honesty_probe \
  --base-url https://cheap-relay.example/v1 --claimed-model MODEL_ID \
  --compare-base-url https://api.openai.com/v1 --compare-model MODEL_ID \
  --compare-api-key-env OPENAI_API_KEY_OFFICIAL

# Claude / Anthropic-shaped endpoints:
export ANTHROPIC_API_KEY="sk-ant-..."
python3 -m llm_honesty_probe \
  --base-url https://any-provider.example --protocol anthropic --claimed-model claude-sonnet-...

Read the limitations, seriously

Overstating an honesty tool defeats its purpose. So, plainly:

These are signals, not proof. Nothing here cryptographically establishes which weights a provider runs. A provider that mirrors the tokenizer and matches capability and holds context and stays stable is, for practical purposes, giving you the model — but you can't prove intent from the outside.
The probe set is small and known. A provider that special-cases these exact probes could game a single-endpoint run. That's why the strongest mode is a differential diff you control.
"Suspicious" has innocent explanations. A smaller model can be the correct answer if that's what you asked for; a short context limit can be the model's real limit; different infra changes latency. Use --compare to disambiguate.
temperature=0 isn't perfectly deterministic on real hardware, so the determinism signal is intentionally low confidence.

Treat a clean run as reassuring, not a guarantee. Treat a flag as a reason to look closer, not a conviction.

What "honest" looks like from a provider

The reason I can publish this without flinching is that verifiability is the whole pitch. A gateway worth using should be:

Benchmarkable. Fine with you running this against it. (If a provider gets cagey when you ask to benchmark, that's your answer.)
Native on the protocols you use. OpenAI Chat Completions and Anthropic Messages, so Claude Code works properly — not just the OpenAI shape.
One key, many models, with an account-scoped /v1/models so what you see is what you can call.
Transparent on cost and status — itemized usage, nothing about billing hidden.

TL;DR

Don't ask the model what it is. Do test capability and behavior.
Five signals: tokenizer fingerprint, capability floor, long-context needle, stability, and (low-weight) self-report.
Pin temperature=0, measure percentiles, diff against a reference, and re-run weekly.
Signals, not proof — but signals are exactly what you're missing right now.

Account-scoped /models is a feature: stop shipping static model lists

Seven — Thu, 16 Jul 2026 16:16:59 +0000

Disclosure and availability: DaoXE is a multi-model multi-protocol API gateway we operate. Not available in mainland China. This article is for developers in regions allowed by the service terms.

I used to put model IDs in README tables.

Then a teammate's key failed with model_not_found while mine worked.

Same base URL. Same client. Different account catalog.

This post is why I treat /v1/models as live data, not documentation.

Series

Two catalogs people confuse

Catalog	Source	Scope
Public marketing / models.dev	website, provider lists	“what exists in the product world”
Account `/v1/models`	`GET /v1/models` with your key	“what this key can call now”

They overlap. They are not identical.

The failure mode

Blog says model X
You hardcode X in Cline / Continue / SDK
Key B never had X (plan, region, entitlement, rotation)
You debug “the gateway is broken”

The gateway answered correctly: that account cannot use X.

The fix workflow

export DAOXE_API_KEY="your_api_key"
export DAOXE_BASE_URL="https://daoxe.com/v1"

curl --fail-with-body --show-error --silent \
  -H "Authorization: Bearer $DAOXE_API_KEY" \
  "$DAOXE_BASE_URL/models"

Then:

Copy an exact id from the JSON
Smoke one Chat Completions call with that id
Paste that id into the client
Re-run /models after plan changes

Design implications for apps

Anti-pattern	Prefer
Ship a frozen model enum in client builds	Refresh models at setup / login
Share one screenshot of models with the team	Each person lists with their key
Treat models.dev as entitlement proof	Treat it as discovery only
Cache model IDs forever	Cache with TTL + refresh on 404/not_found

Soft CTA

Product side:

https://daoxe.com/?utm_source=devto&utm_medium=organic&utm_campaign=global_launch_models_scope

Public notes: CLIENT_SETUP.md

Comment with whether your client supports model refresh from /models. No API keys in comments.

The base URL shape checklist: /v1, trailing slashes, and double paths

Seven — Thu, 16 Jul 2026 15:42:37 +0000

Disclosure and availability: DaoXE is a multi-model multi-protocol API gateway we operate. Public base: https://daoxe.com/v1. Not available in mainland China. This article is for developers in regions allowed by the service terms.

Most gateway failures I see are not model quality.

They are path join bugs.

The host is reachable. The key works in curl. The client still hits 404.

This post is the base URL checklist I run before blaming the API.

Series

What “OpenAI-compatible base” usually means

Many clients expect a root like:

https://daoxe.com/v1

They then append:

/chat/completions
/models
/embeddings

So the final path becomes:

https://daoxe.com/v1/chat/completions

If you already put /chat/completions in the base, or you omit /v1 when the client does not add it, you get nonsense paths.

Failure table

Setting	Final path symptom	Fix
Base missing `/v1` when client expects full OpenAI base	404 on `/chat/completions` at host root	use `https://daoxe.com/v1`
Base already ends with `/v1` and client adds `/v1/...`	`/v1/v1/chat/completions`	drop one `/v1` per client docs
Trailing slash quirks	intermittent join bugs	pick one style; retest with curl to the exact URL the client logs
Docs host ≠ live API host	wrong environment	use the live API host from the product, not a blog screenshot

Verify with curl to the exact joined URL

Do not only test “the host is up.”

Test the same path the client will call:

export DAOXE_API_KEY="your_api_key"
export DAOXE_BASE_URL="https://daoxe.com/v1"
export DAOXE_MODEL="paste_exact_id_from_models"

curl --fail-with-body --show-error --silent \
  -H "Authorization: Bearer $DAOXE_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{
    \"model\": \"$DAOXE_MODEL\",
    \"max_tokens\": 8,
    \"messages\": [{\"role\":\"user\",\"content\":\"OK\"}]
  }" \
  "$DAOXE_BASE_URL/chat/completions"

If that is green and the client is red, log the client’s request URL. Compare character-for-character.

Client-specific mental model

Client says	Often means
“OpenAI base URL”	include `/v1`
“API host”	sometimes host only; client adds `/v1`
“Custom provider”	check whether models.dev key implies the base

Read the client’s one line of docs. Do not assume every “OpenAI-compatible” UI joins paths the same way.

Soft CTA

Product side:

https://daoxe.com/?utm_source=devto&utm_medium=organic&utm_campaign=global_launch_baseurl

Public notes: CLIENT_SETUP.md

Comment with client name + the exact request URL from its logs (redact the key).

OPENAI_API_KEY vs DAOXE_API_KEY: which env name your client actually reads

Seven — Thu, 16 Jul 2026 15:16:12 +0000

Disclosure and availability: DaoXE is a multi-model multi-protocol API gateway we operate. It is not available in mainland China. This article is for developers in regions allowed by the service terms.

I had a green curl.

The agent still said “unauthorized.”

The key was fine. The env name was wrong for that client.

This post is the short map I use when base URL works but the tool never picks up the secret.

Series

Two different secrets

Kind	Example	Means
Vendor-shaped	`OPENAI_API_KEY`	many OpenAI SDKs / tools default here
Gateway-shaped	`DAOXE_API_KEY`	docs/catalog for DaoXE; some clients allow custom envar
Anthropic-shaped	`ANTHROPIC_API_KEY` / `x-api-key` field	Messages clients

Curl does not care about the name:

export DAOXE_API_KEY="your_api_key"
curl -H "Authorization: Bearer $DAOXE_API_KEY" https://daoxe.com/v1/models

The agent process might only read OPENAI_API_KEY unless you configure a custom provider field.

Common patterns

A. OpenAI SDK with custom base

export OPENAI_API_KEY="your_gateway_key"   # SDK default name
export OPENAI_BASE_URL="https://daoxe.com/v1"

Some libraries use OPENAI_API_BASE instead of OPENAI_BASE_URL. Check the client docs once.

B. Gateway-native name (when the tool supports it)

export DAOXE_API_KEY="your_gateway_key"
# base often hard-set to https://daoxe.com/v1 in UI or catalog

models.dev lists DaoXE with env DAOXE_API_KEY and api=https://daoxe.com/v1. Catalog-driven tools may expect that exact name.

C. Anthropic Messages clients

Bearer-only Chat Completions success does not prove Messages works. Those clients often need:

x-api-key
anthropic-version
POST /v1/messages

Failure table

Symptom	Likely cause	Fix
curl 200, agent 401	agent reads different env	set the name that client documents
works in shell, fails in IDE	IDE not inheriting shell env	set key in IDE provider UI / launch env
works for OpenAI path, fails Claude path	wrong protocol + auth style	Messages headers + `/v1/messages`
teammate fails with same README	account-scoped models / different secret store	each person lists `/v1/models`

Minimal verification order

curl with explicit -H "Authorization: Bearer …" (no env ambiguity)
export the client’s preferred key name
restart the agent process so it reloads env
one tiny Chat Completions call from that same process path

Public setup notes:

Soft CTA

If you want the product side:

https://daoxe.com/?utm_source=devto&utm_medium=organic&utm_campaign=global_launch_env

Comment with client name + which env names it documents. Do not paste keys.

DEV Community: Seven

OpenAI cut prices 80%. Here's why I'm not celebrating.

The hidden math of cheaper models

"What model are you?" doesn't work

Why this matters more now, not less

Test your endpoint right now

Deploy a real AI chat app in one click — key stays server-side (Vercel/Render/Railway)

The rule that makes these safe

Template 1 — Streaming chat UI on Vercel

Template 2 — OpenAI-compatible proxy on Render

Template 3 — Discord bot on Railway

Env vars cheat sheet

What this buys you

One key, two ecosystems: routing between DeepSeek/Kimi/Qwen and Claude/GPT/Gemini by task

Why route at all

The mechanism: model choice is a string

A simple task-based router (Python)

How to choose the route (without vibes)

Don't forget to verify what you're getting

Honest positioning

TL;DR

Add an AI chat widget to any website — without leaking your API key

The uncomfortable truth: anything in the browser is public

A) BYOK mode — for a page that is yours

B) Proxy mode — for public sites

Security details worth knowing

Configuration highlights

Why an OpenAI-compatible endpoint helps here

TL;DR

One API key, three terminals: driving many models from the shell with llm, mods and aichat

1. llm (Simon Willison) — the extensible one

Option A — zero-YAML via a plugin

Option B — no plugin, native config

2. mods (charmbracelet) — the pipe-native one

3. aichat (sigoden) — the all-in-one one

Which one should you use?

The reason one key matters here

TL;DR

Bring your own key to GitHub Copilot: Chat, CLI, App and SDK (with the honest caveat)

The caveat, first (so nobody's misled)

Why a gateway fits Copilot's three protocols

1. Copilot CLI (easiest — just env vars)

2. Copilot Chat (VS Code)

3. Copilot App (desktop)

4. Copilot SDK (for integrators)

Getting the model ids right

Troubleshooting

What to look for in a BYOK target

TL;DR

One key for CrewAI, AutoGen, LlamaIndex and 8 more: the base_url trick for Python agents

Why this works at all

The 11 frameworks, grouped by mechanism

A) Frameworks with a direct base_url parameter

B) Frameworks with an explicit provider object

C) Frameworks that route through LiteLLM (need the openai/ prefix)

D) Wrap the OpenAI client directly

The gotchas, in one table

Why one endpoint beats juggling keys

TL;DR

Your cheap LLM relay might be swapping the model. Here's how to catch it.

The one check that doesn't work

The four cheats you're actually guarding against

Five signals that actually catch it

1. Tokenizer fingerprint

2. Capability floor

3. Long-context recall (the needle)

4. Stability & performance

5. Self-report (low weight, on purpose)

The design rules that make the signals trustworthy

Do it by hand (the 60-second version)

Automate it: an open-source probe

Read the limitations, seriously

What "honest" looks like from a provider

TL;DR

Your cheap LLM relay might be swapping the model. Here's how to catch it.

The one check that doesn't work

The four cheats you're actually guarding against

Five signals that actually catch it

1. Tokenizer fingerprint

2. Capability floor

1. `llm` (Simon Willison) — the extensible one

2. `mods` (charmbracelet) — the pipe-native one

3. `aichat` (sigoden) — the all-in-one one

A) Frameworks with a direct `base_url` parameter

C) Frameworks that route through LiteLLM (need the `openai/` prefix)