I have run four vendor SDKs, four API keys and four billing dashboards side by side, and it was never worth the effort. An API router (also called an aggregator or a gateway) gives you one HTTP endpoint and lets you name the model family per request. That is how I reach Claude, GPT, DeepSeek and GLM from a single Python client today.
The setup below takes about 30 minutes and sits at intermediate level. You need a terminal with curl and jq 1.6+, Python 3.10+ with pip install openai, and an API key from one router. If you go self-hosted you also need vendor keys and a machine where a proxy can listen on port 4000. The Python client stays under 100 MB of disk; the proxy adds a few hundred more.
One rule before anything else: router names, model IDs, prices and free-tier limits change monthly. Anything below that depends on a specific vendor is a placeholder. Read the real values from /v1/models and the router's pricing page on the day you set this up, not from any article.
What "OpenAI-compatible" actually gets you
A router is one HTTP endpoint that forwards your request to whichever vendor you name, so your code talks to a single URL instead of four SDKs. The "OpenAI-compatible" part is what makes that practical: the endpoint accepts the same request body as OpenAI's /v1/chat/completions. That is why the Python snippet further down works unchanged against any router that claims compatibility, and why moving from GPT to DeepSeek becomes a string change rather than a refactor.
All four families I care about speak that shape. Claude goes through an OpenAI-compatible path or a native Anthropic passthrough, GPT, DeepSeek and GLM (Zhipu) go through /v1/chat/completions. What actually differs is the exact model IDs, the price per million input and output tokens, the context window and, for DeepSeek, whether a free or trial quota applies. Those are the four things to confirm in the vendor docs before you commit to anything.
Hosted aggregator or self-hosted proxy?
Pick the shape before you write code, because the credential model is different.
| Criterion | Hosted aggregator | Self-hosted proxy |
|---|---|---|
| What it is | A vendor-run API that already holds accounts with Anthropic, OpenAI, DeepSeek and Zhipu | A process you run, for example LiteLLM, holding your vendor keys and exposing one local port |
| Credentials you manage | One router key | Four vendor keys plus a local proxy key |
| Free/cheap access | Depends on current trial credits and per-model pricing, verify on the pricing page | You pay each vendor at list price, no markup and no free tier added |
| Billing | One invoice, vendor-level detail varies | Per-vendor invoices with full detail |
| Data path | Prompts pass through a third party, so check its retention policy | Prompts go only to the vendors you configured |
| Best fit | Fast prototyping, one-key demos | Teams with existing vendor contracts or compliance constraints |
In practice I start hosted when I want one key and no YAML, and move to the proxy when a prompt cannot leave my own infrastructure.
Step 1: collect the base URL, the key and the model IDs
Three strings drive everything after this: base URL, key, model ID. Guessing the model ID is the most common reason a 404 shows up two steps later, so read the list from the API instead of copying it from a blog post.
export ROUTER_BASE_URL="https://<router-base-url>/v1"
export ROUTER_API_KEY="<your-api-key>"
curl -s "$ROUTER_BASE_URL/models" \
-H "Authorization: Bearer $ROUTER_API_KEY" \
| jq -r '.data[].id' | sort | head -20
Illustrative output, where your IDs will differ:
anthropic/<claude-model-id>
deepseek/<deepseek-model-id>
openai/<gpt-model-id>
zhipu/<glm-model-id>
This works because /v1/models is part of the compatible surface, so a router that claims compatibility has to answer it. If HTML comes back instead of JSON, your base URL is wrong. Once you have the four IDs, pin them in a local file so later steps stay reproducible:
cat > models.env <<'EOF'
CLAUDE_MODEL=<id-from-the-list-above>
GPT_MODEL=<id-from-the-list-above>
DEEPSEEK_MODEL=<id-from-the-list-above>
GLM_MODEL=<id-from-the-list-above>
EOF
Check it with source models.env && echo $CLAUDE_MODEL, which should print an ID and not an empty line.
Step 2: point one client at the router
Overriding base_url in the OpenAI SDK replaces four vendor SDKs. Model choice becomes a dictionary lookup, and a fallback becomes a retry.
import os
from openai import OpenAI
client = OpenAI(
base_url=os.environ["ROUTER_BASE_URL"],
api_key=os.environ["ROUTER_API_KEY"],
timeout=60.0,
)
MODELS = {
"claude": os.environ["CLAUDE_MODEL"],
"gpt": os.environ["GPT_MODEL"],
"deepseek": os.environ["DEEPSEEK_MODEL"],
"glm": os.environ["GLM_MODEL"],
}
def ask(family: str, prompt: str) -> str:
r = client.chat.completions.create(
model=MODELS[family],
messages=[{"role": "user", "content": prompt}],
max_tokens=128,
)
print(family, r.usage.prompt_tokens, r.usage.completion_tokens)
return r.choices[0].message.content
print(ask("deepseek", "Reply with the single word: ready"))
A run prints something like deepseek 12 1 and then ready. Keep those token counts: multiplied by the per-million prices from Step 1, they give you the real cost of the call, which is more honest than a headline price.
For the self-hosted route, all credentials stay on the server and clients get only a local key:
model_list:
- model_name: claude
litellm_params:
model: anthropic/<anthropic-model-id>
api_key: os.environ/ANTHROPIC_API_KEY
- model_name: gpt
litellm_params:
model: openai/<openai-model-id>
api_key: os.environ/OPENAI_API_KEY
- model_name: deepseek
litellm_params:
model: deepseek/<deepseek-model-id>
api_key: os.environ/DEEPSEEK_API_KEY
- model_name: glm
litellm_params:
model: <provider-prefix>/<glm-model-id>
api_key: os.environ/ZHIPU_API_KEY
litellm --config config.yaml --port 4000
The proxy translates the OpenAI schema into each vendor's API and maps model_name onto a real model ID, so your app only ever sends claude or glm. The GLM provider prefix and the exact CLI flag are version-specific, so check the proxy docs if startup fails. Verify with curl -s localhost:4000/v1/models -H "Authorization: Bearer <local-key>" | jq '.data | length', which should return 4.
Step 3: smoke test answers, latency and cost
A router can list a model and still fail on the call, because of a provider outage, an exhausted quota or a permission you are missing. I would rather find that in a test than in a demo, so I loop over all four families and capture status, timing and the usage block in one pass:
for m in "$CLAUDE_MODEL" "$GPT_MODEL" "$DEEPSEEK_MODEL" "$GLM_MODEL"; do
curl -s -o /tmp/out.json -w "%{http_code} %{time_total}s\n" \
"$ROUTER_BASE_URL/chat/completions" \
-H "Authorization: Bearer $ROUTER_API_KEY" \
-H "Content-Type: application/json" \
-d "{\"model\":\"$m\",\"messages\":[{\"role\":\"user\",\"content\":\"Reply with the single word: ready\"}],\"max_tokens\":16}"
jq -r '.usage // "no usage block"' /tmp/out.json
done
Two things to look at. If a model answers but returns no usage block, you cannot compute cost from the response, and you should treat that as a limitation before building anything on top of it. And round-trip time in the time_total column is what you compare between families when you pick a default.
Finally, break a key on purpose: swap one character in ROUTER_API_KEY and run a single call. The error shape tells you how the router reports auth failures and quota exhaustion, which is exactly what you need to write a retry or fallback later.
The six-point checklist before you spend money
I tick all of these on the router's docs page before putting a card down:
- [ ] The model list contains all four families you actually need
- [ ] Free trial or free-tier conditions are stated explicitly, not implied
- [ ] Price per million input and output tokens is listed per model
- [ ] Rate limits (requests per minute, tokens per minute) are documented
- [ ] Prompt logging and retention policy is written down
- [ ] The OpenAI-compatible base path is documented as
/v1, not/api
If two or more boxes are empty, keep looking. The cheap part of a router is the price; the expensive part is discovering the gaps after your code depends on it.
Takeaway
A router turns four integrations into one base URL and a small dictionary of model IDs. Hosted gets you there fastest and exposes you to a third party's retention policy; self-hosted keeps prompts in your own path but means managing four sets of vendor keys. Either way, the discipline is the same: pull IDs from /v1/models, record the usage numbers on a real call, and verify the pricing and rate limits yourself, because all of it will be different by the time you read this.
This article was originally published on AI Release — https://ai-release.org. Shorter notes and updates on the same topic end up in the Telegram channel: https://t.me/ai_release1.
Top comments (0)