Start with a Question: Why Is Adding a New Provider So Easy?
If you go look at core/llm.py, you'll find something interesting: the code already supports OpenAI, DeepSeek, Qwen, Kimi, Zhipu, SiliconFlow, Ollama, vLLM, and ten other different services, but the entire file has no ten sections of wildly different adapter code.
The secret is in the PROVIDER_PROFILES table.
The Conclusion First
MyCodeAgent's LLM layer manages providers using table-driven design. Adding a new provider only requires three steps:
| Step | What to Do |
|---|---|
| 1. Add a row to the table | Add a profile to PROVIDER_PROFILES
|
| 2. Extend the type annotation | Add the name to the SUPPORTED_PROVIDERS Literal |
| 3. Configure environment variables | Add the new provider's variable description to .env.example
|
No need to change the call chain, write a new class, or add if-else branches.
I. PROVIDER_PROFILES: The Core of Provider Routing
Let's look at DeepSeek's profile first — it's the most typical of all profiles:
# core/llm.py
PROVIDER_PROFILES = {
"deepseek": {
"key_envs": ("DEEPSEEK_API_KEY", "LLM_API_KEY"), # look up API key in priority order
"detect_envs": ("DEEPSEEK_API_KEY",), # which variables to check for auto-detection
"base_url_envs": ("LLM_BASE_URL",), # which env var to read the endpoint from
"base_url": "https://api.deepseek.com", # default endpoint if not configured
"model": "deepseek-chat", # default model name
"url_markers": ("api.deepseek.com",), # infer provider from base_url
},
...
}
These five fields control the entire provider routing logic:
key_envs: Priority list for looking up the API key. The framework checks environment variables left to right, using the first one with a value. This lets users use either the dedicated DEEPSEEK_API_KEY or the generic LLM_API_KEY.
detect_envs: Which variables to use for auto-detection. When the user hasn't specified a provider, the framework scans all profiles' detect_envs to see which provider's variables are set in the environment, and automatically selects that one. If multiple providers match simultaneously, it throws an error requiring the user to specify explicitly (avoiding ambiguity).
base_url: Default endpoint. Used when the user hasn't configured LLM_BASE_URL.
url_markers: Infer the provider from the user's configured base_url string. For example, if the user sets LLM_BASE_URL=https://api.deepseek.com/v1, the framework checks that this URL contains "api.deepseek.com" and automatically recognizes the provider as deepseek.
model: Default model. Used when the user hasn't specified LLM_MODEL_ID.
II. The Complete Priority Order for Provider Resolution
After understanding the table structure, let's see how the framework determines which provider to ultimately use:
def _resolve_provider(self, provider, api_key, base_url):
# Priority 1: provider parameter explicitly passed in code
if provider:
return self._normalize_provider(provider)
# Priority 2: LLM_PROVIDER environment variable
env_provider = self._get_env("LLM_PROVIDER")
if env_provider:
return self._normalize_provider(env_provider)
# Priority 3: auto-detection (scan detect_envs, or infer from base_url string)
return self._auto_detect_provider(api_key, base_url)
Three levels of priority, with fallbacks at each level. For users, the most common configuration approach is setting in .env:
LLM_PROVIDER=deepseek
DEEPSEEK_API_KEY=sk-xxxxx
LLM_MODEL_ID=deepseek-chat
Or even more conveniently, let the framework auto-detect:
DEEPSEEK_API_KEY=sk-xxxxx # set only this one; the framework will automatically recognize it as DeepSeek
III. Connecting a New Provider: "SomeNewAI" as an Example
Suppose a new service called "SomeNewAI" appears on the market with an OpenAI-compatible API (this is now the standard for almost all new model providers), at the endpoint https://api.someneai.com/v1.
Step One: Add a row to PROVIDER_PROFILES
# core/llm.py — add to PROVIDER_PROFILES:
"someneai": {
"key_envs": ("SOMENEAI_API_KEY", "LLM_API_KEY"),
"detect_envs": ("SOMENEAI_API_KEY",),
"base_url_envs": ("LLM_BASE_URL",),
"base_url": "https://api.someneai.com/v1",
"model": "someneai-pro",
"url_markers": ("api.someneai.com",),
},
Step Two: Extend the type annotation
# core/llm.py — add the new name to the SUPPORTED_PROVIDERS Literal
SUPPORTED_PROVIDERS = Literal[
"openai",
"deepseek",
...
"someneai", # new addition
"auto",
]
Step Three: Update .env.example
# .env.example — add new provider description in the LLM configuration section
# SomeNewAI
# SOMENEAI_API_KEY=your-key-here
Done. Users can now use it like this:
LLM_PROVIDER=someneai
SOMENEAI_API_KEY=sk-xxxxx
LLM_MODEL_ID=someneai-pro
Or override in the startup command:
uv run python main.py --provider someneai --api-key sk-xxxxx --model someneai-pro
IV. What If the New Provider Has Special Quirks?
OpenAI-compatible APIs are the mainstream now, but different providers have minor implementation differences. core/llm.py already has some "quirk handling" for specific services:
# core/llm.py — quirk handling when building a request
def _build_request(self, messages, tools, ...):
# Some providers don't support temperature=0 and will throw an error
if self.provider in ("zhipu",):
temperature = max(temperature, 0.01)
# Some providers don't support multiple system messages; they need to be merged
if self.provider in ("kimi", "moonshot"):
messages = self._merge_system_messages(messages)
# Some providers don't support tool_choice="auto"
if self.provider in ("some_provider",):
request.pop("tool_choice", None)
If the new provider also has similar quirks, just add a conditional branch for it in _build_request().
The framework separates "routing" (which provider? what key? what endpoint?) from "quirk handling" (does the request format need adjusting?): routing is in the table, quirks are in _build_request().
V. Local Models: How Ollama Is Connected
For fully local models (like Ollama), the connection method is the same — the base_url in the profile just points to localhost:
"ollama": {
"key_envs": ("OLLAMA_API_KEY", "LLM_API_KEY"),
"detect_envs": ("OLLAMA_API_KEY", "OLLAMA_HOST"),
"base_url_envs": ("OLLAMA_HOST", "LLM_BASE_URL"),
"base_url": "http://localhost:11434/v1",
"model": "llama3.2",
"default_key": "ollama", # default API key value (Ollama doesn't validate keys, anything works)
"url_markers": ("ollama",),
},
Users just need to:
# First start the Ollama service
ollama serve
# Then configure in .env
LLM_PROVIDER=ollama
LLM_MODEL_ID=llama3.2
# OLLAMA_API_KEY can be left empty; the framework will use "ollama" as the default value
Design Highlights
1. Table-driven, not inheritance
Each provider's differences live only at the data layer (the table), not the code layer (one class per provider). Adding a provider is adding a data row, not adding a code class. This keeps maintenance cost very low — all providers' routing logic is concentrated in one place, and differences are visible at a glance.
2. Three-level API key lookup
The key_envs list design allows "dedicated key takes priority, generic key as fallback." DEEPSEEK_API_KEY has higher priority than LLM_API_KEY, letting users with multiple providers configure separate keys for each, rather than changing LLM_API_KEY every time they switch providers.
3. Auto-detection eliminates explicit declaration
Scanning detect_envs to automatically identify the provider means users only need to set the API key without also setting LLM_PROVIDER. Friendly for newcomers, reduces the number of required configuration items.
Summary
| Design Choice | Approach | Engineering Value |
|---|---|---|
| Provider routing | Table-driven (PROVIDER_PROFILES) | Adding a provider adds a data row, doesn't change code logic |
| API key lookup | Multi-level key_envs | Dedicated key takes priority, generic key as fallback |
| Provider identification | Three-level priority (parameter → env var → auto-detect) | Flexible, minimal configuration to get running |
| Quirk handling | Per-provider branches in _build_request() | Routing logic and quirk logic are separate, each can evolve independently |
The next article covers Skills — using Markdown to define a reusable "expert behavior," the lightest-weight extension method in MyCodeAgent.
About the Source Code for This Series
All analysis in this series is based on the open source project MyCodeAgent.
The source code already has companion comments added at key locations in the order covered by this series — you can read alongside the code, or clone it directly to run, modify, and extend it to build your own agent.
git clone https://github.com/chendongqi/MyCodeAgent
cd MyCodeAgent
cp .env.example .env # fill in your LLM API key
uv sync
uv run python main.py
Visit PrimeSkills — a carefully curated AI Agent and skills marketplace where every piece of content is validated through real enterprise-grade workflows. No hype, only what actually works.
For more practical knowledge and interesting products, visit my personal homepage
Top comments (0)