DEV Community

Manish
Manish

Posted on

How to Switch LLM Providers in openJiuwen Agent Core Without Rewriting Your Agent

You built an agent against OpenAI. Now finance wants to try DeepSeek's pricing, or a customer contract requires a different endpoint. Changing providers should take minutes. Often it takes days.

That happens when one function holds everything. The API key, base URL, and retry settings sit next to the model name, temperature, and prompt, so a provider change spreads across every file that touches the model.

openJiuwen Agent Core splits those concerns into two objects. A ModelClientConfig describes the connection: the key, base URL, and retries. A ModelRequestConfig describes how the model should respond: the model name, temperature, and token limit. The workflow component that uses them knows nothing about either.

In this tutorial, you'll run a small summarization agent three ways:

  1. Against OpenAI gpt-4o.
  2. Against DeepSeek deepseek-chat, with only the config objects changed.
  3. With the provider picked by alias at runtime. Across all three runs, the workflow graph, prompt template, output schema, WorkflowAgent, and Runner code stay the same.

Prerequisites

You'll need:

  • Python 3.11, 3.12, or 3.13 (the companion repo was developed on 3.12)
  • An OpenAI API key
  • A DeepSeek API key
  • Comfort with async Python, virtual environments, and shell environment variables

No prior openJiuwen experience is assumed. The full source is in the companion repo.

Two config objects, two concerns

ModelClientConfig and ModelRequestConfig split provider switching into two separate types:

  • ModelClientConfig holds the connection settings: client_provider, api_key, api_base, timeout, max_retries, and verify_ssl. These change when you point at a different vendor's servers.
  • ModelRequestConfig holds the sampling settings: model, temperature, top_p, max_tokens, and stream. These shape how the model answers a single request. Both objects go into LLMCompConfig, each in its own slot (model_client_config and model_config). That gives a provider swap a small, predictable footprint: you build a new pair of configs, and everything downstream of LLMCompConfig (graph, prompt, agent, runner) keeps its existing code.
flowchart TB
    subgraph Stable["Never changes between providers"]
        G["Workflow graph<br/>Start to LLM to End"]
        P["Prompt template and JSON output schema"]
        A["WorkflowAgent and Runner"]
    end
    subgraph Swap["The only things you swap"]
        C["ModelClientConfig<br/>client_provider, api_key, api_base,<br/>timeout, max_retries, verify_ssl"]
        R["ModelRequestConfig<br/>model, temperature, top_p,<br/>max_tokens, stream"]
    end
    C --> LC["LLMCompConfig"]
    R --> LC
    LC --> G
    P --> G
    G --> A

One detail to watch: ModelRequestConfig accepts fields it doesn't declare, and the OpenAI client forwards them to the OpenAI SDK as keyword arguments. So a misspelled or misplaced setting passes validation, then fails at request time with an "unexpected keyword argument" error. Pass any request field that ModelRequestConfig doesn't declare through extra_body instead, as in extra_body={"frequency_penalty": 0}. The SDK merges those entries into the JSON request body.

This in-process split can also replace a common workaround. Some teams put an HTTP gateway or proxy in front of every model call, so the application sees a single endpoint. With Agent Core, the routing choice lives in two Python objects inside your process, so there's no extra service to deploy and monitor.

What the demo agent does

The agent takes a piece of text and returns a one-sentence summary, using a three-step workflow:

  1. Start receives a query.
  2. An LLMComponent applies the prompt and requests JSON output.
  3. End returns the summary as a dict with a response key.
flowchart LR
    S["Start<br/>receives <code>query</code>"] --> L["LLMComponent<br/>prompt + JSON output"] --> E["End<br/>returns the summary"]

The LLMComponent is the only step that talks to a model, which keeps the swap cheap: replace its two configs, and a different provider answers. The Start and End nodes keep their existing wiring.

Inside build_workflow(...), each node is registered under a name. The ${...} expressions in inputs_schema pass values between nodes, and add_connection sets the order:

def build_workflow(
    client_config: ModelClientConfig,
    request_config: ModelRequestConfig,
    *,
    workflow_id: str,
) -> Workflow:
    llm_config = build_llm_comp_config(client_config, request_config)
    llm = LLMComponent(llm_config)

    card = WorkflowCard(
        id=workflow_id,
        name=workflow_id,
        version="1.0",
        description="One-sentence summarisation workflow",
        input_params={
            "type": "object",
            "properties": {
                "query": {"type": "string", "description": "Text to summarise"}
            },
            "required": ["query"],
        },
    )
    flow = Workflow(card=card)

    start = Start()
    end = End({"responseTemplate": "{{output}}"})

    flow.set_start_comp("start", start, inputs_schema={"query": "${query}"})
    flow.add_workflow_comp("llm", llm, inputs_schema={"query": "${start.query}"})
    flow.set_end_comp("end", end, inputs_schema={"output": "${llm.output}"})
    flow.add_connection("start", "llm")
    flow.add_connection("llm", "end")

    Runner.resource_mgr.add_workflow(
        WorkflowCard(id=generate_workflow_key(card.id, card.version)),
        lambda: flow,
    )

    return flow
Enter fullscreen mode Exit fullscreen mode

${query} is the workflow input, ${start.query} is the output of the node named start, and ${llm.output} is the output field of the node named llm.

The repo keeps this shared code in one place, src/workflow_builder.py. There, build_llm_comp_config combines the two configs with the prompt, build_workflow wires Start to LLM to End, and build_agent wraps the workflow in a WorkflowAgent. Each scenario file supplies its own config pair and reuses these builders:

# File What it shows
1 src/baseline_agent.py The reference setup: an agent backed by OpenAI gpt-4o
2 src/provider_swap.py The same agent backed by DeepSeek deepseek-chat, with new configs and new IDs
3 src/multi_model.py A registry mapping "openai" and "deepseek" to config factories, plus extra_body for request fields that ModelRequestConfig doesn't declare

src/main.py runs all three in order and checks the required API keys before each scenario. If the DeepSeek key is missing, the run stops after Scenario 1 finishes.

Project setup

Clone the repo, create a virtual environment, and install the pinned dependencies before you run any scenario:

git clone https://github.com/manishh/openjiuwen--agent-cores-fix-for-switching-llm-providers
cd openjiuwen--agent-cores-fix-for-switching-llm-providers
python -m venv .venv
source .venv/bin/activate   # On Windows: .venv\Scripts\activate
pip install -r requirements.txt
Enter fullscreen mode Exit fullscreen mode

requirements.txt pins workswarm==0.2.7 and pytest. workswarm in turn installs openjiuwen 0.1.19, the Agent Core library the code imports. The scripts in src/ import only from openjiuwen.

The code reads API keys from environment variables. It doesn't load a .env file, so export the keys in your shell before you run anything:

export OPENAI_API_KEY=sk-...
export DEEPSEEK_API_KEY=sk-...

# Optional: override the default endpoints
export OPENAI_API_BASE=https://api.openai.com/v1
export DEEPSEEK_API_BASE=https://api.deepseek.com
Enter fullscreen mode Exit fullscreen mode

Scenario 1 needs OPENAI_API_KEY, and Scenario 2 needs DEEPSEEK_API_KEY. Scenario 3, and a full run of all three, need both.

If you keep the keys in a .env file (the repo ships a .env.example template), load them into your shell:

set -a
source .env
set +a
Enter fullscreen mode Exit fullscreen mode

The repo's .gitignore already excludes .env, which keeps real keys out of any tracked file.

The two *_API_BASE variables are optional. They set the api_base field of ModelClientConfig, and they default to https://api.openai.com/v1 and https://api.deepseek.com. Override them to send a provider's traffic to a regional endpoint or an internal mirror; the agent code stays the same.

Building the baseline WorkflowAgent

src/baseline_agent.py builds the agent in the order its step comments show: the two configs (steps 1 and 2), the workflow (steps 3 and 4), the agent (step 5), and the run (step 6). Read it top to bottom before you run it.

The ModelClientConfig points at OpenAI. It reads OPENAI_API_KEY, optionally reads OPENAI_API_BASE, and sets timeout=60, max_retries=3, and SSL verification on. The ModelRequestConfig selects gpt-4o with temperature=0.7, top_p=1.0, max_tokens=1024, and stream=False. The run is a single Runner.run_agent call, with the text to summarize passed as query.

Two small functions build the configs, and each one owns a single concern:

def build_openai_client_config() -> ModelClientConfig:
    return ModelClientConfig(
        client_provider="OpenAI",
        api_key=os.environ["OPENAI_API_KEY"],
        api_base=os.getenv("OPENAI_API_BASE", "https://api.openai.com/v1"),
        timeout=60,
        max_retries=3,
        verify_ssl=True,
    )


def build_openai_request_config() -> ModelRequestConfig:
    return ModelRequestConfig(
        model="gpt-4o",
        temperature=0.7,
        top_p=1.0,
        max_tokens=1024,
        stream=False,
    )
Enter fullscreen mode Exit fullscreen mode

run_baseline then wires everything together. It builds the workflow, wraps it in an agent, and runs it:

client_config = build_openai_client_config()
request_config = build_openai_request_config()

flow = build_workflow(
    client_config,
    request_config,
    workflow_id="summarise_openai_v1",
)

agent = build_agent(
    agent_id="summarise_agent_openai",
    description="Summarisation agent backed by OpenAI gpt-4o",
    flow=flow,
)

result = await Runner.run_agent(agent, {"query": "Agent Core enforces a hard separation ..."})
output = result.get("output")
Enter fullscreen mode Exit fullscreen mode

Note: The query text is shortened here; the file passes a longer sentence.

Run it from the repository root with -m, so the src package imports resolve:

python -m src.baseline_agent     # scenario 1 only (needs OPENAI_API_KEY)
Enter fullscreen mode Exit fullscreen mode

Agent Core logs a lot of INFO lines, so scan for the line prefixed with the provider name. Your summary text will differ:

[OpenAI gpt-4o] {'response': 'Agent Core separates connection and inference configurations to allow provider switching ...'}
Enter fullscreen mode Exit fullscreen mode

Note: Expect one ERROR ... add resource failed ... resource already exist line per agent built. build_workflow registers the workflow with the Runner explicitly, and WorkflowAgent.add_workflows registers it a second time, which is rejected. The next log line reads Successfully added workflow instance, and the run completes normally. The line is noisy and easy to mistake for a real failure the first time you see it.

Swapping the provider

Swapping the provider means changing the two config objects; the workflow, agent, and runner calls stay the same. Open src/provider_swap.py next to src/baseline_agent.py and diff them. The config builders carry the real changes: client_provider, api_key, api_base, and model now point at DeepSeek and deepseek-chat. The rest of the diff is labels: the workflow ID, agent ID, description, and printed messages.

Here's the diff for the two config builders. Everything else in the run function keeps its shape:

ModelClientConfig(
-    client_provider="OpenAI",
-    api_key=os.environ["OPENAI_API_KEY"],
-    api_base=os.getenv("OPENAI_API_BASE", "https://api.openai.com/v1"),
+    client_provider="DeepSeek",
+    api_key=os.environ["DEEPSEEK_API_KEY"],
+    api_base=os.getenv("DEEPSEEK_API_BASE", "https://api.deepseek.com"),
     timeout=60,
     max_retries=3,
     verify_ssl=True,
 )

 ModelRequestConfig(
-    model="gpt-4o",
+    model="deepseek-chat",
     temperature=0.7,
     top_p=1.0,
     max_tokens=1024,
     stream=False,
 )
Enter fullscreen mode Exit fullscreen mode
python -m src.provider_swap      # scenario 2 only (needs DEEPSEEK_API_KEY)
Enter fullscreen mode Exit fullscreen mode
[DeepSeek deepseek-chat] {'response': 'Agent Core separates connection configuration (ModelClientConfig) from inference ...'}
Enter fullscreen mode Exit fullscreen mode

The same query goes in, and the same JSON shape comes out: a dict with a response key. The prompt template and output schema live in workflow_builder.py and travel with the graph. DeepSeek's API is OpenAI-compatible, and Agent Core reuses its OpenAI client for it, so changing client_provider, api_key, api_base, and model is enough for the LLMComponent to talk to DeepSeek.

Keep in mind: Identical code can still produce different output. Models differ in how strictly they follow JSON instructions and how they phrase a summary. In our runs, both providers returned valid JSON with different wording. If downstream code parses the response field, test it against each provider you plan to use.

Provider-specific nuances and multi-model setup

A provider registry maps string aliases to config factory functions, so you can pick a provider at runtime without touching the workflow. In src/multi_model.py, the registry is _PROVIDER_FACTORIES, which maps "openai" and "deepseek" to factory functions that each return a (ModelClientConfig, ModelRequestConfig) tuple. get_agent(alias) looks up the factory, feeds the tuple into the shared workflow builder, and returns a ready agent. An unknown alias raises a ValueError that lists the valid ones.

The registry is a plain dict:

_PROVIDER_FACTORIES = {
    "openai": _make_openai_configs,
    "deepseek": _make_deepseek_configs,
}
_DEFAULT_PROVIDER: ProviderAlias = "openai"


def get_agent(alias: ProviderAlias = _DEFAULT_PROVIDER) -> WorkflowAgent:
    if alias not in _PROVIDER_FACTORIES:
        raise ValueError(f"Unknown provider alias '{alias}'. "
                         f"Available: {list(_PROVIDER_FACTORIES)}")
    client_config, request_config = _PROVIDER_FACTORIES[alias]()
    flow = build_workflow(
        client_config,
        request_config,
        workflow_id=f"summarise_{alias}_multi",
    )
    return build_agent(
        agent_id=f"summarise_agent_{alias}_multi",
        description=f"Summarisation agent — {alias} provider (multi-model config)",
        flow=flow,
    )
Enter fullscreen mode Exit fullscreen mode

Switching providers is now one argument:

agent = get_agent("deepseek")
result = await Runner.run_agent(agent, {"query": query})
Enter fullscreen mode Exit fullscreen mode

The demo builds the default agent ("openai") first and the alternative ("deepseek") second, and sends both the same query:

python -m src.multi_model        # scenario 3 only (needs both keys)
Enter fullscreen mode Exit fullscreen mode
[openai — default] {'response': 'The dual-config architecture separates provider topology and inference behavior ...'}
[deepseek — alternative] {'response': 'The dual-config architecture enforces a type-level separation between provider topology ...'}
Enter fullscreen mode Exit fullscreen mode

The DeepSeek factory also shows how to send request fields that ModelRequestConfig doesn't declare, by passing them in extra_body:

ModelRequestConfig(
    model="deepseek-chat",
    temperature=0.7,
    top_p=1.0,
    max_tokens=1024,
    stream=False,
    extra_body={"frequency_penalty": 0},
)
Enter fullscreen mode Exit fullscreen mode

To add a third provider:

  1. Write a function in src/multi_model.py that returns a (ModelClientConfig, ModelRequestConfig) pair.
  2. Register it in _PROVIDER_FACTORIES under a new alias.
  3. Add that alias to the ProviderAlias type. The workflow and agent code stay as they are. Use a client_provider value that your installed openjiuwen version supports; an unsupported name fails when you build ModelClientConfig, and the error lists the valid providers.

To run all three scenarios back to back, use the entry point:

python -m src.main               # all three scenarios in order
Enter fullscreen mode Exit fullscreen mode

Offline checks with the test suite

The smoke tests check that the Agent Core classes import and that the config objects accept the expected parameter names. They make no API calls and need no LLM keys, so they give you a fast check after you upgrade openjiuwen.

Run them from the repository root with the virtual environment active:

pytest -v
Enter fullscreen mode Exit fullscreen mode
tests/test_smoke.py::test_model_client_config_import PASSED              [  9%]
tests/test_smoke.py::test_model_request_config_import PASSED             [ 18%]
tests/test_smoke.py::test_workflow_imports PASSED                        [ 27%]
tests/test_smoke.py::test_runner_import PASSED                           [ 36%]
tests/test_smoke.py::test_workflow_agent_imports PASSED                  [ 45%]
tests/test_smoke.py::test_model_client_config_instantiation PASSED       [ 54%]
tests/test_smoke.py::test_model_request_config_instantiation PASSED      [ 63%]
tests/test_smoke.py::test_model_request_config_extra_body PASSED         [ 72%]
tests/test_smoke.py::test_generate_workflow_key PASSED                   [ 81%]
tests/test_smoke.py::test_workflow_card_and_workflow_instantiation PASSED [ 90%]
tests/test_smoke.py::test_llm_comp_config_correct_param_name PASSED      [100%]
============================== 11 passed ==============================
Enter fullscreen mode Exit fullscreen mode

A green suite confirms that the import paths and config parameter names your code depends on still exist. It can't confirm that live provider calls succeed; for that, run the scenarios with real keys using python -m src.main.

The repo's .github/workflows/verify.yml runs pytest -v in CI on every push.

Common problems

Symptom Cause and fix
[ERROR] Missing environment variables: ... The key isn't exported in the current shell. Re-run the export commands or source your .env file.
unexpected keyword argument 'extra_fields' Provider-specific request fields belong in extra_body, not extra_fields.
Import errors for src Run from the repository root with python -m src.<module>.

| ERROR ... add resource failed ... resource already exist | Expected: the workflow is registered twice. Ignore it if the run
completes. |

Where to take the pattern next

Once every provider is a factory that returns a config pair, routing becomes ordinary Python. The demo stops at choosing an alias, but the same registry supports patterns like these:

  • Send each type of request to the cheapest model that handles it well.
  • Pin a tenant to an approved provider for compliance.
  • Catch a failed call and retry with the next alias in a fallback list.

Each of these takes a few lines of your own code around get_agent(alias), and the workflow stays put. Two practical tips:

  • Build each alias's agent once and reuse it. Every get_agent call registers the workflow again and logs the duplicate-registration error.
  • Catch KeyError in a fallback loop. Each factory reads its API key when it's called, so a missing key raises a KeyError as soon as that alias is built, not when the request is sent. Browse the source and star the project at the agent-core repository.

Read the openJiuwen agent core documentation for the rest of the SDK, and check it against the source of the version you installed, since docs and code can drift apart.


Frequently asked questions

What is the difference between ModelClientConfig and ModelRequestConfig in openJiuwen Agent Core?

ModelClientConfig holds the connection settings: client_provider, api_key, api_base, timeout, max_retries, and verify_ssl. ModelRequestConfig holds the sampling settings: model, temperature, top_p, max_tokens, and stream. Both feed a single LLMCompConfig, so a provider swap means building a new pair. The connection config always changes, and the model name usually changes with it. Settings such as temperature can carry over unchanged.

Can I switch LLM providers at runtime without restarting my application?

Yes. The registry in src/multi_model.py maps string aliases to config factory functions. Calling get_agent() with a different alias builds a new config pair and a new agent from the same workflow code, inside the running process. The workflow code doesn't change, though each call creates a new workflow and agent instance, so build each alias once and reuse it.

Does the provider swap work with any LLM provider, or only OpenAI-compatible APIs?

The LLMComponent swap works directly with providers that expose an OpenAI-compatible API, such as DeepSeek. For other providers, check which client_provider values your installed openjiuwen version supports; that list determines which vendors the client layer can reach.

What happens if a provider returns JSON that doesn't match the expected output schema?

If an LLMComponent is set to response_format={"type": "json"}, Agent Core parses the reply and validates it against the node's output_config. Invalid JSON or a missing or mistyped required field raises an error and stops the workflow run. It doesn't retry or fall back to the raw string, so add your own retry or fallback around the run if you need one. If your downstream code parses the response field, validate it, and test against each provider before deploying.

How do I pass provider-specific request parameters that aren't part of the standard interface?

Use the extra_body field on ModelRequestConfig. Passing them as extra_fields, or as undeclared top-level fields, raises an unexpected keyword argument error at request time.

What does the smoke test suite actually verify?

The suite confirms that the Agent Core classes import correctly and that ModelClientConfig, ModelRequestConfig, and LLMCompConfig accept the expected constructor parameter names. It makes no API calls and needs no LLM keys, which makes it a quick check after upgrading the openjiuwen package. It doesn't test live provider calls.

Can I use this pattern for compliance-driven provider isolation or cost-based routing?

Yes. Once every provider is a factory returning a config pair, routing decisions become ordinary Python. You can pin a tenant to an approved provider, choose a cheaper model per request type, or build a fallback list that retries with the next alias on a failed call. That logic lives in your own code around get_agent(), while the workflow stays put.


Top comments (0)