DEV Community

Cover image for How to Run Any Model in DeepSeek Harness ?
Hassann
Hassann

Posted on Originally published at apidog.com

How to Run Any Model in DeepSeek Harness ?

DeepSeek Harness (dsh) ships with DeepSeek models, but model providers are configurable. Point a provider block at an OpenAI-compatible endpoint, reference a credential, and run agent sessions against the model behind that URL. This works for local Ollama, company gateways, Qwen through DashScope compatibility mode, and catalog providers such as Anthropic and OpenAI.

Try Apidog today

This guide explains the provider configuration and includes three implementation recipes: local Ollama, a hosted OpenAI-compatible endpoint, and built-in catalog providers. The configuration details come from the official providers guide on the master branch, fetched August 20, 2026.

Note: dsh is a developer preview. The README warns about compatibility-breaking changes, so verify the documentation against the version you have installed before using these settings in production.

If you are new to the harness, start with what DeepSeek Harness is and how it works, then return here for provider setup.

Why change models in an agent harness?

An agent harness runs a loop:

  1. The model plans.
  2. The model calls tools.
  3. The harness returns tool results.
  4. The model continues with the updated context.

The harness owns that loop. The model is replaceable.

Cost

Agent sessions can consume tokens quickly because tool results are added back into context. You can route routine sessions to a cheaper model, use DeepSeek V4-Flash instead of V4-Pro, and keep a frontier model configured for harder tasks.

Data locality

For repositories or prompts that cannot leave your network, point a provider at a model running on your own hardware. Prompts, file contents, and tool outputs stay local while you retain the same harness workflow.

Local development

A local model is useful for plugin development and loop testing. It avoids API costs and network dependency during iteration. Switch back to a larger hosted model when model quality matters.

Provider routes are owned by the dsh-llm-pi-ai plugin. The plugin config catalog describes it as holding the “provider routes this instance owns.”

Configure a provider block

Custom providers live in $DSH_HOME/settings.yaml. You can also create them in the web UI under Settings → Models.

llm-pi-ai:
  providers:
    my-gateway:
      apiKeyEnv: GATEWAY_API_KEY
      api: openai-completions
      baseURL: https://gateway.example/v1
      models:
        - id: legacy-chat
        - id: vision-preview
          input: [text, image]
Enter fullscreen mode Exit fullscreen mode

Provider fields

  • my-gateway: The provider ID. Treat it as permanent. Choose a stable identifier; the UI display name is configured separately.
  • apiKeyEnv: The environment variable containing the API key. Store only the variable name in configuration, never the secret itself.
  • api: The endpoint protocol. Use openai-completions for OpenAI-compatible APIs.
  • baseURL: The API root URL used by the harness.
  • models: A list of available model IDs. Each id must match the identifier expected by the endpoint.
  • input: Declares supported input modalities. Custom models default to text-only. Add input: [text, image] for vision models.
  • defaultInput: A provider-level input fallback. A model-level input setting overrides it.
  • compat: Compatibility settings for endpoints that differ from standard OpenAI behavior:
    • supportsDeveloperRole: false for backends that reject the developer role.
    • maxTokensField: max_tokens for backends expecting the older token-limit field.

You can configure compat at the provider level or per model.

When adding a custom provider through the UI, use Fetch available models if the endpoint implements OpenAI-compatible GET /models. The UI can populate the model list automatically.

Store API keys safely

Secrets are stored write-only in:

$DSH_HOME/.credentials.yaml
Enter fullscreen mode Exit fullscreen mode

After saving a key through the UI, dsh returns only a redacted descriptor. The plaintext value is not shown again.

Keep settings.yaml limited to references such as apiKeyEnv and credential descriptors. This lets you share provider configuration without exposing secrets and rotate keys without editing model settings.

Recipe 1: Run a local model with Ollama

Ollama exposes an OpenAI-compatible endpoint at:

http://localhost:11434/v1
Enter fullscreen mode Exit fullscreen mode

See Ollama’s OpenAI compatibility guide.

Configure it as a custom provider:

llm-pi-ai:
  providers:
    ollama-local:
      apiKeyEnv: OLLAMA_API_KEY
      api: openai-completions
      baseURL: http://localhost:11434/v1
      models:
        - id: gpt-oss:20b
        - id: qwen3
Enter fullscreen mode Exit fullscreen mode

Local setup checklist

  1. Start Ollama.
  2. Pull the model you want to use: 
   ollama pull gpt-oss:20b
Enter fullscreen mode Exit fullscreen mode
  1. Confirm the installed model names: 
   ollama list
Enter fullscreen mode Exit fullscreen mode
  1. Set a placeholder key for the provider schema: 
   export OLLAMA_API_KEY=ollama
Enter fullscreen mode Exit fullscreen mode

Ollama does not require an API key locally, but the dsh schema expects a credential reference.

  1. Verify that Ollama serves models: 
   curl http://localhost:11434/v1/models
Enter fullscreen mode Exit fullscreen mode

The model ID in settings.yaml must match the Ollama tag exactly, including its version tag.

For the complete local model workflow, see how to run GPT-OSS using Ollama.

Before configuring dsh, test this endpoint in Apidog:

GET http://localhost:11434/v1/models
Enter fullscreen mode Exit fullscreen mode

If the request returns your model list, the server and base URL are correct. If it fails, fix the Ollama setup before debugging the harness configuration.

Small local models are useful for testing plugins and agent loops, but agent workflows rely heavily on tool calling and long context. Expect weaker planning and less reliable tool use than with larger frontier models.

Recipe 2: Use a hosted OpenAI-compatible endpoint with Qwen via DashScope

Alibaba Cloud Model Studio (DashScope) documents OpenAI compatibility through /compatible-mode/v1. Its OpenAI compatibility page lists regional, workspace-specific endpoints.

For example, a Singapore workspace uses:

https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1
Enter fullscreen mode Exit fullscreen mode

Configure Qwen in dsh:

llm-pi-ai:
  providers:
    qwen-dashscope:
      apiKeyEnv: DASHSCOPE_API_KEY
      api: openai-completions
      baseURL: https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1
      models:
        - id: qwen3-max
Enter fullscreen mode Exit fullscreen mode

Replace {WorkspaceId} with the workspace domain from the Model Studio console.

Set the credential in the environment where dsh runs:

export DASHSCOPE_API_KEY=your_api_key
Enter fullscreen mode Exit fullscreen mode

Check the vendor documentation for current model IDs. For related Qwen API details, see the Qwen 3.8 API guide.

This same configuration pattern works for any vendor with documented OpenAI compatibility:

  • Moonshot Kimi API
  • OpenRouter
  • vLLM deployments
  • Internal company gateways

Usually, only these fields change:

apiKeyEnv: YOUR_PROVIDER_API_KEY
baseURL: https://your-provider.example/v1
models:
  - id: your-model-id
Enter fullscreen mode Exit fullscreen mode

If you have configured open-source models in Codex, the role is similar: dsh’s provider YAML serves the same purpose as Codex model_providers configuration.

Hosted endpoint compatibility settings

If your provider rejects requests because of roles or token parameters, add compat:

llm-pi-ai:
  providers:
    qwen-dashscope:
      apiKeyEnv: DASHSCOPE_API_KEY
      api: openai-completions
      baseURL: https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1
      compat:
        supportsDeveloperRole: false
        maxTokensField: max_tokens
      models:
        - id: qwen3-max
Enter fullscreen mode Exit fullscreen mode

For vision models, explicitly declare image input support:

models:
  - id: your-vision-model
    input: [text, image]
Enter fullscreen mode Exit fullscreen mode

Custom models are text-only unless configured otherwise.

Recipe 3: Use built-in catalog providers

You do not need a custom provider block for mainstream cloud providers. dsh includes catalog providers for:

  • DeepSeek
  • Anthropic
  • OpenAI

For these providers, setup is primarily API-key configuration.

Other catalog entries use their native authentication flows:

Provider Authentication
AWS Bedrock AWS credentials
Google Vertex ADC project configuration
Azure API version configuration
Codex OAuth

Use catalog providers when you want the lowest-friction path to Claude, GPT, or DeepSeek models. Use custom providers for local runtimes, regional vendors, OpenAI-compatible aggregators, and internal gateways.

DeepSeek V4-Pro launched alongside the harness in August 2026. Refer to api-docs.deepseek.com for DeepSeek API details.

Select a model and understand session behavior

Adding a provider makes its models available. Selecting a model under Settings → Models sets the default for new sessions.

Two behaviors matter:

  1. Existing sessions keep their original model. Changing the default does not alter the model used by active or previous sessions.
  2. Deleting the current default provider blocks the composer. You must select another model before continuing.

This session pinning improves reproducibility. For example, when comparing harness behavior in DeepSeek Harness vs Claude Code, each transcript remains tied to the model it started with.

Troubleshooting provider configuration

Wrong or unreachable baseURL

Confirm that the endpoint ends at the expected API root:

OpenAI-compatible: /v1
DashScope compatible mode: /compatible-mode/v1
Enter fullscreen mode Exit fullscreen mode

Then test the model route outside dsh:

curl \
  -H "Authorization: Bearer $GATEWAY_API_KEY" \
  https://gateway.example/v1/models
Enter fullscreen mode Exit fullscreen mode

Use Download Apidog to send the same request and inspect the actual status code and response body instead of a wrapped harness error.

For offline development or unstable providers, mock these endpoints in Apidog:

GET  /models
POST /chat/completions
Enter fullscreen mode Exit fullscreen mode

Then point baseURL at the mock server while developing your integration.

Missing or empty environment variable

apiKeyEnv names an environment variable. It does not create it.

If dsh cannot read that variable, requests are sent without valid authentication and usually fail with 401.

Check the variable in the same environment that starts dsh:

echo $GATEWAY_API_KEY
dsh web
Enter fullscreen mode Exit fullscreen mode

A process started from a GUI or service manager may not inherit your shell profile.

Image input does not work

Custom models are text-only by default. Add image support per model:

models:
  - id: vision-preview
    input: [text, image]
Enter fullscreen mode Exit fullscreen mode

Or set it for all provider models:

defaultInput: [text, image]
Enter fullscreen mode Exit fullscreen mode

Unsupported roles or token fields

If your backend rejects a developer role or token limit field, configure:

compat:
  supportsDeveloperRole: false
  maxTokensField: max_tokens
Enter fullscreen mode Exit fullscreen mode

A previously working configuration stopped working

dsh is a developer preview. Pin the version you deploy, read release notes before upgrading, and expect configuration schemas to change.

The deepseek-harness repository is the source of truth.

Model providers are only one part of customization. You can also connect the tools that agents call. See using Apidog CLI inside DeepSeek Harness for that workflow.

FAQ

Does DeepSeek Harness officially support Ollama?

The official provider documentation does not mention Ollama by name. It supports endpoints using the openai-completions protocol, and Ollama documents an OpenAI-compatible API at:

http://localhost:11434/v1
Enter fullscreen mode Exit fullscreen mode

The Ollama example combines those documented pieces. Test it with your installed dsh version because the project is a developer preview.

Where does dsh store API keys?

dsh stores keys write-only in:

$DSH_HOME/.credentials.yaml
Enter fullscreen mode Exit fullscreen mode

The UI returns a redacted descriptor after saving. settings.yaml stores only references such as apiKeyEnv, not plaintext API keys.

Can different sessions use different models?

Yes. Changing the selected model changes the default for new sessions only. Existing sessions continue using the model they started with.

For example, use DeepSeek V4-Flash for routine sessions, switch to a larger model for a difficult task, and keep earlier sessions unchanged.

My endpoint works with curl but fails in dsh. What should I check?

Compare the exact request payloads.

The harness may send a developer role or a newer token-cap field that your backend does not support. Use the documented compatibility settings:

compat:
  supportsDeveloperRole: false
  maxTokensField: max_tokens
Enter fullscreen mode Exit fullscreen mode

Replay the harness-shaped request in an API client to identify which field the backend rejects.

Top comments (0)