DEV Community

sbt112321321
sbt112321321

Posted on

Claude Code with non-Anthropic models — a working setup & what broke

#python #api #ai #tutorial

I’ve been running Claude Code against a few non-Anthropic reasoning models for the past couple of weeks. The promise of models with larger context windows and different reasoning styles is real, but the integration path isn’t as smooth as docs suggest. Here’s my current setup, what actually works, and what I learned the hard way.

Why bother?

Claude Code’s agent loop is excellent, but sometimes I need:

  • Longer context for large codebase refactors (some models offer 1M tokens)
  • Different reasoning styles for architectural decisions
  • A fallback when Anthropic’s API has degraded performance in my region

The setup

The key insight: some third-party API gateways expose Anthropic-compatible endpoints. Instead of fighting with litellm proxies or custom middleware, you can point Claude Code directly at an OpenAI-compatible or Anthropic-compatible endpoint by configuring the underlying model provider.

Here’s what I’m using:

Provider configuration in Claude Code settings (~/.claude/settings.json):

{
  "modelOverrides": {
    "claude-sonnet-4-20250514": {
      "provider": "openai-compatible",
      "baseURL": "https://api.novapai.ai/v1",
      "apiKey": "${NOVAPAI_API_KEY}",
      "model": "deepseek-v4-pro"
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

For Anthropic-compatible endpoints, the config is even simpler. If the endpoint speaks the Messages API natively, you set ANTHROPIC_BASE_URL and ANTHROPIC_API_KEY:

export ANTHROPIC_BASE_URL="https://api.novapai.ai/v1"
export ANTHROPIC_API_KEY="sk-your-key-here"
Enter fullscreen mode Exit fullscreen mode

Then Claude Code picks it up automatically — no model override needed if the endpoint maps model names correctly.

Models I’ve actually tested:

Model Best use case Quirks
DeepSeek-V4 Pro Large refactors, reasoning-heavy tasks Sometimes overthinks simple edits
Kimi 2.6 Fast iterations, quick fixes Occasional hallucinated file paths
MiniMax 2.7 Balanced perf, good for daily driving Tool calling occasionally misses params
Qwen3 235B Complex architectural reasoning Slower token generation, but thorough

What I learned the hard way

1. Tool calling format mismatches
Not all providers handle the tool_use content blocks identically. MiniMax 2.7 occasionally returns tool_calls in OpenAI format even when the endpoint claims Anthropic compatibility. Symptom: Claude Code silently fails on tool execution, leaving you staring at a null response. Fix: wrap the provider in a lightweight proxy that normalizes tool call formats, or stick to models that have been tested against Anthropic’s schema.

2. Stop sequences behave differently
Anthropic models respect stop_sequences strictly. Some third-party models treat them as suggestions. This caused Claude Code’s structured output parsing to break intermittently — the model would generate past the expected stop token, and Claude Code would reject the entire response. Took me two evenings of debugging to isolate.

3. Rate limiting isn’t transparent
The gateway I used (NovaPi AI) has its own rate limiting layer. When hitting limits, the error messages weren’t the standard Anthropic 429 responses Claude Code expects. Instead, I got generic 503s that Claude Code interpreted as transient network failures and retried aggressively — leading to a tight loop that burned through my quota faster. If you try this, check how your provider surfaces rate limits.

4. Streaming chunk inconsistencies
Some providers batch streaming chunks differently. Claude Code’s streaming parser expects chunks at certain boundaries. When a provider sends larger aggregated chunks, the incremental display in terminal gets janky — text appears in bursts rather than smooth streaming. Not a dealbreaker, but annoying during long generations.

Is this production-ready?

For personal use and side projects: yes, with caveats. For team workflows: I’d be cautious. The debugging surface area expands significantly when you introduce a translation layer (even if it claims compatibility). I’d love to see better observability tools for tracing where exactly a model call diverges from expected behavior.

The gateway I’m using (NovaPi AI — novaiai.ai) handles the compatibility shim reasonably well for the models listed above, and their uptime has been solid. But the integration only works cleanly because their endpoint explicitly targets the Messages API spec.

Questions for the community

I’m genuinely curious about others’ experiences here:

  1. Has anyone stress-tested these models with Claude Code’s multi-turn agent loops beyond 50+ tool calls? I’m seeing some context degradation with Qwen3 235B around turn 30-40 where it starts repeating previous tool calls.

  2. What’s your approach to testing tool-call fidelity when switching providers? I’ve been running a small benchmark suite against known codebases, but it feels ad-hoc.

  3. Are there other gateway services doing the Anthropic-compatible shim well that I should test? I’d rather not maintain my own proxy layer if there are reliable options out there.

Would love to hear war stories and alternative setups.

Top comments (0)