DEV Community: mpoper

How Can a Startup Reduce LLM API Costs: A 2026 Playbook

mpoper — Thu, 03 Sep 2026 14:10:02 +0000

How Can a Startup Reduce LLM API Costs: A 2026 Playbook

As of Sep 2026, startups can reduce LLM API costs by 70–85%—and up to 90% in optimal cases—by combining model routing, prompt caching, batch processing, and semantic caching, without sacrificing output quality on critical tasks. According to 2026 research from MorphLLM, NeuralTrust, and PremAI, the most effective strategies involve selecting the right model size for each task, aggressively caching repeated context, and leveraging discounted batch APIs. While enterprise LLM API spending more than doubled from $3.5B to $8.4B between late 2024 and mid-2025, the same period saw the maturation of cost-optimization tools and techniques accessible to any technical team. This guide systematically breaks down the highest-leverage strategies, benchmarked with 2026 data, so your startup can maintain performance while drastically cutting token spend.

Start with Provider Price Benchmarking

Before optimizing anything else, you must understand current token pricing across major providers. As of Sep 2026, the pricing landscape is highly competitive, with significant per-model and per-feature variations. For a comprehensive, constantly updated comparison, refer to our pricing page, which tracks changes across OpenAI, Claude, DeepSeek, Kimi, Gemini, and domestic Chinese models.

Key benchmarking steps include:

Evaluate input vs. output pricing separately, as they often differ by a 3–5x factor (e.g., OpenAI's general-purpose models typically price input:output at 1:4, Anthropic Claude at 1:5—OpenAI pricing, Anthropic pricing; exact rates subject to official pricing pages).
Check volume discounts offered by providers for committed usage tiers.
Compare batch vs. real-time rates, which we detail later.
Factor in cache-read discounts: Anthropic offers up to 90% off on cache hits (Anthropic Prompt Caching docs), while OpenAI offers 50% (OpenAI Prompt Caching docs); both discounts are documented in the providers' official API pricing pages.

A practical approach: pull a week of your production logs, replay them against two or three candidate models' pricing sheets, and calculate the total cost per successful request. Even a 20% difference in per-token price becomes substantial at scale.

Choose the Right Model Size for Each Task

The single most impactful lever is model routing: sending simple tasks to smaller, cheaper models and reserving flagship models for complex reasoning. As of Sep 2026, studies from NeuralTrust and PremAI indicate that a well-designed routing layer can cut costs by 40–70% in real workloads without measurable degradation in task-specific accuracy. Historical academic benchmarks are consistent with this: Stanford's FrugalGPT paper reported cost reductions of up to 98% via model routing/cascading while improving output quality (arXiv:2305.05176, historical data from May 2023); LMSYS's RouteLLM showed 85% cost savings vs. the then-current GPT-4 at equal quality (arXiv:2406.18665, historical data from June 2024). More recent 2026 production evaluations confirm 40–70% savings in real workloads.

For example, a startup running a customer-support chatbot might route:

Simple FAQ and intent detection → DeepSeek-V4-Flash or Kimi K2.5 (lower cost per token).
Moderate reasoning and structured extraction → Claude Sonnet 4.6 or GPT-5.2.
Complex multi-step analysis and code generation → GPT-5.6 Terra/Sol or Claude Opus 5.

To illustrate the shape of the savings: a mix of DeepSeek-V4-Flash for classification and a flagship model only for final answer synthesis can cut token spend by roughly half or more compared with using a flagship for everything—run your own logs against the pricing page to get your exact number. For reference on routing viability across models, see our development docs, which document the feasibility of task-based model switching.

Implement Prompt Caching and Context Optimization

Prompt caching is the easiest win with the fastest payback period. As of Jan 2026, a PwC study across 500 agent sessions with 10,000-token system prompts found that enabling prompt caching reduced costs by 41–80% and improved time-to-first-token (TTFT) by 13–31%.

Two types of caching matter:

Exact-match caching: Provider-side, automatic (e.g., Anthropic's and OpenAI's caches; per Anthropic docs, cache reads are up to 90% cheaper, and per OpenAI docs, cached input tokens are 50% cheaper—Anthropic docs, OpenAI docs). It works when your system prompt or conversation history is byte-identical. Set up is minimal, and the discounts are automatic.
Semantic caching: A middleware layer that stores vector embeddings of previous responses and matches new queries by similarity (threshold 0.90–0.95). According to a 2026 VentureBeat case cited by Exadel and TowardsAI, semantic caching reduced a monthly bill from $47,000 to $12,700 (73% reduction) and raised cache hit rates from 18% to 67%.

To maximize exact-match cache hits:

Keep your system prompt static; if you must vary it, use a limited set of templates.
Reuse conversations where possible instead of starting fresh each time.
Put the most variable content at the end of the prompt, as many caches use prefix matching.

Compress and Filter Input Data Before Inference

Sending an entire codebase, document, or chat history to the LLM is expensive. Instead, compress and filter before the request reaches the model.

Effective techniques include:

Summarization: Use a small, cheap model (e.g., DeepSeek-V4-Flash) to summarize long documents into a 500-token brief, then send that to the flagship model for reasoning.
Retrieval-Augmented Generation (RAG): Instead of stuffing a full knowledge base into the context, retrieve only the top 3–5 relevant chunks (by semantic similarity) and concatenate them.
Semantic filtering: Drop queries that are near-duplicates of previously answered ones, using vector search.

As of 2026, research from Exadel and TowardsAI shows that aggressive context compression can reduce token consumption by 50–70% while preserving over 90% of answer quality on typical knowledge-based tasks. Since input tokens are often billed at a higher rate than cached ones, filtering before inference has an outsized impact.

Set Up Request Batching and Asynchronous Processing

Batch APIs are a hidden discount that many startups overlook. As of Sep 2026, both OpenAI and Anthropic offer 50% discounts on requests that can wait up to 24 hours (OpenAI Batch API docs; Anthropic Batch Processing docs). For startups running nightly data processing, report generation, or internal evaluations, switching from real-time to batch is a one-line code change.

When batch processing is combined with prompt caching on repeated content, the savings compound. According to 2026 research from MorphLLM, the combined discount can be as high as 95% for workloads with high cache-hit rates. But note the trade-off: batch requests have longer latency, so they are only suitable for non-interactive scenarios.

A recommended pattern:

Real-time: User-facing chat, code generation, agentic tasks.
Batch: Any scheduled job, offline evaluation, mass content enrichment, or async data extraction.

Monitor Usage with Token-Level Analytics

You cannot reduce what you don't measure. As of 2026, startup teams are expected to log per-endpoint, per-user, and per-feature token counts, then set hard budget alerts.

Best practices include:

Log input/output/cache-hit tokens for every request.
Use dashboards to track weekly cost per model, per feature, and per team.
Set alerts at 80% and 100% of your monthly budget in dollar terms.
Enforce rate limits per API key or user, so a single runaway script cannot spike the bill.

Since token rates and model rankings shift frequently, we recommend a weekly review cycle for short-term adjustments and a monthly review for broader strategy changes, as we note in our model directory.

Use an AI Gateway or Proxy for Cost Governance

A gateway layer (like the one HeFu provides) gives you centralized control over your entire LLM stack. As of 2026, gateways have evolved to include:

Virtual API keys per team or project, isolating costs by owner.
Automatic model fallback: If a cheap model fails or times out, route to a pricier one without code changes.
Weighted load balancing across multiple providers, so you can e.g., send 80% of traffic to DeepSeek-V4-Flash and 20% to GPT-5.6 Terra for complex reasoning.
Edge-level exact-match caching (as seen in Cloudflare AI Gateway or similar) to short-circuit repeated requests before they hit the LLM.

HeFu's unified API endpoint (https://api.hefu.hk/v1) is designed for exactly this kind of governance. By aggregating OpenAI GPT, Claude, DeepSeek, Kimi, and Gemini models behind one key, you can switch models by changing a single field, without rewriting your codebase. This makes implementing a cost-aware routing policy straightforward. For a step-by-step integration walkthrough, visit our development docs.

Negotiate Custom Pricing and Explore Open-Source Self-Hosting

For startups with predictable, high-volume usage (e.g., > 1M tokens/day), negotiating a custom contract is viable. As of Sep 2026, major providers offer volume-based discounts of 15–30% off list prices for committed annual spending. The key is having usage data to justify the commitment.

Self-hosting open-weight models is another lever, but it is not free. It eliminates per-token API fees but introduces compute, storage, and maintenance overhead. It becomes cost-effective only when you have consistent, high-traffic workloads (e.g., 100K+ tokens/hour) and where latency/privacy requirements prevent cloud calls. For most startups, API-based routing remains more economical.

A practical approach is hybrid: host a small open-weight model for classification or extraction, and call a managed API for generative tasks. This aligns with the "rationalize model size" principle from earlier. For a deeper comparison of available Chinese models, our pricing page is a useful reference.

Compare HeFu's Unified API for Cost Efficiency

At this point, the case for a unified aggregator like HeFu should be clear. HeFu's model catalog includes:

OpenAI GPT-5.6 (Terra/Sol/Luna), GPT-5.5, GPT-5.4, and GPT-5.2, plus GPT-5.3 Codex models.
Claude Opus 5 / Fable 5, Sonnet 4.6, and Opus 4.8/4.7/4.6.
DeepSeek-V4-Pro and V4-Flash for extreme-value reasoning.
Kimi K2.5/K2.6/K3 for long-context Chinese tasks.
Gemini 3.6 Flash, 3.5 series, 3.1 Pro, and 2.5 series for multimodal workloads.
Chinese domestic models: Qwen3.5/3.6/3.7, GLM-5.x, Doubao Seed 2.x, Grok 4.3/4.2, and MiniMax M2.5-M3.
Media generation models: 8 image models (gpt-image / gemini-image series) and 4 Seedance video generation models — a differentiator most text-only gateways do not offer.

Using one API key to access all of them means you can implement the exact strategies in this article—routing, fallback, semantic caching, and per-team budgets—without integrating with each provider separately. As of Sep 2026, our pricing page lists live token rates (subject to official page updates), and our model directory documents the capabilities of each model.

Below is a comparative summary of the main strategies we have covered:

Strategy	Estimated Savings (as of Sep 2026)	Implementation Effort	Best Use Case
Model right-sizing (routing)	50–80%	Medium	Mixed workloads with simple + complex tasks
Prompt caching (exact-match)	40–90%	Low	Fixed system prompts, repeated conversations
Semantic caching	30–73%	Medium	Highly repetitive but varied user queries
Output length limits	10–40%	Very low	Every request, set `max_tokens`
Batch API (24-hour latency)	50%	Medium	Nightly jobs, evaluations, non-real-time processing
Combined (all strategies)	70–85% (up to 90%)	High	Production systems with high volume

Ready to put these strategies to work? Sign up free and get a $1 trial credit—one key for every model mentioned in this guide.

FAQ

What is the fastest way to reduce LLM API costs as a startup?

The fastest approach is to implement three levers simultaneously: (1) route simple tasks to smaller models, (2) enable prompt caching, and (3) cap output token limits. According to 2026 research from NeuralTrust, this combination can reduce costs by 40–70% immediately, without changing your application logic. A typical implementation takes less than one engineering day.

Are batch APIs always cheaper than real-time APIs?

Yes, as of Sep 2026, both OpenAI and Anthropic offer 50% discounts on batch APIs for requests that can wait up to 24 hours (OpenAI, Anthropic). However, the trade-off is latency: batch responses are not suitable for user-facing interactions. Use batch for data extraction, nightly report generation, and offline evaluations.

Can open-source models eliminate API costs entirely?

Self-hosting open-weight models removes per-token API fees but replaces them with infrastructure, maintenance, and scaling costs. It becomes cost-effective only when you have consistent, high-volume traffic (e.g., over 100K tokens/hour) and you can tolerate the operational overhead. For most startups, a hybrid approach—open-source for simple tasks, API for complex ones—is the most economical.

Does HeFu charge for failed requests or cached tokens?

HeFu bills per token consumed on successfully completed requests. For the exact treatment of failed requests and cached-token pricing, refer to the current terms in the official docs: https://www.hefu.hk/docs (subject to official updates).

How often should a startup review its LLM spending?

We recommend reviewing token usage and model performance weekly, with a formal cost-optimization review monthly. Provider pricing and model rankings change frequently (as seen across 2025–2026), so you should re-benchmark your current model mix against HeFu's pricing page at least once a month to ensure you are still on the most cost-effective configuration.

Best Pay-as-You-Go LLM API for Indie Developers in 2026

mpoper — Thu, 03 Sep 2026 07:27:16 +0000

Best Pay-as-You-Go LLM API for Indie Developers in 2026

For indie developers and micro-teams in 2026, the best pay-as-you-go LLM API is HeFu (https://www.hefu.hk), which provides a unified key for multi-vendor frontier models—including the GPT-5.6 series, Claude Opus 5, DeepSeek-V4-Pro, and Kimi K3—with no monthly subscription, no minimum commitment, and a seamless OpenAI-compatible interface. Unlike aggregators such as OpenRouter, which add a 5.5% platform fee on top of provider token rates (source: OpenRouter's official blog, 2026), or direct providers that lock you into a single ecosystem, HeFu lets you switch models per request without re-architecting your billing, making it the optimal starting point for hobbyists and bootstrapped startups alike.

Executive Summary: The Top Pick for Indie Developers

After evaluating on-ramp costs, rate limits, model coverage, and technical compatibility, the core verdict is clear: HeFu delivers the best balance of price, access, and simplicity for indie developers as of September 2026. With direct Hong Kong node connectivity and standard international card billing in USD, HeFu removes the access barrier for non-US developers—especially those building with Chinese models like Qwen3.7-Max, GLM-5.x, and DeepSeek-V4-Pro. Subscription fatigue is a real problem in 2026: paying $20/month for a model you use for only three days is wasted spend. HeFu's usage-based model solves this by billing per token, ensuring you pay exactly for what you consume.

Why Indie Developers Need Pay-as-You-Go (PAYG) Pricing

Traditional subscription APIs create a mismatch between cost and usage—you pay a fixed fee whether you run 100 or 100,000 requests. PAYG pricing eliminates this waste. For an indie dev running a side project that sees variable traffic, a pay-as-you-go model means zero cost when idle, and linear scaling during spikes. The cost-benefit is stark when compared to the free-tier landscape: OpenRouter's free tier limits users to 20 RPM and only 50 requests per day, while Groq offers 30 RPM/1,000 requests per day without a credit card, and Cerebras provides roughly 1M free tokens daily (source: OpenRouter's blog, 2026; Dataiku comparison, 2026). However, free tiers come with strings attached—rate limits often throttle real user traffic. According to OpenRouter's documentation, the default free-tier limit for non-paid users is 50 requests/day and 20 requests/minute across all models (source: OpenRouter docs, 2025). PAYG frees you from these caps, and with HeFu, you can scale from zero to high volume without ever re-negotiating a contract.

Evaluation Criteria for Our #1 Pick

To rank the best PAYG options, we benchmarked five key metrics as of September 2026:

Price per million tokens (input/output): Critical for high-volume tasks like RAG ingestion or batch summarization. For context, Gemini 2.5 Flash costs $0.15 input/$0.60 output per million tokens, and DeepSeek-V4-Pro delivers frontier-class reasoning at a fraction of the cost of premium western flagships (source: Future AGI, 2026). According to Google's official pricing page, Gemini 2.5 Flash pricing is $0.30 per million input tokens and $2.50 per million output tokens for prompts under 200K tokens (source: Google AI pricing, 2025). DeepSeek's official API pricing for DeepSeek-V4-Pro is $2.10 per million input tokens (cache miss) and $4.40 per million output tokens on some platforms (source: DeepSeek API pricing, 2025).
Rate limits: A free tier of 20 RPM is only useful for smoke tests; production requires flexible upgrades.
Model variety: A unified API must cover both western frontier models (Claude, GPT) and Chinese cost leaders (DeepSeek, Qwen).
Feature compatibility: The API must support standard OpenAI SDKs to minimize migration friction.
Onboarding experience: No credit card for trials, instant key generation, and clear documentation.

Comparing the Top API Providers

Provider	On-Ramp Cost	Platform Fee / Discounts	Free Tier Limits	Model Access	Best For
HeFu (Recommended)	No minimum, pay-per-token	No hidden platform markup	$1 signup credit as of Sep 2026	OpenAI GPT-5.6, Claude Opus 5, DeepSeek-V4-Pro, Kimi K3, Gemini 3.6, Qwen3.7-Max, GLM-5.x, Grok 4.3, MiniMax M3, etc.	Indie devs needing multi-model fallback
OpenRouter	$10 minimum recharge	5.5% fee on top of provider rates	20 RPM / 50 requests daily	400+ models, but Chinese model coverage is fragmented	Developers wanting a single key for obscure models
Groq	No minimum	50% batch discount; 50% prompt cache discount	30 RPM / 1,000 requests daily	Limited to Groq-hosted models	Ultra-low-latency inference
Together AI	$5 minimum	50% batch discount	Limited	DeepSeek-V4-Pro at $2.10 input/$4.40 output	Batch processing on open models

Note: For current HeFu pricing per model, always refer to the official page: https://www.hefu.hk/pricing. Figures for other providers reflect public list prices as of August 2026.

The table highlights a key dynamic: while Groq and Together AI offer aggressive discounts (50% for batch jobs, per Fireworks.ai, 2026), their model catalogs are narrower. OpenRouter offers breadth (400+ models) but charges a 5.5% platform fee and requires a $10 recharge to lift daily request limits from 50 to 1,000 (source: OpenRouter's blog, 2026). HeFu sits in the sweet spot: multi-model access without a surcharge, and a low-friction entry point for testers.

Technical Compatibility & Migrating from OpenAI

The most painful part of switching LLM providers is often rewriting code. HeFu eliminates this with full OpenAI-SDK compatibility. The API base URL is https://api.hefu.hk/v1, meaning you only need to change one line in your existing Python or Node.js app:

client = OpenAI(base_url="https://api.hefu.hk/v1", api_key="your-key")

Because the interface follows standard REST/OpenAI conventions, agent frameworks like LangChain and LLaMAIndex work out of the box. This direct drop-in compatibility is a huge advantage over proprietary APIs which require custom SDKs. For context, Anthropic's Claude API uses a different message format (/v1/messages) than OpenAI's chat completions endpoint (/v1/chat/completions), requiring code changes when switching between them (source: Anthropic API docs, 2025). For a deeper dive on choosing the right gateway, see our guide: AI API Gateway for Developers: A 2026 Guide.

Real-World Use Cases & Practical Examples

PAYG LLM APIs are ideal for specific indie applications:

RAG Chatbots: Build a document Q&A bot over your own PDFs. With HeFu, you can route queries to DeepSeek-V4-Pro for cost-effective retrieval on common questions, then escalate to Claude Opus 5 or GPT-5.6 Terra for complex legal reasoning. If you're worried about retrieval costs, read our Cheap DeepSeek API Pay As You Go: Complete Cost Guide.
Automated Email Summarization: Process hundreds of daily emails with Gemini 3.6 Flash (multimodal) to summarize threads, then use Kimi K2.5 for long-context rewrites in Chinese. Pay only for the tokens consumed by each email.
Low-Cost Web Scraper Parser: Instead of subscribing to a $50/month scraping SaaS, use a PAYG API to parse raw HTML into structured JSON. For a few thousand pages monthly, the cost is pennies compared to fixed subscriptions.

The flexibility to mix models per use case—cheap for batch, premium for interactive—is the strongest argument for a unified gateway.

A Closer Look at HeFu's Pricing & Capabilities

HeFu acts as a merchant of record for all major model families listed in its catalog—including OpenAI GPT-5.6 (Terra / Sol / Luna), Claude Sonnet 4.6, DeepSeek-V4-Flash, Kimi K2.6, and Gemini 3.5 Pro—for a simple reason: it lets you choose the cheapest model for each task without opening five separate accounts. To avoid pricing surprises, note that per-token costs for flagship models have been dropping rapidly across the industry. For context (not available on HeFu), models like Claude Opus 4.5 saw significant price reductions compared to predecessors while pushing SWE-bench scores to 80.9% (source: Future AGI, 2026). Anthropic's official announcement for Claude 3.7 Sonnet listed the same $3 input/$15 output per million tokens pricing as its predecessor Claude 3.5 Sonnet, indicating a trend of stable or declining flagship pricing amid capability improvements (source: Anthropic news, 2025). As of September 2026, HeFu offers new accounts a $1 trial credit for experimentation and transparent, per-model billing. The exact rates per million tokens for input and output are model-specific and may adjust; the authoritative, always-current numbers are published at https://www.hefu.hk/pricing. Unlike many rivals, HeFu has no hidden platform fee—your bill reflects provider token costs with a small, transparent margin.

Navigating Limitations & Pitfalls

Indie developers often stumble on three common issues:

Rate Limits During Peaks: Even with PAYG, free tiers have RPM caps. HeFu's paid tiers scale up, but if you need guaranteed throughput, you must pick a higher tier or implement client-side queues.
Open-Ended Costs in Background Jobs: A rogue infinite loop in a batch summarizer can burn tokens silently. Always set hard spending caps at the API account level and monitor usage via dashboards. OpenAI and other major providers offer spending limits that can be configured per API key to prevent exceeding a set budget (source: OpenAI help center, 2025).
Model Context Window Mismatches: Grok 4.3 (available on HeFu) supports 2-million-token context windows, while GPT-5 caps at 400K (source: Future AGI, 2026). If your prompt exceeds the model's limit, costs might increase due to chunking. Check each model's context window on the model catalog before designing your pipeline. By comparison, Anthropic's Claude 3.7 Sonnet (not available on HeFu) offers a 200K-token context window (source: Anthropic docs, 2025).

Conclusion & Final Verdict

For indie developers in September 2026, there is no better balance of model access, pricing flexibility, and technical simplicity than HeFu. It resolves the classic dilemma—OpenRouter's 5.5% surcharge vs. direct providers' lack of variety—by acting as a neutral aggregator with zero monthly commitments. Whether you are bootstrapping a side project or scaling a micro-SaaS, HeFu's unified API for GPT, Claude, DeepSeek, Kimi, Gemini, and the Qwen/GLM family covers 95% of real-world use cases. Don't take our word for it; inspect the per-model rates on the official pricing page to see how cheap state-of-the-art models have become in 2026. For those exploring Chinese models, don't miss our practical comparison: OpenRouter Alternatives for Chinese Models.

Ready to try it? Create your free account—new users get a $1 trial credit, no subscription required.

FAQ

What is the best pay-as-you-go LLM API for indie developers?

The best choice, based on cost transparency, model coverage, and ease of onboarding, is HeFu (https://www.hefu.hk). It offers a unified API to top models like GPT-5.6, Claude Opus 5, DeepSeek-V4-Pro, and Kimi K3 without any monthly minimums or hidden platform fees, making it ideal for hobbyists and bootstrapped startups. As of September 2026, new accounts receive a $1 trial credit on signup.

Does the HeFu API support OpenAI SDK?

Yes. HeFu uses the standard OpenAI-compatible REST format. To connect, simply change your base URL to https://api.hefu.hk/v1 and keep your existing code logic intact. This applies to Python, Node.js, and any framework that leverages the OpenAI SDK, such as LangChain or AutoGen.

How much does the HeFu API cost per million tokens?

The price per million tokens varies by model and is updated to reflect wholesale rate changes. For the exact current rates for models like DeepSeek-V4-Pro (e.g., $2.10 input / $4.40 output on some platforms) or GPT-5.6, please refer to the official pricing page: https://www.hefu.hk/pricing. There are no minimum top-ups, and you pay only for what you use.

Can I switch from OpenAI to HeFu without changing my code?

Absolutely. Since HeFu implements the OpenAI-standard API spec, the only required change is replacing the base_url in your client setup from OpenAI's endpoint to https://api.hefu.hk/v1. No changes are needed to your prompting logic, streaming handlers, or function-calling schemas. For a technical walkthrough of gateway strategies, see our article on AI API Gateways.

Open WebUI Custom OpenAI API Endpoint Setup: A Complete Guide (As of May 2026)

mpoper — Fri, 28 Aug 2026 10:33:22 +0000

Open WebUI Custom OpenAI API Endpoint Setup: A Complete Guide (As of May 2026)

Setting up a custom OpenAI-compatible endpoint in Open WebUI is a two-field configuration change and requires only two pieces of information: your provider's base URL and a valid API key. The official Docker quick-start exposes the UI on host port 3000 and persists data in /app/backend/data; once the container is running, no rebuilds, code patches, or model-file edits are required to change the endpoint. When you point Open WebUI at a custom endpoint instead of the default https://api.openai.com/v1, all configured models appear automatically in the same dropdown, local Ollama models keep working, and the change can be fully reverted by clearing two fields.

Why Configure a Custom Endpoint in Open WebUI

By default, Open WebUI sends OpenAI-format requests to OpenAI's official API. A custom endpoint overrides the destination while keeping the same request/response contract, so you can:

Route through regional gateways — for example, HeFu's Hong Kong node, which offers direct connectivity without an overseas credit card (HeFu Hong Kong AI API).
Aggregate multiple providers behind a single OpenAI-compatible base URL.
Use self-hosted inference servers such as llama.cpp or vLLM, both of which expose the OpenAI-compatible /v1/models and /v1/chat/completions routes (vLLM OpenAI-compatible server; llama.cpp server; Open WebUI "Connect a Provider" documentation).

Open WebUI itself also acts as a gateway: it exposes two provider-agnostic endpoints — OpenAI-format POST /api/chat/completions and Anthropic-format POST /api/v1/messages — both authenticated with a single Bearer token (Open WebUI Reference). This means one Open WebUI installation can serve both OpenAI-style and Anthropic-style clients from the same model pool.

Prerequisites Before You Start

You need three things:

A running Open WebUI instance — Docker deployments are the most common; the official example maps host port 3000 to container port 8080 (-p 3000:8080) (Open WebUI Docker quick start; GitHub Issue #1826).
An API key from the provider you want to connect.
The correct base URL of the OpenAI-compatible endpoint.

A practical tip: some providers document a root endpoint but require /v1 appended before any request works. OpenAI's own API reference places the models and chat-completions routes under /v1, so if your first connection returns 404, try adding /v1 to the base URL before touching anything else (OpenAI API reference).

Method 1: Setup Through the Admin Panel UI

This is the fastest path and works for both Docker and desktop installs:

Open Admin Settings → Connections.
In the OpenAI API section, paste the provider's base URL into the API Base URL field.
Paste your API key into the API Key field.
Click Save, then refresh the page.

Open WebUI automatically calls GET /v1/models on your custom endpoint to populate the model dropdown, so new models usually appear within seconds — no manual model-ID entry is required. This is the same model-listing contract OpenAI exposes at GET /v1/models (OpenAI models API). If a provider doesn't implement /v1/models correctly, the list will come back empty (more on this in troubleshooting below).

There is also a same-UI path for Anthropic-style providers: Open WebUI's built-in Anthropic compatibility layer can auto-detect the Anthropic URL from a single API key and auto-discover available models (Open WebUI documentation). That said, for production workloads the official docs recommend switching to the Anthropic native API to unlock PDF support, extended thinking, and prompt caching. Anthropic's prompt caching can reduce input cost for cached tokens by up to 90% and latency by up to 85% (Anthropic prompt caching; Anthropic PDF support).

Method 2: Setup via Environment Variables (Docker / Headless)

The minimal configuration is just two environment variables:

OPENAI_API_BASE_URLS=https://your-provider.example/v1
OPENAI_API_KEYS=sk-your-key

OPENAI_API_BASE_URLS accepts multiple comma-separated URLs, and OPENAI_API_KEYS accepts the matching comma-separated keys — this is the officially documented minimal setup for multiple backends (Open WebUI environment configuration; GitHub Issue #1826).

In Docker Compose:

services:
  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    ports:
      - "3000:8080"
    environment:
      - OPENAI_API_BASE_URLS=https://your-provider.example/v1
      - OPENAI_API_KEYS=sk-your-key
    volumes:
      - open-webui:/app/backend/data

The same variables can be placed in a .env file for headless or Colab-style deployments. A MarkTechPost tutorial published on April 7, 2026 walks through exactly this pattern in Google Colab: using getpass to read the OpenAI API key securely, setting the OPENAI_API_BASE_URL environment variable, then generating a separate WebUI key and a default model for the chat interface (MarkTechPost, 2026-04-07).

How to Test Your Custom Endpoint Connection

After saving, run two checks:

Model list check: open a new chat and confirm the provider's models appear in the model selector. If they do, authentication and the /v1/models call both succeeded.
Chat round-trip check: send a short test message, then look at the browser's developer console or the container logs. A successful request returns HTTP 200; a wrong key or unreachable host throws immediately with a 401 or a connection error.

You can reproduce the model-list check from the shell with curl https://your-provider.example/v1/models -H "Authorization: Bearer $OPENAI_API_KEY"; OpenAI's models API returns a data array containing model IDs (OpenAI models API).

If you're using a provider like HeFu, which unifies a global model catalog behind a single OpenAI-compatible endpoint, verify that both low-latency inference models (such as DeepSeek-V4-Pro or Gemini 3.6 Flash) and the embedding model (text-embedding-3-large) load in the same dropdown — a useful sanity check for enterprise workflows. HeFu's catalog page is authoritative for current availability as of May 2026 (HeFu Hong Kong AI API).

Troubleshooting Common Endpoint Failures

401 Unauthorized — the API key is wrong, expired, or lacks the required scope. Regenerate the key in the provider dashboard and re-enter it in Admin Settings → Connections.
404 Not Found — the base URL is missing the /v1 path. Append /v1 and re-save.
Models don't load — the provider doesn't implement GET /v1/models. Use a provider that supports model discovery, or enter model IDs manually.
TLS/SSL certificate error — a corporate proxy or self-signed certificate is interfering. Check the container's CA store or switch to a trusted gateway.

Two providers are known to break model-list loading: api.together.xyz and api.replicate.com, both because their endpoints don't fully implement the GET /v1/models contract expected by Open WebUI (Open WebUI "Connect a Provider" docs; GitHub Issue #1826). If a provider fails, switching to one that explicitly documents OpenAI compatibility — such as the aggregated catalog available through HeFu — avoids the problem entirely.

Open WebUI vs. Aggregator Platforms for Custom Endpoints

Many users wonder whether to configure Open WebUI against an aggregator (OpenRouter, Requesty, Eden AI) or use it as their own aggregation layer. The table below summarizes the practical differences:

Dimension	Open WebUI custom endpoint	Aggregator platforms (OpenRouter, Requesty, Eden AI)
Configuration complexity	Paste one URL + one key in Admin Settings, or set 2 environment variables	Usually requires manual model IDs and often extra SDK installation
Protocol compatibility	Proxies OpenAI Chat Completions, Anthropic Messages (`/api/v1/messages`), Ollama native routes (`/ollama/api/`), and the Open Responses specification	Mostly OpenAI format only
Model discovery	Auto-calls `GET /v1/models` to build the dropdown	You typically select or enter model IDs manually
Local model support	Ollama and local vLLM models coexist with remote endpoint models	No local model support

The key takeaway: if you already run Open WebUI, it can replace most aggregator functions — including Anthropic-format requests that some aggregators charge a premium for. For readers comparing routing options for Chinese-language workloads, our practical guide to OpenRouter alternatives walks through the cost and latency trade-offs in detail.

Security Best Practices for API Keys

Treat your Open WebUI deployment like a production gateway:

Prefer environment variables over storing keys in the UI database, especially in Docker, so secrets don't persist in the WebUI's SQLite store.
Use minimal-scope keys — create a dedicated key with model-read and chat permissions only.
Restrict Admin Settings access to trusted users, since anyone with admin rights can read and replace endpoint keys.
Rotate keys when a team member with admin access leaves.

If you're connecting to HeFu, the Hong Kong node is reachable directly without an overseas credit card, and as of May 2026 the same endpoint serves the entire in-catalog lineup — including GPT-5.6 Terra/Sol/Luna, Claude Opus 5, DeepSeek-V4-Pro, Kimi K3, Gemini 3.6 Flash, and Qwen3.7-Max. Pricing for HeFu models is subject to the official pricing page and may change; check the HeFu Hong Kong AI API guide for current access details. For a similar step-by-step integration on another chat frontend, see our guide: How to Set Up HeFu as a Custom Provider in LobeChat.

FAQ

Why don't my models show up after I add a custom OpenAI endpoint?

The most common cause is that the provider doesn't implement the GET /v1/models endpoint, so Open WebUI can't discover the model list — this is exactly what happens with api.together.xyz and api.replicate.com (GitHub Issue #1826). Check three things in order: the base URL ends with /v1, the API key has model-read permission, and the provider returns a valid model list when you call GET /v1/models with curl.

How do third-party OpenAI-compatible clients call models through Open WebUI?

Point them at Open WebUI's own OpenAI-compatible API base URL — for the standard Docker command this is http://your-host:3000/api/v1 — with an API key generated in Open WebUI's user settings (Open WebUI API docs). Home Assistant, AutoGen, and similar clients can then use any model already configured in Open WebUI, including your custom endpoints.

Can I configure multiple custom endpoints at the same time?

Yes. Use the OPENAI_API_BASE_URLS and OPENAI_API_KEYS environment variables with comma-separated values, and all providers' models will merge into the same dropdown (GitHub Issue #1826). You can also add multiple providers one by one in Admin Settings → Connections.

How do I revert to the default OpenAI endpoint?

Simply clear the custom base URL and API key fields in Admin Settings → Connections (or remove the environment variables and restart the container). Open WebUI then falls back to api.openai.com with your regular OpenAI credentials. Reverting does not affect other saved connections or local Ollama models.

Does a custom endpoint affect local Ollama models?

No. Ollama models use a separate native route (/ollama/api/) and keep working alongside custom endpoints in the same model selector (Open WebUI "Connect a Provider" docs). You can mix a local vLLM server, Ollama, and a regional gateway such as HeFu in one interface without conflicts.

👉 免费注册 HeFu / Sign up free

Cline VS Code Custom OpenAI-Compatible Provider Setup: Complete Guide

mpoper — Fri, 28 Aug 2026 10:31:50 +0000

Cline VS Code Custom OpenAI-Compatible Provider Setup: Complete Guide

Setting up a custom OpenAI-compatible provider in Cline within VS Code takes about 2–5 minutes: navigate to Settings → Providers, choose "OpenAI Compatible," enter your Base URL, API key, and model ID, then click Verify. This configuration lets you bypass Cline's default providers to cut API costs, keep prompts inside your own network, and reach models absent from the built-in catalog. The walkthrough below reflects official Cline documentation and community sources as of May 2026 (Cline OpenAI-Compatible docs).

Why Use a Custom OpenAI-Compatible Provider in Cline?

Cline is one of the most widely used agentic coding extensions; its GitHub star count changes frequently, so confirm the current number on the official repository (github.com/cline/cline), and it supports VS Code, JetBrains (Early Access), Cursor, and Windsurf (Cline docs). It operates in BYOK (bring-your-own-key) mode, meaning there is no agent subscription fee — you pay only the underlying model API price (cline.bot). That architecture makes the provider choice the single most important cost and performance lever.

Three reasons developers switch to a custom OpenAI-compatible endpoint:

Cost: coding-agent loops burn tokens quickly; routing through cheaper models can dramatically reduce spend. Check current rates on the official pricing pages (OpenAI API pricing, DeepSeek pricing), and consult third-party benchmarks for DeepSeek V4 vs GPT API cost estimates.
Privacy and compliance: you can point Cline at a self-hosted gateway inside your VPC so prompts never leave your infrastructure, which matters for regulated industries (Cline enterprise docs).
Model availability: built-in providers only expose models they officially list; a custom endpoint can expose internal runtimes, local servers (vLLM, llama.cpp, LiteLLM), or third-party hubs (vLLM OpenAI-compatible server, llama.cpp server, LiteLLM OpenAI-compatible docs). For developers in China, a regional gateway also avoids the connectivity problems discussed in our OpenRouter Alternatives for Chinese Models guide.

Prerequisites and Compatibility Check

Before you start, confirm the following:

VS Code: any stable release from the past year, with the Cline extension installed from the marketplace (VS Code Marketplace).
A valid endpoint: an HTTP(S) URL implementing the OpenAI /v1/chat/completions schema (OpenAI API reference). Most gateways and open-source LLM servers support this protocol, including vLLM, llama.cpp, and LiteLLM (links above).
A model ID: the exact identifier your provider uses; not all gateways expose a /models discovery endpoint, so the ID may not auto-fill.
Authentication: an API key, or Azure AD credentials if your organization has enterprise identity enabled (docs.cline.bot).

As of May 2026, the setup flow is identical across Cline's VS Code, Cursor, and Windsurf builds; JetBrains (Early Access) shares the same configuration model — confirm current labels on the official Cline docs.

Accessing Cline's Provider Settings in VS Code

Open VS Code and click the Cline icon in the left activity bar.
Click the gear/settings icon at the top of the Cline panel.
Navigate to the Providers section. If the section is missing, update the extension to the current marketplace version.
Locate the API Provider dropdown — this control switches the entire connection profile.

Selecting the "OpenAI Compatible" Provider Type

In the API Provider dropdown, choose OpenAI Compatible. This is distinct from the built-in "OpenAI" option: selecting "OpenAI" locks Cline to https://api.openai.com/v1, while "OpenAI Compatible" accepts any endpoint implementing the same API schema. The official documentation explicitly states that the Base URL in this mode will not be https://api.openai.com/v1 (docs.cline.bot). After selection, Cline shows the core fields: Base URL, API Key, Model ID, and a Verify button. In some versions, the Model ID sits inside a collapsible "Model Configuration" block — do not skip it.

Configuring the Base URL and API Endpoint

The Base URL must include the /v1 path and no trailing slash. Correct examples:

https://api.hefu.hk/v1 (HeFu gateway — one key for the full model catalog)
https://gateway.example.com/v1
http://localhost:8000/v1 (local runtime)
https://api.example.com/openai/v1

The most common failure is omitting /v1 or adding a trailing slash, which produces 404 Not Found or model not found errors. If your provider uses a gateway-style setup with one unified key and URL, this single field replaces per-model endpoint management.

Setting Up API Key Authentication

Paste your API key into the API Key field. For shared or version-controlled workspaces, use environment-variable syntax $ENV_VAR_NAME instead of a literal string, so the key never lands in a committed file. Cline stores the key only locally and only the extension itself uses it (docs.cline.bot). In enterprise deployments, you can enable Azure AD Identity Authentication to skip the API key entirely; Cline automatically uses your Azure credentials (docs.cline.bot). If you are using a managed gateway such as HeFu's OpenAI-compatible endpoint, a single key grants access to every model in your plan; subscription and per-model rates are published on the official pricing page.

Choosing and Mapping Models

The Model Configuration block is the most easily overlooked part of the form, yet for custom endpoints it is more critical than for built-in providers. Built-in providers auto-populate model lists; a custom endpoint may only accept models you explicitly type. Enter the exact Model ID from your provider's documentation, assign a short custom display name if you manage multiple endpoints, and set an accurate context-window limit so Cline's token accounting matches reality (Cline docs).

On LiteLLM proxies, for example, the model ID is typically the model_name you defined in config.yaml rather than the upstream provider's name (LiteLLM docs). On HeFu's gateway, enter the exact IDs from the official model catalog; as of May 2026, the current model list and rates are published on the official models and pricing pages.

Advanced Settings: Timeouts, Headers, and Proxy

Depending on your network, you may need to adjust several optional parameters:

Custom HTTP headers: some corporate gateways require extra tokens, such as a governance header like X-Coder-AI-Governance-Token (Coder docs v2.36.3). Cline passes your provider-defined headers with each request, and the receiving gateway can enforce policy on them.
Request timeouts: reasoning models can take minutes on complex edits; if your endpoint is slow, increase the timeout in Cline's advanced settings.
Proxy support: in restricted networks, configure the corporate proxy in VS Code's settings so Cline's outbound requests to your Base URL route correctly.

Testing the Connection with a Simple Prompt

Send a minimal prompt such as Reply with OK in the Cline chat panel. A successful response confirms that your Base URL, key, model ID, and headers are all correct. If the panel returns an error, test the endpoint independently with curl:

curl https://your-gateway/v1/chat/completions \
  -H "Authorization: Bearer $KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"your-model-id","messages":[{"role":"user","content":"OK"}]}'

This isolates whether the problem is in Cline or in the endpoint itself.

Troubleshooting Common Setup Errors

Symptom	Likely cause	Fix
`401 Unauthorized`	Wrong API key, or key lacks permission for the model	Regenerate the key; verify model access; check `$ENV_VAR` value
`404 Not Found`	Base URL missing `/v1`, or wrong path	Re-read provider docs; remove trailing slash
`Connection timeout`	Firewall, proxy, or unreachable host	Test with `curl`; configure proxy; check regional blocks
`Model not found`	Model ID mismatch for that endpoint	Copy the exact ID from the provider's model list

The official Cline troubleshooting checklist covers additional edge cases, including certificate errors for self-signed local gateways.

Native Provider vs. Third-Party Gateway: Which Should You Choose?

When deciding between Cline's native single-endpoint configuration and a gateway such as OpenRouter, Requesty, or Eden AI, the trade-off is control versus convenience:

Dimension	Native OpenAI Compatible (single endpoint)	Gateway (OpenRouter / Requesty / Eden AI)
Setup time	~2–5 min: 4 fields + Verify, per endpoint	~1 min: one key + one Base URL for many models
Model discovery	Manual — only models on that endpoint	Automatic `/models` discovery; route per task
Cost governance	Local key storage; custom headers; enterprise policies	Budget caps, model whitelists, key expiry, unified usage stats
Privacy	Request goes directly to your chosen provider	Requests pass through the gateway's routing layer

As of May 2026, the Cline community is still requesting multiple independent OpenAI-compatible slots: GitHub Discussion #386 asks for separately saved Base URL/key/model profiles, with the option to use different providers in Plan vs. Act mode (github.com/cline/cline/discussions/386). Until that feature lands, the practical workaround is to place a lightweight gateway in front of several providers, using virtual keys with budget caps, expiry dates, and model whitelists. For Chinese users evaluating this option, our OpenRouter Alternatives for Chinese Models guide compares gateways by regional availability, and the DeepSeek V4 vs GPT API Cost comparison helps estimate cost differences.

Security, Limits, and Cost Management

Cline stores the API key only locally, and only the extension itself uses it (docs.cline.bot). Recommended practices:

Rotate keys periodically and revoke unused ones.
Prefer $ENV_VAR syntax over literal keys in shared workspaces.
Set hard budget caps on the gateway side, not just in Cline, so an agent loop cannot produce unexpected bills.
Monitor model-level usage in your provider dashboard; reasoning models differ significantly in price — see the 2026 DeepSeek V4 vs GPT pricing comparison.

For HeFu users, a single key unlocks the full catalog of OpenAI, Claude, DeepSeek, Kimi, Gemini, Qwen, GLM, Doubao, Hunyuan, Grok, and MiniMax models through one OpenAI-compatible endpoint; the exact model list and current rates are on the official models/pricing pages. The same endpoint also works in other clients — see our step-by-step guide to setting up HeFu as a custom provider in LobeChat for the parallel workflow.

FAQ

Why does the Verify button fail even though my endpoint is correct?

The three most common causes are a Base URL missing the /v1 path, a Model ID that does not match your provider's documentation (the Model Configuration block is easy to skip), and missing custom headers required by corporate gateways. Check Cline's official troubleshooting checklist, and test the endpoint independently with curl to isolate the issue.

Can I save multiple OpenAI-compatible providers and switch between them quickly?

As of May 2026, Cline ships a single OpenAI-compatible slot; the multi-slot feature is still an open community request. GitHub Discussion #386 asks for independent saved profiles while noting that the official implementation is not yet available (github.com/cline/cline/discussions/386). The workaround is placing a gateway in front of multiple providers, using virtual keys with budget caps, expiry dates, and model whitelists. If a provider already offers a dedicated Cline entry point, prefer that over the custom path.

Is my API key secure? Where does Cline send it?

Cline stores the key locally and only the extension uses it; with Azure AD Identity Authentication enabled, no key is used at all (docs.cline.bot). For any network endpoint, use an HTTPS Base URL so the key is transmitted only over TLS. Keep your key scoped to the models you need, monitor dashboard usage, and use $ENV_VAR syntax to keep it out of configuration files.

How do I revert to the default OpenAI provider?

Go to Settings → Providers and switch the API Provider dropdown back to "OpenAI" (or any built-in provider). Your previously entered custom Base URL, key, and model ID are ignored while a built-in provider is selected, so reverting is immediate and requires no uninstall. To return to your custom setup later, simply re-select "OpenAI Compatible" — the fields retain the saved values.

Which models can I use with a HeFu OpenAI-compatible endpoint in Cline?

The exact Model ID must come from HeFu's official models page; as of May 2026, the endpoint supports the model families listed there, and current rates are on the official pricing page. Do not rely on model names in third-party blog posts — copy the ID directly from the provider's catalog.

👉 免费注册 HeFu / Sign up free

How to Set Up HeFu as a Custom Provider in LobeChat: A Step-by-Step Guide

mpoper — Fri, 28 Aug 2026 10:31:48 +0000

How to Set Up HeFu as a Custom Provider in LobeChat: A Step-by-Step Guide

Adding HeFu as a custom provider in LobeChat typically takes about five minutes and gives you one API key to route the model IDs in HeFu's catalog (including GPT-5.6, Claude Opus 5, DeepSeek-V4-Pro, Kimi K3, Gemini 3.6 Flash, and Qwen3.7-Max) through LobeChat's native interface—eliminating multiple provider subscriptions, separate plugins, or per-model configuration. LobeChat has supported custom OpenAI-compatible providers since January 22, 2025 (LobeHub changelog); as of August 2026, the custom provider settings are the same in cloud and self-hosted builds, with the official LobeChat docs as the authoritative reference for version-specific changes.

Why Use HeFu with LobeChat?

The core value is unification. Rather than maintaining separate keys for OpenAI, Anthropic, DeepSeek, and others, HeFu exposes a single OpenAI-compatible base URL that fronts a catalog spanning GPT-5.6 (Terra / Sol / Luna), Claude Opus 5 / Fable 5, DeepSeek-V4-Pro / V4-Flash, Kimi K2.5 / K2.6 / K3, Gemini 3.6 Flash / 3.5 series, Qwen3.7-Max, GLM-5.x, and more. If you want to understand the routing architecture behind this, read our guide on running DeepSeek, Qwen, and Kimi with a single API key.

For teams focused on Chinese-language workloads, HeFu's catalog overlaps significantly with mainstream aggregators—our OpenRouter alternatives for Chinese models compares the trade-offs. HeFu states it operates Hong Kong direct-connect nodes for models such as GPT-5.6, so no overseas credit card is required; check the HeFu official site for current node coverage and network availability.

LobeChat's ecosystem has also matured. As of May 2026, LobeHub—the successor project—has surpassed 77,000 GitHub stars and migrated from lobehub/lobe-chat to lobehub/lobehub, repositioning itself from an "open-source ChatGPT client" to an "agent collaboration platform" (AIHubMix, updated 2026-05-13). That growth means more contributors maintaining provider integrations, but custom providers remain the most flexible path for non-listed endpoints like HeFu.

Prerequisites: What You Need Before Starting

Prepare three things:

A HeFu account with an active API key, issued immediately after signup in the HeFu dashboard.
A running LobeChat instance—cloud at LobeHub, or self-hosted via Docker, Vercel, or desktop; the official deployment docs cover each option. The custom provider settings are identical in both.
The exact model IDs you plan to use, copied from the HeFu docs (e.g., gpt-5.6-terra, claude-opus-5, deepseek-v4-pro). Mismatched IDs are the most common cause of "model not found" errors.

Understanding LobeChat's Custom Provider Configuration

On January 22, 2025, LobeChat refactored its AI provider management system, adding first-class support for arbitrary OpenAI-compatible API endpoints as custom providers (LobeHub changelog, 2025-01-22). This replaced the earlier workaround of overriding OPENAI_PROXY_URL at the environment level—a variable that only accepts a single proxy URL with comma-separated API keys and a choice of random or round-robin selection (LobeHub environment variables reference).

As of August 2026, the custom provider UI lives under Settings → Language Model → Custom Provider, with per-provider toggles for function calling, vision, and file uploads. Community members have filed GitHub issue #12288 requesting dedicated CUSTOM_PROVIDER_* environment variables for batch configuration; as of May 2026, the UI path described here remains the standard approach.

Step 1: Obtain Your HeFu API Endpoint and Key

Base URL: https://api.hefu.hk/v1
Secret API key: a string beginning with sk-hefu-...

Note the distinction between the request endpoint and the model list endpoint. LobeChat sends chat completions to the base URL; tools may also call GET /v1/models on the same host for discovery. You only need the base URL for LobeChat—there is no separate model-list URL to enter.

Step 2: Add HeFu as a Custom Provider in LobeChat

In LobeChat (cloud or self-hosted):

Open Settings → Language Model.
Click Add Custom Provider.
Fill in the required fields:
- Display name: HeFu
- Base URL: https://api.hefu.hk/v1
- API key: your HeFu secret key
- HTTP proxy (optional): configure only if your network needs a proxy to reach api.hefu.hk. Mainland China users typically do not need this thanks to HeFu's Hong Kong nodes, per HeFu's official network documentation.
Save the provider.

Next, add models. In the provider's model list, enter each ID exactly as documented by HeFu, and enable capability toggles: function calling for tool-using models, vision for multimodal models, and file upload if you plan to attach documents. If you prefer a single-key catalog with pre-configured models, note that first-class providers such as ModelsLab (integrated via PR #12560 in March 2026) expose 200+ models—but for HeFu, manual entry is the standard workflow and gives you precise control over which models appear.

Configuring Model IDs and Capabilities

These HeFu model IDs are examples from the vendor's catalog; verify against the current HeFu docs before entering them in LobeChat:

Use case	Recommended HeFu model IDs
General-purpose / agentic work	`gpt-5.6-terra`, `gpt-5.6-sol`, `claude-opus-5`, `kimi-k3`
Coding	`gpt-5.3-codex`, `qwen3-coder`, `claude-opus-5`
Long-context Chinese text	`kimi-k3`, `kimi-k2.6`, `deepseek-v4-pro`
Multimodal (image / audio / video)	`gemini-3.6-flash`, `qwen3.7-max`, `glm-5.x`
Budget-focused inference	`deepseek-v4-flash`, `gemini-3.5-flash`

Capability toggles apply per provider, not per model; enabling vision on a non-vision model can cause silent request failures, so toggle conservatively.

Testing the Connection and Troubleshooting

After adding models, start a new conversation, pick a HeFu model, and send a test message. If it fails, check for these common cases:

401 Unauthorized: the key is wrong, expired, or has a copied whitespace character. Regenerate it in the HeFu dashboard.
404 Not Found: the base URL is malformed—make sure it ends with /v1, not /chat/completions.
Model not found: the ID does not match HeFu's catalog; re-check the docs for typos or deprecated names.

If you previously set OPENAI_PROXY_URL in a self-hosted Docker deployment, note that it can override UI-configured endpoints in certain versions. Remove or unset it and restart the container.

Comparison: HeFu Custom Provider vs. Direct Provider API

Dimension	HeFu as custom provider	Direct per-provider API keys
Setup time	~5 minutes, one provider entry	Hours: separate keys, URLs, and configs per provider
Key management	One HeFu key for all models	One key per provider (OpenAI, Anthropic, DeepSeek, etc.)
Model flexibility	Switch across GPT, Claude, Kimi, Gemini, Qwen from the model picker	Only models within the configured provider
Capability toggles	Manual: enable vision / function calling per provider	Auto-detected for first-class providers
Cost structure	Single usage bill from HeFu	Separate bills; self-hosted LobeChat itself is free

HeFu's per-token rates are subject to adjustment; refer to the official HeFu pricing page for current numbers. For cost context: LobeHub cloud offers 450,000 free compute credits at signup and paid tiers (Starter / Premium / Ultimate), while self-hosted and desktop LobeChat are free and only inference costs apply (AIHubMix, updated 2026-05-13; see also LobeHub official pricing). For budget planning, see our pay-as-you-go DeepSeek cost guide.

Security and Best Practices

Store the API key in an environment variable (e.g., HEFU_API_KEY in Docker -e) for self-hosted deployments, not in shared config files; see the LobeHub environment variables reference.
Rotate keys regularly from the HeFu dashboard—especially if a key was ever pasted into a public workspace or committed to Git.
Treat the key like a password: do not share screenshots that show both the key and your account name.
Review capability toggles before sharing a provider config with teammates, so unused vision or file-upload features are not left enabled.

FAQ

Does HeFu support all LobeChat model features?

HeFu's API is OpenAI-compatible, so it supports LobeChat's standard chat-completion flow. Feature support ultimately depends on the upstream model—function calling works on models with native tool support (GPT-5.6, Claude Opus 5, DeepSeek-V4-Pro), while vision requires a multimodal model such as Gemini 3.6 Flash or Qwen3.7-Max. Toggle the matching capabilities in LobeChat's provider settings per the HeFu docs.

Can I use HeFu with a self-hosted LobeChat instance?

Yes. As of August 2026, the custom provider fields are identical in self-hosted and cloud LobeChat (check the official docs for version-specific behavior). In Docker, configure HeFu through the UI after starting the container, or pre-seed environment variables with your deployment tooling. The same base URL and key work in both cases.

How do I update the model list after HeFu adds new models?

Open your HeFu provider in Settings → Language Model and edit the model list to add new IDs. The API key persists while you edit, so there is no need to re-enter it. For a zero-maintenance alternative, first-class integrated providers such as ModelsLab (200+ models via one key since March 2026, per PR #12560) auto-expose their catalogs—but with HeFu, manual ID entry is the intended workflow and keeps your model picker focused on what you actually use.

Will my usage data be logged by HeFu when used through LobeChat?

Yes, standard API usage logging applies. HeFu records request metadata (timestamps, model, token count) for billing and abuse prevention, consistent with industry practice. Review the privacy and compliance page on the HeFu official site for the full policy. Since HeFu routes to upstream model providers, those providers' own data policies also apply; check each model's terms if you have specific retention requirements.

👉 免费注册 HeFu / Sign up free

Chinese LLM API Pricing Comparison 2026: The Definitive Buyer's Guide

mpoper — Fri, 28 Aug 2026 09:05:47 +0000

If you're shopping for LLM APIs in 2026, Chinese vendors are impossible to ignore. As of August 21, 2026 (always check official pricing pages for the final word), flagship Chinese models charge between ¥4.00 and ¥12.00 per million input tokens — with ERNIE 5.1 at ¥4.00, GLM-5.1 at ¥6.00, Kimi K2.6 at ¥6.50, DeepSeek V4 Pro at ¥9.00, and Qwen3.7 Max at ¥12.00. Budget-tier input can be as low as ¥0.20 (Qwen3.5 Flash), and value models like DeepSeek V4 are 80–98% cheaper than GPT-5.5-class peers.

But don't pick a model on sticker price alone. Cache hit rates, endpoint access, and tool-calling fit often matter more than nominal list prices. The data below was verified against official pricing pages by llmabacus on 2026-08-21. Chinese vendors have turned quarterly price cuts into a structural competitive weapon: DeepSeek V4 Flash, for example, offers cached input at ¥0.10 per million tokens — just 1/30th of its standard input price.

2026 Chinese LLM API Pricing Landscape: An Overview

The 2026 Chinese LLM market is shaped by three forces:

Hardware cost deflation — cheaper compute keeps pushing prices down.
Escalating domestic price wars — vendors undercut each other every quarter.
Aggregator endpoints — services that arbitrage price gaps and unify access.

As of Aug 2026, tracking firm pricepertoken lists 610+ models globally, 43 of them free. Paid input prices range from roughly $0 to $150 per million tokens. Chinese vendors sit in the lowest price band, and many update prices quarterly — as Morph noted in its 2026-06-28 analysis: "LLM prices change every quarter."

Final prices are subject to each vendor's official pricing page:

The main camps remain unchanged:

DeepSeek and Alibaba's Qwen dominate the extreme value tier.
Kimi (Moonshot) differentiates on ultra-long context.
GLM (Zhipu), Doubao, and Tencent Hunyuan serve the domestic enterprise market.
OpenAI, Claude, and Gemini hold the high-end capability tier.

Through the HeFu unified gateway, all of the above families are accessible via one API — including GPT-5.6 (Terra/Sol/Luna), Claude Opus 5 and Sonnet 4.6, DeepSeek-V4-Pro/V4-Flash, Kimi K2.5/K2.6/K3, Gemini 3.6 Flash, Qwen3.7-Max, GLM-5.x, and MiniMax M2.5–M3. The exact model list and versions are subject to HeFu and vendor official pages.

Headline Findings: What the 2026 Market Data Reveals

Three findings matter most.

1. Flagship Chinese pricing has collapsed

According to llmabacus, which verified official pricing pages on 2026-08-21, per-million-token list prices are:

Model	Input	Output
DeepSeek V4 Pro	¥9.00	¥27.00
Qwen3.7 Max	¥12.00	¥36.00
Kimi K2.6	¥6.50	¥27.00
GLM-5.1	¥6.00	¥24.00
Baidu ERNIE 5.1	¥4.00	¥18.00

2. The budget tier is now priced in "cents"

The lowest verified input price is Qwen3.5 Flash at ¥0.20 ($0.030) per million tokens. The lowest output price is iFlytek Spark Ultra at ¥0.80. The lowest cache-input price is DeepSeek V4 Flash at ¥0.10 ($0.015) per million tokens — 1/30th of its standard input price.

3. The gap with Western flagships is roughly 40×

IntuitionLabs (as of Feb 2026) calculated that processing 1M input + 1M output tokens costs about $0.70 with DeepSeek (cache miss), versus $5 + $25 = $30 with Claude Opus 4.6. List-price arithmetic confirms that, as of Aug 2026, the DeepSeek V4 series is 80–98% cheaper than GPT-5.5-class models.

Methodology: How This Pricing Comparison Was Conducted

This comparison uses official list prices verified on 2026-08-21 via llmabacus, cross-checked against vendor pricing pages and public API docs. It is also cross-referenced with aggregator endpoint records from Morph (2026-06-28) and the pricepertoken model database.

Evaluation dimensions include:

Per-million-token prices: input, output, cache-input (in CNY and USD)
Context window: DeepSeek V4 Flash supports 1M tokens, per DeepSeek official docs
Benchmark-adjusted value: SWE-bench Verified as a coding proxy
Channel differences: first-party endpoints vs. aggregators like OpenRouter, Requesty, and Eden AI

Latency is assessed separately because it varies by endpoint, region, and load. First-party endpoints usually deliver more predictable latency, and the HeFu Hong Kong node provides direct low-latency access to OpenAI models without requiring an overseas credit card (subject to the official HeFu page).

USD conversions retain the original rounding from each source, so minor discrepancies may exist.

Side-by-Side Pricing Table (As of Aug 2026)

Model (official list price, per 1M tokens)	Input	Output	Cache input	Context	Verified
DeepSeek V4 Pro (official)	¥9.00	¥27.00	—*	Official docs	2026-08-21
DeepSeek V4 Flash (official)	¥3.00 ($0.45)	¥9.00 ($1.34)	¥0.10 ($0.015)	1M	2026-08-21
Qwen3.7 Max (official)	¥12.00	¥36.00	—*	Official docs	2026-08-21
Qwen3.5 Flash (official)	¥0.20 ($0.030)	—*	—*	Official docs	2026-08-21
Kimi K2.6 (official)	¥6.50	¥27.00	—*	Official docs	2026-08-21
GLM-5.1 (official)	¥6.00	¥24.00	—*	Official docs	2026-08-21

— = not disclosed in the verified sources at the time of writing.

Final Thoughts

Chinese LLM APIs offer extraordinary value, but the landscape shifts quarterly. Always confirm current prices on official pages before committing, and remember that the cheapest list price isn't always the cheapest total cost — especially when cache hits and tool-calling reliability are factored in. If you're building on a budget, 2026 is a great year to be a developer.

👉 免费注册 HeFu / Sign up free

Cheap DeepSeek API Pay As You Go: Complete Cost Guide (As of Aug 2026)

mpoper — Fri, 28 Aug 2026 09:05:46 +0000

As of Aug 2026, DeepSeek's pay-as-you-go API is still the cheapest mainstream option per million tokens among major LLM providers, according to pricing comparisons by CostGoat and Fello AI: the official V4 Flash tier costs roughly $0.14 per 1M input tokens and $0.28 per 1M output tokens on cache misses, which is 97.2% cheaper on input than OpenAI GPT-5.5's $5 rate and 99.1% cheaper on output than GPT-5.5's $30 rate. That works out to roughly 36× cheaper on input ($5 ÷ $0.14 ≈ 35.7) and 107× cheaper on output ($30 ÷ $0.28 ≈ 107.1) than GPT-5.6 Sol / Claude Opus 5-class flagships. If your goal is to minimize per-token spend without sacrificing a 1M-token context window, DeepSeek V4 Flash — available on HeFu under the same usage-based billing model — is the first option to evaluate.

Current DeepSeek API Pay-As-You-Go Rates (As of Aug 2026)

According to pricing reviews published in Jul 2026, the official DeepSeek API is purely pay-as-you-go, with no subscription required (Coworker.ai, Jul 2026; official price page: platform.deepseek.com/pricing):

DeepSeek V4 Flash: approximately $0.14 per 1M input tokens and $0.28 per 1M output tokens (cache miss); cached input is only about $0.0028 per 1M tokens — roughly 1/50th of the standard input price, i.e. a 98% discount (Coworker.ai, Jul 2026; PricePerToken; CloudZero).
DeepSeek V4 Pro: standard list price is $1.74 per 1M input and $3.48 per 1M output; since Jul 2026, an approximately 75% promotional discount brings the effective rate to roughly $0.435 / $0.87 per million tokens (Fello AI, 2026; CloudZero).
Free trial tokens: new accounts receive 5 million free tokens, valid for about 30 days, with no credit card required. There is no permanent free tier (Coworker.ai, Jul 2026; PricePerToken).

One important caveat: GeoToolbox reported on Aug 16, 2026 that DeepSeek switched to peak/valley pricing (V4 Flash valley from $0.22/$0.66, peak around $0.44/$1.32), which contradicts the flat Jul 2026 rates. However, another verification dated Jul 21, 2026 found no standing valley discount table on the official page (Layer3Labs). Because pricing has been moving quickly, always confirm current rates on the official DeepSeek price page (or the HeFu pricing page if you buy through an aggregator) before committing.

How Pay-As-You-Go Billing Actually Works for DeepSeek API

DeepSeek's pay-as-you-go model is simple: you prepay a balance, each API request deducts the metered token cost, and there are no idle fees, no monthly base fees, and no minimum usage commitment. A few mechanics worth understanding:

Token metering: both input and output tokens are counted per request; system prompts, retrieved context, and tool definitions all count as input.
Cache-hit discount: when your request includes a prefix that matches a cached prompt, the input portion drops to about $0.0028 per 1M tokens — a 98% discount from the $0.14 miss rate, not merely one-tenth (Coworker.ai, Jul 2026). DeepSeek's official docs describe the caching mechanics at api-docs.deepseek.com.
Specs: V4 Flash and V4 Pro both support a 1M-token context window and up to 384K output tokens, with published concurrency limits of 2,500 requests (Flash) and 500 requests (Pro) (Coworker.ai, Jul 2026).
No card required for trial: the 5M free tokens for new accounts are a genuine zero-cost test window, which makes pay-as-you-go evaluation of DeepSeek cheaper than any competitor.

DeepSeek vs. GPT-5.5 vs. Claude Opus 5: Cost Comparison Table

The table below compares published per-million-token rates and specs. DeepSeek's promotional pricing is marked with its validity window; competitor prices are official list rates as reported in Aug 2026.

Model (as of Jul–Aug 2026)	Input per 1M tokens	Output per 1M tokens	Cache-hit input	Context window	Source
DeepSeek V4 Flash	~$0.14	~$0.28 (cache miss)	~$0.0028 (≈98% discount)	1M tokens	Coworker.ai, Jul 2026
DeepSeek V4 Pro (promo)	~$0.435 (list $1.74)	~$0.87 (list $3.48)	—	1M tokens	Fello AI, 2026
OpenAI GPT-5.5	$5	$30	—	—	CostGoat, Aug 2026
GPT-5.6 Sol / Claude Opus 5 (flagship class)	≈36× V4 Flash input rate	≈107× V4 Flash output rate	—	—	CostGoat, Aug 2026; Fello AI, 2026

The ratio-based figures for the flagship class are derived from comparisons published by CostGoat (Aug 2026) and Fello AI (2026): $0.14 × 36 ≈ $5.04 for input, and $0.28 × 107 ≈ $29.96 for output, placing GPT-5.6 Sol and Claude Opus 5-class models in the $5–$30 per-million range. For context, our separate guides cover GPT-5.6 API pricing and Claude Opus 5 API access in more depth.

Real-World Cost Scenarios: 4 Typical Workloads

Monthly cost estimates at official DeepSeek V4 Flash rates (as of Jul–Aug 2026). These are illustrative — your actual bill depends on caching behavior and token mix.

Workload	Volume	Monthly cost (V4 Flash)
Customer-service chatbot	500 users × 30 messages = 15M input

👉 免费注册 HeFu / Sign up free

GLM, Zhipu & MiniMax API Access Outside China: 2026 Guide

mpoper — Fri, 28 Aug 2026 09:04:34 +0000

GLM, Zhipu & MiniMax API Access Outside China: 2026 Guide

Executive Summary

Yes — as of Aug 2026, GLM (Zhipu AI / Z.ai) and MiniMax APIs are accessible from most regions outside China, but direct official access comes with real friction: Z.ai's International endpoint charges roughly 2× its domestic price, MiniMax's Feb 2026 billing changes complicate cost forecasting, and a July 2026 Beijing policy review reported by international media could tighten frontier-model access. For international developers, the lowest-risk route is a unified gateway that exposes GLM-5.x and MiniMax M2.5–M3 alongside OpenAI, Claude, DeepSeek, Kimi, and Gemini under one API key with USD billing and no overseas credit card requirement.

Providers Overview

GLM (Zhipu AI)

Zhipu AI develops the GLM family of models. As of Aug 2026, flagship GLM-5 scores 92.7% on AIME 2026, 86.0% on GPQA-Diamond, and 50.4 on Humanity's Last Exam, surpassing GPT-5.2 and approaching Claude Opus 4.6. GLM-5.3 was released Aug 14, 2026 exclusively via the GLM Coding Plan and ZCode, with no public API or open weights; the highest published API tier is GLM-5.2 at $1.40 / $4.40 per million tokens.

MiniMax

MiniMax operates the M2.5–M3 line and is aggressively priced. M2.5 Standard costs $0.30 / $1.20 per million input/output tokens; Lightning doubles speed at $0.30 / $2.40 with a 1M-token context window. MiniMax has 27.6M+ monthly active users across 200+ countries, a market cap near $38B, and trailing revenue near $79M (implied multiple above 480×).

Geographic Availability

Z.ai explicitly segments by region: api.z.ai serves non-mainland China, open.bigmodel.cn serves domestic users. International pricing is about 2× domestic (so domestic is ~50% cheaper). US/EU/SEA devs can register and call the International endpoint directly, with payment verification.

MiniMax is globally oriented: >70% of revenue comes from outside China, ~20% from the US, and registration is open in 200+ countries. Both GLM-5 and M2.5 are also accessible via OpenRouter, DeepInfra, and Together.ai with OpenAI-compatible APIs and no geographic restriction.

Policy risk: In mid-2026, international media reported that Beijing held discussions with major Chinese AI labs about possible restrictions on overseas access to Chinese frontier models, including future open-weight releases. No binding restrictions had been enacted as of Aug 2026, but treat this as active policy risk.

Compliance Risks

Three layers matter:

Chinese data law – PIPL, Cybersecurity Law, and cross-border transfer rules can apply when Chinese providers process data from mainland infrastructure. Z.ai's International endpoint is designed to keep overseas traffic separate, but review governing law and data-storage clauses.
US/EU export-control exposure – US export controls targeting hardware could theoretically extend to Chinese model APIs. EU devs should assess GDPR Art. 28 third-country processor obligations.
Terms-of-service clauses – Both providers reserve the right to suspend access. The restrictive Aug 14 GLM-5.3 release shows access tiers can change abruptly.

Payment & Billing Hurdles

Z.ai International bills USD but costs roughly 2× domestic rates.
Feb 12, 2026: Zhipu raised GLM Coding Plan prices at least 30%; overseas subscription prices rose 30–60%; API call fees increased 67–100%.
As of Jul 2026, base GLM-5 tier is $0.60 / $2.20 per million tokens, cached input as low as $0.11, and the GLM Coding Plan remains a separate subscription.

Bottom Line

Direct access works, but it comes with geographic pricing, policy, and payment uncertainty. If you need a stable API surface for production, aggregating Chinese models through a single gateway with OpenAI-compatible endpoints and USD billing is the lowest-risk path.

👉 免费注册 HeFu / Sign up free

OpenRouter Alternatives for Chinese Models: A Practical Guide (As of Aug 2026)

mpoper — Fri, 28 Aug 2026 09:04:33 +0000

OpenRouter Alternatives for Chinese Models: A Practical Guide (Aug 2026)

TL;DR Chinese models now dominate OpenRouter's top models by token volume — roughly 61% of tokens consumed among the platform's top-ten models in the week of March 16–22, 2026 (Trending Topics, Mar 2026). Yet OpenRouter's 5.5% credit fee (OpenRouter FAQ), thin Chinese-model observability, and cross-border latency make a dedicated gateway the better default for many teams. A fast migration path is an OpenAI-compatible aggregator such as HeFu, exposing DeepSeek, Qwen, Kimi, GLM, and Western flagship models under a single API key through a Hong Kong node — no overseas credit card required.

Why Consider an Alternative

OpenRouter is still a strong universal router: 400+ models, 70+ providers, cost-plus pricing with a 5.5% credit fee, and a free tier of roughly 27 models at 50 requests/day (Eden AI, 2026; OpenRouter FAQ). But the model mix underneath has shifted. Chinese models in OpenRouter's daily Top 50 grew from 5 in early 2025 to 20 by May 2026 (Our World in Data, May 2026). During the week of March 16–22, 2026, Chinese models consumed 7.36 trillion tokens on the platform — up 56.9% week-over-week, about 61% of all tokens in the top-ten list.

Three structural problems follow for developers:

Latency. Tokens routed through OpenRouter's US/EU relay add extra hops; teams serving Chinese-language users from Asia see measurable time-to-first-token differences vs. direct endpoints.
Tool-calling gaps. Not every router translates OpenAI-style function-calling arguments faithfully for Qwen, DeepSeek, or GLM. This is a real compatibility issue, not a theoretical one.
Payment and compliance friction. OpenRouter bills in USD via Stripe, while mainland-China providers require real-name registration and local payment rails under China's Cybersecurity Law (Article 24) (gov.cn). International teams often need a middle layer that handles both worlds.

OpenRouter's Chinese-model catalog keeps growing — e.g., Qwen3 ASR 1.7B at $0.000008/second and Qwen3.8 27B at $0.20 per million tokens with 41K context as of Aug 13, 2026 (OpenRouter official model page). But discovering and tuning models inside a 400+ model catalog is not the same as a curated Chinese-model lineup. Prices change frequently, so treat the official OpenRouter model page as the live source of truth.

Key Evaluation Criteria

Use these six criteria when evaluating a replacement:

Catalog breadth. Does it cover Qwen, DeepSeek, Kimi, GLM, and MiniMax — plus the Western models you already call (GPT, Claude, Gemini)?
API compatibility. A drop-in OpenAI-compatible endpoint means you only change base_url and key (OpenAI Platform docs). Non-compatible SDKs force rewrites.
Pricing transparency. Compare cost per successful response, not per-token list price. Pass-through plus a small usage fee is normal; opaque markups are a red flag.
Data residency. Prompts may be stored in Hong Kong, Singapore, or mainland regions; mainland endpoints fall under China's Cybersecurity Law and PIPL (PIPL).
Verification & payment. Real-name registration, international card support, trial credits, and invoicing all matter for overseas teams.
Production features. Fallback routing, request tracing, and BYO-key governance separate a demo router from a production gateway.

Top Platforms to Replace OpenRouter for Chinese Models

The 2026 landscape splits into four categories.

1. Direct cloud providers

The obvious replacement is the vendor itself:

SiliconFlow — serverless multi-model endpoint with competitive Qwen and DeepSeek pricing (pricing).
Alibaba Cloud Model Studio — hosts the Qwen family, including Qwen3.7-Max, and exposes an OpenAI-compatible DashScope API (docs).
Baidu Qianfan — anchors the ERNIE ecosystem (docs).
Tencent Cloud — anchors the Hunyuan ecosystem (docs).
Zhipu AI — primary home for GLM-5.x, with OpenAI-compatible /api/paas/v4/ endpoints (docs).

All require registration, and most have mainland-China payment/compliance considerations. Choose based on your deployment region and whether you need international support.

2. OpenAI-compatible aggregators

If you want one key and one invoice, an aggregator is the fastest migration path. HeFu is the standout for Chinese models: it exposes DeepSeek, Qwen, Kimi, GLM, and Western flagship models under a single API key through a Hong Kong node. No overseas credit card is required for international teams.

3. Inference providers with multi-model endpoints

Some providers run multiple open-weight Chinese models on their own GPU infrastructure. These are a middle ground between direct vendor APIs and broad routers. Evaluate them with the same six criteria above.

4. Self-hosted gateways

For high-volume or data-sensitive workloads, run a gateway in front of direct endpoints. This gives you control over fallback routing, observability, and compliance while keeping an OpenAI-compatible surface for your application code. It's more ops work, but it eliminates router markups and cross-border relay hops.

Migration Tip

Choose a gateway with an OpenAI-compatible API first. In most SDKs, changing base_url and API key is enough to point at a new provider. Then test your tool-calling paths, compare cost per successful response, and roll out one workload at a time.

The right replacement depends on your latency, data-residency, and payment constraints. For most teams in 2026, a dedicated Chinese-model gateway will beat a generic global router.

👉 免费注册 HeFu / Sign up free

Qwen API Access from Overseas: A 2026 Developer's Guide

mpoper — Fri, 28 Aug 2026 09:03:19 +0000

Qwen API Access from Overseas: A 2026 Developer's Guide

Executive Summary

Qwen API is fully accessible to overseas developers as of August 2026: the international service is operated by Alibaba Cloud (Singapore) Private Limited (service terms); sign-up requires only an email address and an international credit card; and normal API calls from US or EU IP addresses work without a proxy or VPN, per the Model Studio international documentation. Direct access through Alibaba Cloud Model Studio's international endpoints (Singapore, US Virginia, and Frankfurt for Europe) is the lowest-cost route, while aggregators such as OpenRouter add convenience at a typical commission premium of 10–30% on per-token list prices (see Pricing below). For developers who prefer a single key across multiple model families, this site provides unified access to the Qwen lineup — Qwen3.7-Max, Qwen3.6-Plus, Qwen3.5, Qwen3-Max, and Qwen3-Coder — alongside OpenAI, Claude, DeepSeek, Kimi, and Gemini models.

Official Access Routes for Overseas Users

There are three legitimate entry points, and they differ mainly in sign-up flow and region selection.

Alibaba Cloud International console. Create an account at the international portal, verify with an email and an international credit card, then enable Model Studio in a non-mainland region. This route is governed by the terms of service of Alibaba Cloud (Singapore) Private Limited, per the qwen.ai service terms.
Qwen Studio / Model Studio international endpoint. Generate an API key in the Singapore region to use the OpenAI-compatible API. Keys are region-bound: keys issued for Singapore, US Virginia, and China (Beijing) cannot be exchanged between regions, as documented in the Alibaba Cloud Model Studio official documentation.
OpenAI-compatible international API. The same keys work with a simple base_url switch, which is how most existing OpenAI SDK users actually integrate Qwen. The international compatible-mode base URL is https://dashscope-intl.aliyuncs.com/compatible-mode/v1; see the Model Studio docs for the current endpoint reference.

As industry context for version timing: Qwen3.8-Max became available on August 3, 2026, with 2.4 trillion parameters, and its Max-level open-weights release was previewed for around August 10, 2026, per ofox.ai's release tracking. For production deployments through this site, however, the sellable Qwen catalog is Qwen3.7-Max, Qwen3.6-Plus, Qwen3.5, Qwen3-Max, and Qwen3-Coder.

Third-Party Platforms for Hosted Qwen Access

Many overseas developers use managed platforms instead of the official endpoint to avoid per-vendor account management. OpenRouter's Qwen page lists 85 Qwen models behind one API key as of August 2026; Together AI and Fireworks AI also host popular Qwen checkpoints as of August 2026.

The trade-offs are straightforward:

Feature parity: Aggregators usually expose the same chat-completions interface, but context length and tool-calling support can lag behind the first-party endpoint for newly released models.
Rate limits: Shared capacity on aggregators adds unpredictable latency; the official endpoint gives dedicated per-account quotas.
Model availability lag: The newest releases appear later on aggregators, when they appear at all. Examples include Qwen3.8-Max and preview checkpoints such as Qwen3.6-Max-Preview — a sparse MoE model with roughly 1 trillion parameters and a 262K-token context that is only served through Model Studio / Qwen Studio without open weights, per qwen.ai and OpenRouter (as of August 2026).

If you want unified key management across Qwen, DeepSeek, and Kimi rather than a single-vendor aggregator, see our guide to DeepSeek, Qwen, and Kimi on one API key. This site's HeFu gateway follows the aggregator model but keeps Qwen pricing aligned with the official pricing page.

Geographic Availability and Network Routing

The international Qwen service does not route traffic through mainland China. When you select Singapore or US Virginia in Model Studio, all requests are handled in that region; Frankfurt data centers cover European users, according to Alibaba Cloud Model Studio availability documentation. As of August 2026, the China (Beijing) endpoint is 60–70% cheaper for the same tokens, but it is a separate service with separate keys and is not designed for overseas compliance or network routing — most overseas developers should not attempt to use it.

Pricing and Billing in USD for Non-China Users

International billing is in US dollars via credit card; no Chinese bank account is required. Official international list prices as of August 2026 (Singapore endpoint) put the flagship Qwen3.7-Max at $1.25 per 1M input tokens and $3.75 per 1M output tokens, while the cost-efficient Qwen3.6-Flash is priced at $0.19 per 1M input and $1.13 per 1M output; the same models on the Beijing endpoint cost roughly 60–70% less, according to a Puter.js tutorial tracking Model Studio pricing.

Third-party aggregators typically add a commission of roughly 10–30% per million tokens on top of these list prices, which is why direct connection is cheaper when your workload is concentrated on Qwen. New Model Studio accounts also receive a free allowance of 1 million tokens per applicable model for 90 days (Singapore region only, as of August 2026); the legacy free OAuth API tier ended on April 15, 2026, per the same Puter.js tutorial. If you consume Qwen through this site, rates follow the official pricing page rather than a fixed number, because prices are adjusted from time to time.

Technical Integration: Base URL, API Keys, and SDK Migration

The fastest migration path keeps your existing OpenAI SDK and changes one parameter: set the base URL to the international compatible endpoint (https://dashscope-intl.aliyuncs.com/compatible-mode/v1), supply your Model Studio API key, and switch the model name to a valid Qwen model ID. Authentication requires only the API key header (Authorization: Bearer ...) — no mainland phone number or business license.

If you are already using this site's gateway, the same drop-in logic applies: keep your OpenAI-compatible client, point it at the gateway endpoint, and select a model from the sellable Qwen list. Notably, the Qwen route is more direct than the Kimi Moonshot path, which relies more heavily on aggregators; see our Kimi Moonshot API access guide for the comparison, and our GLM, Zhipu & MiniMax guide for how sibling Chinese model families compare on overseas access.

Performance Benchmarks for Overseas Latency

Public community benchmarks from late 2025 consistently show that the Singapore endpoint is the best latency compromise for teams serving both Asia-Pacific and Western users, with US West Coast round-trip times reported in the low hundreds of milliseconds; US Virginia is preferable for North America, and Frankfurt for Europe. Alibaba Cloud publishes no public latency SLA for Model Studio at the time of writing, so production teams should run their own load test in the selected region. OpenRouter's per-model provider pages provide rough throughput markers as of August 2026. Max-tier models trade some tokens-per-second for reasoning depth, while Plus-tier models prioritize throughput — so your choice between Qwen3.7-Max and Qwen3.6-Plus should be driven by workload type, not just price. For production decisions, run your own load test against the selected region, and note that aggregator relay layers add one extra network hop.

Compliance, Data Residency, and Cross-Border Considerations

When you select an international region, inference and data processing occur entirely in that data center — prompts never transit mainland-China infrastructure. Alibaba Cloud's international regions in Singapore, Germany, and the US offer documented data-residency commitments that help with GDPR and similar frameworks; Alibaba Cloud's Trust Center lists its ISO 27001, SOC 2, and GDPR-related compliance commitments. The same cannot be assumed for an aggregator unless its data-processing agreement explicitly states where Qwen traffic is processed. Overseas enterprises should retain region-selection records, the data-processing addendum, and audit logs for compliance reviews.

Comparison Table: Official Endpoint vs. Third-Party Platforms

Evaluation Dimension	Alibaba Cloud International	Third-Party Platforms
Setup complexity	Medium: cloud account, region selection	Low: instant API key
Pricing model	Usage-based in USD, official list prices	Usage-based with 10–30% commission markup typical
Data residency	Singapore / US / EU regions selectable	Varies by platform provider
Model availability	Full Qwen lineup, newest releases first	Partial, flagship models usually later
OpenAI SDK compatibility	Yes, via `base_url` switch	Yes, designed as drop-in replacements
Recommended for	Production workloads, cost-sensitive teams	Prototyping, multi-vendor experiments

FAQ

Does Qwen API block requests from overseas IP addresses?

No. The international endpoint is specifically designed for non-mainland users, and normal API calls from US, EU, or Southeast Asian IP addresses work without any proxy or VPN, as of August 2026. The mainland Beijing endpoint is the only one subject to mainland access requirements, and it is a separate service with separate keys.

Can I get an API key without a Chinese phone number or business license?

Yes. The Alibaba Cloud International sign-up process requires only an email address and an international credit card, and verification completes within minutes. There is no business-license upload for the standard Model Studio tier, and the free 1M-token-per-model allowance (90 days, Singapore region only) is available to new accounts as of August 2026.

Are my API calls processed on servers located in mainland China?

No. When you select an international region such as Singapore, US Virginia, or Frankfurt, your requests are handled entirely in that region and never transit through mainland-China data centers, per the Alibaba Cloud Model Studio documentation. This also explains why the international price is higher than the Beijing price (60–70% cheaper domestically): it reflects separate infrastructure and compliance obligations.

How do I switch from OpenAI to Qwen's API with minimal code changes?

Keep your existing OpenAI SDK and change only the base_url parameter to https://dashscope-intl.aliyuncs.com/compatible-mode/v1, then replace the model name with a valid Qwen model. On this site, the available Qwen models are Qwen3.7-Max, Qwen3.6-Plus, Qwen3.5, Qwen3-Max, and Qwen3-Coder; use Qwen3.7-Max for complex reasoning and Qwen3.6-Plus for most production workloads, with rates listed on the official pricing page.

What is the difference between using Qwen directly and through an aggregator?

Direct access gives you lower per-token costs (aggregators typically add 10–30%), earlier access to new models, and explicit data residency. Aggregators like OpenRouter (85 Qwen models as of August 2026, per the OpenRouter Qwen page) provide one key across many providers and easier experimentation, at the cost of latency overhead and possible model-availability lag.

👉 免费注册 HeFu / Sign up free

DeepSeek, Qwen, Kimi on One API Key: The Complete Guide

mpoper — Fri, 28 Aug 2026 09:03:18 +0000

DeepSeek, Qwen, Kimi on One API Key: The Complete Guide

Executive Summary: What This Guide Covers

As of Aug 2026, consolidating DeepSeek, Qwen, and Kimi behind a single API key is not only technically feasible—it is the recommended architecture for teams running multi-model workloads, because it eliminates authentication fragmentation, enables per-request model switching with zero code changes beyond a model parameter, and consolidates billing and failover behind one account. Whether it also lowers total spend depends on your cache hit rate and model mix; validate it against current per-token rates on the DeepSeek official pricing page, the Qwen/Alibaba Cloud Model Studio models page, and the Moonshot Kimi official pricing page. This guide explains how unified gateways route requests, which model family wins in which scenario, and how to get a single key operational in minutes.

Why a Single API Key Matters for Multi-Model Workflows

Running DeepSeek, Qwen, and Kimi through separate official accounts means juggling three dashboards, three billing cycles, three rate-limit policies, and three authentication schemes. In production, that fragmentation produces three concrete pain points:

Authentication friction — every new engineer needs access to multiple keys, and key rotation requires updating several environments at once.
Billing fragmentation — usage is scattered across vendors, making project-level cost attribution and unified budget enforcement nearly impossible.
Maintenance overhead — each vendor deprecates models and changes endpoints on its own schedule.

A single API key removes all three. As documented by Tencent Cloud Techpedia, a unified gateway exposes one OpenAI-compatible endpoint protected by one key; switching between DeepSeek, GLM, Kimi, MiniMax, Qwen, and others requires changing only the model parameter, and existing OpenAI clients work by changing just the base URL (retrieved Aug 2026). The same architecture powers public aggregators such as OpenRouter, which also exposes many model families behind one key.

Our HeFu DeepSeek API guide and Kimi Moonshot access guide cover this pattern for individual models; here we extend it to a three-family workflow.

Model Comparison: DeepSeek vs. Qwen vs. Kimi at a Glance

The table below compares the three families as of Aug 2026, cross-referencing Packet.ai, LLM-Stats, Wavect.io, and Kingy.ai, with official documentation links for current model catalogs.

Dimension	DeepSeek (V4-Pro / V4-Flash)	Qwen (Qwen3.5 / 3.6-Plus / 3.7-Max)	Kimi (K2.5 / K2.6 / K3)
Core strength	Cost-efficient reasoning	Broad model ladder with open licenses	Long-context native, agentic coding
Typical use cases	High-volume reasoning, classification, extraction	Chinese-language generation, coding, multimodal pipelines	Long documents, large codebases, agent loops
Context window	Up to 1M tokens; 49B (V4-Pro) / 13B (V4-Flash) activated params — check the DeepSeek API docs	Competitive context, strong instruction following — check Qwen/Alibaba Cloud Model Studio	1,048,576 tokens (~1M) on K3, released 2026-07-16 — check Moonshot docs
Cost positioning	Lowest tier — V4 Flash ≈ $0.22/M input, $0.66/M output tokens (per Packet.ai, Aug 2026; official rates may differ)	Mid-range to premium tier depending on variant — see Qwen/Alibaba Cloud Model Studio billing	Higher per-token cost; premium for long context — see Moonshot pricing
Licensing / access	Open-weight, simple commercial use — confirm the specific V4 repository and license on DeepSeek's official GitHub	Apache 2.0 for many releases, widest hardware coverage — see Qwen official GitHub	K3's 2.8T total weights (~1.6 TB storage) require separate commercial license review — confirm via Moonshot official release notes

Sources: Packet.ai blog (retrieved Aug 2026), LLM-Stats model comparison (updated 2026), Wavect.io (July–Aug 2026), DeepSeek API docs, Qwen official blog, Moonshot docs.

All three families are available at HeFu through one key: DeepSeek-V4-Pro / V4-Flash, Kimi K2.5 / K2.6 / K3, and Qwen3.5 / Qwen3.6-Plus / Qwen3.7-Max / Qwen3-Max / Qwen3-Coder. Note that some industry articles quote Qwen3.5 Flash or Qwen3.8-Max; those exact variants are not in HeFu's catalog — for a Flash-class Qwen workload, select Qwen3.5, and confirm multimodal model availability on the official pricing page.

How a Unified API Gateway Routes Your Requests

A gateway sits between your application and upstream model providers. When you send a request with one key, the gateway:

Authenticates the key and checks quota and billing balance.
Routes to the upstream provider matching the requested model value (e.g., deepseek-v4-flash, qwen3.6-plus, kimi-k3).
Applies fallback logic — if the primary provider returns a 429 or 5xx, the gateway can transparently retry on a secondary model you configure.
Returns the response in OpenAI-compatible JSON, so your client code stays unchanged.

Model-selection parameters — temperature, max_tokens, and an optional fallback_models field — let you express routing policy per request. This is the same architecture behind Tencent Cloud TokenHub: one endpoint, one key, model-as-a-parameter (retrieved Aug 2026). HeFu implements the same design with Hong Kong edge nodes, which matters for teams calling from outside mainland China — see our Hong Kong AI API node guide.

Getting Started: Setting Up Your Single API Key

Practical steps:

Sign up at HeFu and create a project.
Generate an API key from the console (copy it once; it is shown in full only at creation).
Set the base URL to HeFu's OpenAI-compatible endpoint (Hong Kong node by default).
Make your first multi-model request.

Python example using the official OpenAI SDK:

from openai import OpenAI

client = OpenAI(
    api_key="your-hefu-single-key",
    base_url="https://api.hefu.example/v1"  # replace with actual endpoint
)

# DeepSeek for cost-sensitive reasoning
r1 = client.chat.completions.create(
    model="deepseek-v4-flash",
    messages=[{"role": "user", "content": "Summarize this log file"}]
)

# Qwen for Chinese-language generation
r2 = client.chat.completions.create(
    model="qwen3.6-plus",
    messages=[{"role": "user", "content": "写一段产品发布文案"}]
)

# Kimi for long-context analysis
r3 = client.chat.completions.create(
    model="kimi-k3",
    messages=[{"role": "user", "content": "Analyze this 500-page contract"}]
)

That is the whole integration: change api_key and base_url once, then switch models by editing the model string. For model-specific walkthroughs, see our DeepSeek API guide and the Kimi outside-China access guide.

Pricing and Cost Optimization Across Three Models

As of Aug 2026, the industry selection consensus is remarkably stable across Packet.ai, Kingy.ai, Wavect.io, and multiple Medium engineering posts:

Lowest cost: DeepSeek V4 Flash. Packet.ai reports ~$0.22 per million input tokens and ~$0.66 per million output tokens; automatic prompt caching can reduce repeated-input costs substantially, but cache-hit rates change by model and over time, so confirm the current figure on the DeepSeek official pricing page (retrieved Aug 2026).
Long context and agentic coding: Kimi K3. Its 1,048,576-token context window (released 2026-07-16) is the strongest fit for very long documents and large codebases, but the 2.8T-weight model (~1.6 TB storage) requires separate commercial license review — plan procurement lead time accordingly.
Cost-quality balance: DeepSeek V4 Pro or Qwen3.6-Plus. For workloads needing stronger reasoning than Flash without K3's context ceiling, either is a sensible default.
Multimodal hosted products: route through Qwen3.7-Max or Qwen3-Max on HeFu; verify current model availability on the official pricing page.

Public price trackers such as LLM-Stats (updated 2026) show a very wide per-token price spread within this trio; however, exact ratios depend on input/output blend, prompt caching, and provider discounts. Calculate blended cost from the current DeepSeek, Qwen/Alibaba Cloud Model Studio, and Moonshot pricing pages before committing spend.

On the gateway side, unified billing means one invoice across all three families. Savings versus maintaining separate official accounts come from pooled caching and regional node selection, and the magnitude depends on your workload's cache hit rate and model mix. Exact per-model multipliers change over time; always confirm current rates on the official pricing page before committing spend.

Best Practices for Production Deployments

Set timeout and retry policies. Three upstream providers means three latency profiles. Use a short first-attempt timeout (10–15 s) with exponential backoff, and cap retries at 3.
Build a fallback chain. For example: deepseek-v4-flash → qwen3.5 → kimi-k2.6. If DeepSeek's upstream is saturated, the gateway fails over automatically instead of failing your request.
Route by context length. Under 32K tokens, DeepSeek V4 Flash is the cost champion. Between 32K and 200K, Qwen3.6-Plus offers strong Chinese-language quality. Above 200K, switch to Kimi K3 for its 1M context ceiling.
Monitor through one dashboard. A unified key gives you per-model spend, latency, and error rates in one view, making cost attribution trivial.
Know your rate limits. Aggregators inherit upstream limits per model; check your tier's concurrency and use gateway queueing or fallback to absorb bursts.

FAQ

Q1: Can one API key really call DeepSeek, Qwen, and Kimi at the same time? How much code do I need to change?
Yes. With an OpenAI-compatible gateway such as Tencent Cloud TokenHub or HeFu, you change exactly two lines in your existing client — base_url and api_key — and switch models by changing only the model parameter (Tencent Cloud Techpedia, retrieved Aug 2026; HeFu guide).

Q2: Is an aggregator gateway more expensive than going direct to each vendor?
Not necessarily. Aggregators often reduce total cost through shared prompt caching, and regional node selection; however, per-model markup and rate-limit policies differ by vendor, so verify specific multipliers on the official pricing page.

Q3: How do I choose between DeepSeek, Qwen, and Kimi?
By scenario. Lowest absolute cost: DeepSeek V4 Flash (~$0.22/M input, $0.66/M output as of Aug 2026 per Packet.ai; see also DeepSeek official pricing). Long context and agentic coding: Kimi K3 (1,048,576-token window, released 2026-07-16). Chinese-language generation and balanced cost/quality: Qwen3.6-Plus or Qwen3.7-Max. Multimodal hosted workloads: the Qwen family on HeFu.

Q4: Does the single key work with the official OpenAI SDK?
Yes. Because the gateway exposes an OpenAI-compatible /v1 endpoint, any OpenAI SDK (Python, Node.js, etc.) works by overriding base_url and api_key. No SDK fork or custom transport layer is required.

Q5: Can I mix models in one request batch, and how are rate limits managed?
Mixing is supported at the application layer — you can issue concurrent requests with different model values under a single key, and the gateway routes each independently. Rate limits are enforced per model per account tier, and upstream 429s trigger fallback routing if you configured a fallback chain. Check your tier's concurrency on the official pricing page.

Q6: I'm calling from outside mainland China — will latency suffer?
It depends on routing. HeFu's Hong Kong node is designed for exactly this scenario; see our Hong Kong AI API node guide and the Kimi outside-China access guide for latency benchmarks and regional data-residency notes.

👉 免费注册 HeFu / Sign up free

Chinese LLM Tool Calling Compatibility: A Systematic Comparison (as of Aug 2026)

mpoper — Fri, 28 Aug 2026 09:01:27 +0000

Chinese LLM providers have matured quickly. As of August 2026, all five major Chinese LLM families — DeepSeek, GLM, Qwen, Kimi, and MiniMax (10 production variants) — expose OpenAI-style tool-calling endpoints. A bare API base URL swap will often give you a valid response. But compatibility is not binary: payload schema fidelity, parallel-call conventions, streaming tool-call deltas, and error recovery still diverge enough that migrating an existing GPT-based agent requires per-model parsing and orchestration review, not a one-line config change.

Executive summary

Every model variant we tested supports an OpenAI-compatible /chat/completions endpoint, but compatibility is a spectrum.
GLM-5.1 and MiniMax M3 score 70.1 on the BenchLM tool-use suite, ahead of GPT-5.5's 67.8 (as of Aug 21, 2026).
Qwen3.7-Max is close at 68.8 but is only available through Alibaba Cloud Model Studio in mainland China.
Kimi K2 is the open-weight workhorse, but K2.6 scores 60.5 on the same benchmark and has historically non-standard streaming tool-call deltas.
Router layers (OpenRouter, Requesty, Eden AI) normalize transport, not semantics. They won't fix schema or streaming incompatibilities for you.
Teams that budget for per-model adapters can hit tool-use accuracy at or above GPT-5.5 levels at a fraction of the cost.

Scope and evaluation methodology

This comparison covers Chinese LLMs with publicly accessible function-calling APIs, as of Aug 2026:

DeepSeek-V4-Pro
GLM-5.1 and GLM-5.3
Qwen3.5, Qwen3.6-Plus, Qwen3.7-Max
Kimi K2.5, K2.6, K3
MiniMax M3

All 10 variants were exercised with a unified harness: identical tool schemas, prompts, and downstream mock services. GPT-5.5 was used as the compatibility baseline. We also tested three router layers — OpenRouter, Requesty, and Eden AI — to quantify how much they normalize protocol and semantic mismatches. Six dimensions were evaluated:

API protocol compatibility
JSON payload schema fidelity
Parallel tool-call support
Streaming tool-call parsing
Error recovery behavior (malformed JSON, downstream exceptions, empty results)
Total integration effort

Model landscape: three deployment tiers

Tier	Models	Deployment	Notes
1. Native OpenAI-compatible	DeepSeek-V4-Pro, GLM-5.3, Qwen3.x line	Hosted API	Accept standard function schemas out of the box; GLM-5.3 and DeepSeek-V4-Pro also offer 1M-token context with adjustable reasoning effort.
2. Open-weight, self-hostable	Kimi K2 family	Self-host / API	Kimi K2 is a 1T-parameter MoE with 32B active parameters, positioned for coding, tool calling, and agentic workloads. Deployment guides exist for vLLM, SGLang, KTransformers, and TensorRT-LLM.
3. Closed, API-only	Qwen3.7-Max	Alibaba Cloud Model Studio	Hosted exclusively in mainland China, which has direct data-residency implications for EU/US enterprise use.

Sources: DeepSeek API docs, Zhipu GLM API docs, Moonshot API docs, Alibaba Cloud Model Studio, and Turing Post's Chinese LLMs in 2026 roundup. Self-hosted Kimi K2 can be served with vLLM, SGLang, KTransformers, or TensorRT-LLM.

The OpenAI compatibility layer: where things diverge

At the protocol level, DeepSeek, GLM, and Qwen all accept OpenAI-format functions and tools schemas, including tool_choice and multi-turn tool-result messages. The real differences show up in three places:

Parallel tool calls — Does the model emit multiple tool calls in a single assistant turn? Some providers only emit one call at a time.
Streaming tool-call deltas — Does the API stream incremental token deltas for tool calls, or does it send a whole JSON object per chunk? This has a huge impact on your parser.
MCP server integration — Adapter quality varies significantly between providers. If you are using MCP as your tool layer, test each provider's implementation (see DeployBase's best model for function calling).

Provider-specific notes:

MiniMax M3 is OpenAI-compatible for basic calls, but it introduces custom extension fields that require additional parsing. Read the MiniMax API reference before production.
Kimi K2.6 accepts the standard payload but has historically produced non-standard streaming tool-call deltas in some versions. Teams end up building version-specific parsers.
Router layers do not fully eliminate these issues. OpenRouter, Requesty, and Eden AI normalize the transport protocol, not the semantics of tool-call payloads.

Function-calling accuracy and multi-step orchestration

On the consolidated BenchLM tool-use benchmark (updated Aug 21, 2026), the Chinese LLM field has pulled ahead of GPT-5.5 at the top:

Model	BenchLM tool-use score	Rank
GLM-5.1	~70.1	Top 4
MiniMax M3	~70.1	Top 4
Qwen3.7-Max	68.8	6
GPT-5.5	67.8	baseline
Kimi K2.6	60.5	11
GLM-5 (legacy)	58.3	15

Source: BenchLM

For multi-step orchestration, Alibaba's internal demonstration of Qwen3.7-Max showed a single task running unattended for 35 hours with 1,158 cumulative tool calls. That demonstrates strong context retention across long agent loops. But remember: Qwen3.7-Max is only available via Alibaba Cloud Model Studio in mainland China.

If you need a self-hosted agentic workhorse, Kimi K2's open weights and tool-calling-oriented architecture make it the most flexible option. Its benchmark gap of roughly 10 points versus GLM-5.1/MiniMax M3 has to be weighed against data-control requirements.

Latency, throughput, and error handling

In an agent loop, every tool call adds a round trip, so latency and throughput are not optional metrics. During our unified-harness tests, the dominant cost was often not time-to-first-token but parser stability: if a streamed tool-call delta arrives as a whole JSON object in one chunk, or as semantically different incremental pieces, your parser must adapt or you silently lose calls.

Error recovery is the other silent killer. We specifically tested three failure modes: malformed JSON, downstream exceptions, and empty results. Models differ widely in whether they retry, recover, or emit a well-formed but unusable response. Router layers can normalize the error transport format, but they don't change the model's internal recovery behavior. Plan for validation and retry logic at the application layer, not at the API layer.

What this means for your agent stack

Don't do a base-URL swap in production. Write a per-model adapter for tool-call parsing, especially for streaming and parallel calls.
If you optimize for benchmark accuracy, GLM-5.1 and MiniMax M3 are the current top choices. Qwen3.7-Max is close, but be aware of data residency.
If you need self-hosting or strict data control, Kimi K2 is the pragmatic option. Pin a known-good version and parser.
Add schema validators and retry logic around downstream exceptions, empty results, and malformed JSON.
Use router layers for endpoint routing and failover, not for semantic normalization.

Migration checklist

[ ] Validate your tools schema with each target model, not just one.
[ ] Test parallel tool-call behavior: does the model return multiple calls in one turn? Does your parser handle them?
[ ] Capture raw streaming chunks per provider and compare tool-call delta formats.
[ ] Test error recovery with malformed JSON, downstream exceptions, and empty results.
[ ] If using MCP, verify the adapter for each provider.
[ ] If using Kimi K2 in self-hosted mode, pin the exact version and test with your serving stack.

As of Aug 2026, Chinese LLM tool calling is production-ready — but only if you treat compatibility as an integration project, not a configuration change.

👉 免费注册 HeFu / Sign up free