Build a Runtime Cost Circuit Breaker for Chinese AI API Routers

Most AI cost controls are too late.

A dashboard that updates every hour is useful for finance review. A daily usage export is useful for reconciliation. A spreadsheet with current provider rates is useful during planning. None of those controls can stop a coding agent that just entered a retry loop, a summarizer that is sending the same 400K-token context on every turn, or a customer workflow that switched from a compact execution model to a long-context reasoning model without changing its output cap.

For production teams in the United States, the United Kingdom, Canada, Germany, the Netherlands, Japan, Singapore, Korea, Australia, and other Tier 1/2 markets, the safer pattern is a runtime cost circuit breaker. The router estimates each request before it leaves your application, records the actual usage after the response returns, and opens the circuit when a route, tenant, workflow, or model family starts behaving outside policy.

This article shows a practical design for Chinese AI API routers that call DeepSeek, Qwen, GLM, Kimi, or an OpenAI-compatible aggregator such as AIWave. It is not a benchmark and it is not a price-ranking table. It is an engineering control for teams that want model choice without runaway spend.

The examples use placeholders and environment variables only.

Pricing facts checked on August 14, 2026

Runtime breakers need dated pricing inputs. A route policy without a checked date is not an operational fact; it is an old assumption.

The public source pages below were checked on August 14, 2026:

Provider family	Public pricing shape	Breaker implication
DeepSeek V4 Flash	DeepSeek lists deepseek-v4-flash at $0.0028 per 1M cache-hit input tokens, $0.14 per 1M cache-miss input tokens, and $0.28 per 1M output tokens. It also lists a 1M context window and a future peak/off-peak schedule taking effect on August 16, 2026 at 16:00 UTC.	The breaker needs current and scheduled rate tables, not one permanent constant.
DeepSeek V4 Pro	DeepSeek lists deepseek-v4-pro at $0.003625 cache-hit input, $0.435 cache-miss input, and $0.87 output per 1M tokens, with a 1M context window.	Planning routes should get tighter tenant and workflow caps than execution routes.
Kimi K3	Kimi lists kimi-k3 at $0.30 cache-hit input, $3.00 cache-miss input, and $15.00 output per 1M tokens, with a 1,048,576-token context window.	Long-context routes need prefix reuse checks and strict output budgets.
Z.AI GLM	Z.AI lists GLM-5.2 and GLM-5.1 at $1.40 input, $0.26 cached input, and $4.40 output per 1M tokens. GLM-5 is listed at $1.00 input, $0.20 cached input, and $3.20 output.	Cached input and output growth should be tracked separately.
QwenCloud	QwenCloud documents pay-as-you-go text billing per million tokens, tiered pricing by request size, Batch API at 50% of real-time rates, model-specific cache discounts, and thinking tokens billed as output.	The breaker must store model-specific rules, input bands, and thinking mode behavior.

Sources:

• DeepSeek official pricing
• Kimi K3 pricing
• Z.AI pricing
• QwenCloud pricing documentation
• AIWave pricing page
• AIWave chat completions docs

These prices are source facts for an example control plane. Your application should check the pricing source that it actually bills through, then store the checked date next to the route policy.

What the breaker protects

A runtime breaker protects four failure modes that dashboards usually catch after damage is done.

The first failure mode is request expansion. A prompt template change can turn a 12K-token request into a 120K-token request. The model may still handle it, especially with 1M-context routes, but the unit economics are now different.

The second failure mode is output expansion. Agent loops, verbose tool summaries, and weak stop conditions can push generated tokens far beyond the product value of the response. Output is often the expensive side of the call, so it deserves a separate cap.

The third failure mode is cache collapse. A long-context workflow may look sustainable when repeated prefixes hit cache. A small change in prompt ordering, per-user metadata, or retrieved document order can turn cache-hit input into cache-miss input.

The fourth failure mode is route drift. A router may move traffic from a compact model to a reasoning model, from a Flash route to a Pro route, or from one aggregator route to another. The request still succeeds, but its cost envelope changes.

The breaker should not decide whether a model is good. It should decide whether a request is allowed to spend within the policy that product, finance, and engineering already approved.

Define the policy at the workflow level

Do not start with a global dollar cap. Global caps are blunt and create messy incidents: one noisy tenant can block unrelated customers, or one expensive admin workflow can consume the budget intended for user-facing traffic.

Start with workflow policy:

Workflow	Normal route	Allowed fallback	Preflight ceiling	Rolling window
Support reply draft	deepseek-v4-flash	GLM or Qwen execution route	$0.005 per request	$20 per tenant per day
Code review plan	deepseek-v4-pro	Kimi K3 only with approval flag	$0.05 per request	$100 per tenant per day
Contract summary	GLM or Qwen long-context route	DeepSeek Pro only for escalation	$0.03 per request	$60 per tenant per day
Retrieval answer	Qwen or DeepSeek Flash	Same family compact route	$0.01 per request	$30 per tenant per day

The exact numbers above are examples. The important part is the shape:

• one workflow name;
• one primary route;
• explicit fallback routes;
• one preflight ceiling;
• one rolling tenant window;
• one escalation path when the breaker opens.

The router can then reject, downgrade, trim context, or require human approval before an expensive call happens.

Store prices as versioned route metadata

Keep prices out of prompt code. Store them as route metadata with a source URL and checked date.

from dataclasses import dataclass
from decimal import Decimal
from typing import Optional


@dataclass(frozen=True)
class PriceCard:
    route: str
    model: str
    source_url: str
    checked_date: str
    input_per_mtok: Decimal
    output_per_mtok: Decimal
    cached_input_per_mtok: Optional[Decimal] = None
    effective_from_utc: Optional[str] = None


PRICE_CARDS = {
    "deepseek_flash_current": PriceCard(
        route="deepseek_flash_current",
        model="deepseek-v4-flash",
        source_url="https://api-docs.deepseek.com/quick_start/pricing/",
        checked_date="2026-08-14",
        input_per_mtok=Decimal("0.14"),
        cached_input_per_mtok=Decimal("0.0028"),
        output_per_mtok=Decimal("0.28"),
    ),
    "kimi_k3_current": PriceCard(
        route="kimi_k3_current",
        model="kimi-k3",
        source_url="https://www.kimi.com/resources/kimi-k3-pricing",
        checked_date="2026-08-14",
        input_per_mtok=Decimal("3.00"),
        cached_input_per_mtok=Decimal("0.30"),
        output_per_mtok=Decimal("15.00"),
    ),
}

Use Decimal, not float, for price math. Also keep scheduled price cards when a provider has announced a future change. DeepSeek's page, for example, lists a new peak/off-peak schedule effective August 16, 2026 at 16:00 UTC. A production router should be able to load the correct card by request time.

Estimate before sending the request

Preflight estimation does not need perfect token accounting to be useful. It needs to be conservative enough to block obvious mistakes.

from decimal import Decimal


def estimate_cost_usd(
    card: PriceCard,
    input_tokens: int,
    max_output_tokens: int,
    cached_input_tokens: int = 0,
) -> Decimal:
    uncached = max(input_tokens - cached_input_tokens, 0)
    cached_rate = card.cached_input_per_mtok or card.input_per_mtok
    input_cost = Decimal(uncached) * card.input_per_mtok / Decimal(1_000_000)
    cache_cost = Decimal(cached_input_tokens) * cached_rate / Decimal(1_000_000)
    output_cost = Decimal(max_output_tokens) * card.output_per_mtok / Decimal(1_000_000)
    return input_cost + cache_cost + output_cost


def should_allow_request(policy, estimate: Decimal, tenant_spend_today: Decimal) -> bool:
    if estimate > policy.preflight_ceiling_usd:
        return False
    if tenant_spend_today + estimate > policy.daily_tenant_ceiling_usd:
        return False
    return True

Use the configured max_tokens or max_completion_tokens for the output side. Do not estimate with the average output from yesterday. If the request allows the model to generate 64K tokens, the breaker should evaluate the request as if that output could happen.

For long-context workflows, estimate both cache-hit and cache-miss scenarios. If the route only works economically when cache hits are high, make that an explicit policy:

def require_cache_health(observed_hit_ratio: Decimal, required_hit_ratio: Decimal) -> None:
    if observed_hit_ratio < required_hit_ratio:
        raise RuntimeError("cache_hit_ratio_below_route_policy")

That error is not a provider failure. It is a release control doing its job.

Route with downgrade states

Opening the breaker should not always mean dropping the request. A good router has downgrade states.

The first state is trim. Remove optional retrieved documents, reduce conversation history, or switch from full files to relevant snippets.

The second state is compact. Use a smaller execution route for tasks that do not need deep planning.

The third state is defer. Move the work to an async queue, especially when a provider has real-time versus batch pricing differences.

The fourth state is approval. Ask for a human flag before a high-value workflow uses a long-context or reasoning route.

Here is a small router shape:

import os
from openai import OpenAI


client = OpenAI(
    api_key=os.environ.get("AIWAVE_API_KEY", "YOUR_API_KEY_HERE"),
    base_url="https://aiwave.live/v1",
)


def complete_with_breaker(workflow, messages, token_estimate, tenant_state):
    route = choose_primary_route(workflow)
    card = load_price_card(route)
    policy = load_workflow_policy(workflow)

    estimate = estimate_cost_usd(
        card=card,
        input_tokens=token_estimate.input_tokens,
        cached_input_tokens=token_estimate.cached_input_tokens,
        max_output_tokens=policy.max_output_tokens,
    )

    if not should_allow_request(policy, estimate, tenant_state.spend_today):
        route, messages = downgrade_route(workflow, route, messages, tenant_state)
        card = load_price_card(route)

    response = client.chat.completions.create(
        model=card.model,
        messages=messages,
        max_tokens=policy.max_output_tokens,
        temperature=policy.temperature,
    )

    record_actual_usage(
        workflow=workflow,
        route=route,
        model=card.model,
        usage=response.usage,
        price_card=card,
    )
    return response

The code assumes you have local helpers such as choose_primary_route, load_price_card, and record_actual_usage. Keep those boring. The business value is in the policy and the event log, not in clever routing code.

Record actual usage after every call

Preflight protects the front door. Actual usage closes the accounting loop.

Record at least:

Field	Reason
tenant_id	Per-customer ceilings and anomaly detection
workflow	Product-level budget ownership
route	Router policy review
model	Provider reconciliation
source_url and checked_date	Auditability for pricing assumptions
input_tokens	Prompt expansion detection
cached_input_tokens	Cache health
output_tokens	Runaway generation detection
estimated_usd	Preflight behavior
actual_usd	Reconciliation
downgrade_reason	Product and support review

This is also where an aggregator such as AIWave can be useful. If your application uses one OpenAI-compatible endpoint for 25+ Chinese models, your own router can stay focused on workflow policy while the platform handles provider access and model catalog exposure. You still need your own breaker because only your application knows which tenant, workflow, and customer action created the request.

Alert on slope, not just totals

A daily ceiling is necessary, but it is too slow by itself. Alert on slope:

• spend per tenant over the last 5 minutes;
• output tokens per request over the last 20 calls;
• cache-hit ratio by route over the last 50 calls;
• downgrade count by workflow over the last hour;
• request estimate versus actual cost variance.

Slope alerts catch bad deploys quickly. If a new prompt version doubles output tokens, you want the breaker to open before the daily cap is consumed. If cache-hit ratio drops from 80% to 10%, you want a route-level alert before finance asks why the long-context bill changed.

Keep policy separate from provider preference

Production AI routing has two different decisions:

1. Which model is appropriate for this workflow?
2. Is this specific request allowed to spend under current policy?

Do not merge those decisions into one if-statement. Model choice may depend on quality, latency, context length, tool support, region policy, and customer tier. Spend permission depends on estimated tokens, current price cards, tenant windows, and route health.

When those decisions are separate, the team can change pricing policy without rewriting model selection, and can change model selection without bypassing cost controls.

A release checklist for the breaker

Before turning on a new Chinese AI model route, require:

• a dated price card with source URL;
• a workflow policy with preflight and rolling-window ceilings;
• a max output cap;
• a cache-hit policy for long-context workflows;
• a downgrade path;
• an actual-usage ledger;
• slope alerts;
• a rollback flag that can disable the route without redeploying application code.

This checklist is intentionally small. It fits into a pull request. It gives engineering, product, finance, and security one shared object to review.

Final pattern

Chinese AI APIs are now varied enough that a single static provider table is fragile. DeepSeek exposes cache-hit and cache-miss pricing with scheduled peak/off-peak changes. Kimi K3 makes 1M-token context practical, but the output side needs guardrails. GLM and Qwen routes have their own cached input, tool, thinking, and tiered billing rules. Aggregators such as AIWave make access and switching simpler, but the application still needs workflow-aware controls.

The runtime cost circuit breaker is that control.

Estimate before the request. Record after the response. Open the circuit on route, tenant, workflow, and cache anomalies. Downgrade instead of failing when product value allows it. Store source dates next to every price card.

That is how a team can use multiple Chinese AI model families in production without turning every model launch into a billing surprise.