DEV Community: Mattias chaw

Build a Pricing Source-of-Truth Gate for AI Agent SDKs

Mattias chaw — Thu, 20 Aug 2026 13:10:15 +0000

Build a Pricing Source-of-Truth Gate for AI Agent SDKs

AI agent SDKs often hide pricing in the wrong place.

The model string is in code. The current rate card is in a dashboard. The public docs quote a dated provider page. The finance team uses an export from last week. The support team sees a customer ticket with a different number. None of those views is necessarily dishonest, but together they create a fragile release process.

When upstream providers change token rows, add context tiers, alter cache rules, or publish new concurrency limits, a production SDK should not rely on tribal memory. It needs a pricing source-of-truth gate: a small data contract and CI check that blocks stale rates before they ship.

This article shows one implementation for an OpenAI-compatible gateway that routes Chinese model families such as DeepSeek, Kimi, GLM, and Qwen through a unified API like AIWave. The goal is practical: make pricing facts dated, reviewable, testable, and visible to agent code before traffic moves.

Source Snapshot for August 20, 2026

For this run I checked public pricing and operations pages on August 20, 2026. These are source facts, not permanent product truth.

Source	Current signal to record
AIWave Pricing and Predictable Pricing	AIWave lists all-day DeepSeek V4 Flash rows of $0.638/M input, $1.914/M output, and $0.0203/M cache-hit input; V4 Pro rows of $1.914/M input, $5.742/M output, and $0.0638/M cache-hit input.
DeepSeek Models and Pricing	DeepSeek documents V4 Flash and V4 Pro token pricing, 1M context, 384K maximum output, OpenAI and Anthropic base URLs, and cache-hit/cache-miss input categories.
DeepSeek Rate Limit and Isolation	DeepSeek lists account-level concurrency limits of 2500 for V4 Flash and 500 for V4 Pro, plus HTTP 429 behavior and `user_id` isolation rules.
Kimi K3 Pricing	Kimi K3 lists $0.30/M cache-hit input, $3.00/M cache-miss input, $15.00/M output, and a 1,048,576-token context window.
Z.AI Pricing	GLM-5.3, GLM-5.2, and GLM-5.1 list $1.40/M input, $0.26/M cached input, and $4.40/M output.
QwenCloud Pricing	QwenCloud documents per-million-token billing, request-length tiers, Batch API behavior, context caching, thinking-token billing, and built-in tool fees.

A good gate does not flatten this table into one price_per_token number. Each provider uses its own billing vocabulary. DeepSeek has cache-hit and cache-miss input plus a concurrency surface. Kimi makes long-context caching central. GLM separates cached input from output. Qwen adds request tiers, thinking tokens, and tool calls. The gate should preserve that shape.

What the Gate Should Prevent

The gate exists to stop four release failures.

First, stale documentation. A developer reads a quickstart, copies an SDK route, and builds a budget around a number that was correct before a provider update. That is a trust failure even if the code still runs.

Second, silent route drift. The SDK points to deepseek-v4-pro, but the policy file was updated for a different route family. The application still returns answers, but finance cannot reproduce the bill from the published docs.

Third, cache math mismatch. The docs show input and output only, while the provider bills cache-hit input separately. Long-context agents can look much more expensive or much less expensive than expected depending on whether repeated context hits cache.

Fourth, bad fallback behavior. A 429 retry or fallback route can multiply cost. If the SDK does not carry rate-card dates and retry policy into logs, the incident review becomes guesswork.

These failures are common because pricing is treated as copy. Treat it as code instead.

A Minimal Rate-Card Contract

Start with a checked-in JSON document. It should be small enough for SDK tests and rich enough for finance review.

{
  "generated_at": "2026-08-20T13:00:00Z",
  "cards": [
    {
      "public_model": "deepseek-v4-flash",
      "provider_family": "DeepSeek",
      "source_url": "https://aiwave.live/pricing",
      "source_checked_at": "2026-08-20",
      "unit": "1M tokens",
      "currency": "USD",
      "input": 0.638,
      "cached_input": 0.0203,
      "output": 1.914,
      "context_tokens": 1000000,
      "max_output_tokens": 384000,
      "policy_note": "AIWave all-day public row"
    }
  ]
}

There are two important details here.

First, the source_checked_at field is mandatory. A rate without a date is not a production fact. It is a rumor with decimal places.

Second, the policy_note field explains whether the row is direct provider pricing, a unified gateway row, an official schedule row, a batch row, or an account-specific row. AIWave's DeepSeek rows, for example, should not be described as DeepSeek's official peak schedule. They are AIWave all-day rows. Keeping those layers separate prevents bad comparisons later.

CI Validation

The CI gate should block three classes of errors: missing evidence, unsafe values, and expired snapshots.

from __future__ import annotations

import json
from datetime import date
from pathlib import Path
from urllib.parse import urlparse

MAX_AGE_DAYS = 7
REQUIRED_FIELDS = {
    "public_model",
    "provider_family",
    "source_url",
    "source_checked_at",
    "unit",
    "currency",
    "input",
    "output",
}


def load_cards(path: str) -> list[dict]:
    data = json.loads(Path(path).read_text(encoding="utf-8"))
    return data["cards"]


def validate_card(card: dict, today: date) -> list[str]:
    errors: list[str] = []
    missing = REQUIRED_FIELDS - set(card)
    if missing:
        errors.append(f"missing fields: {sorted(missing)}")

    if urlparse(card.get("source_url", "")).scheme != "https":
        errors.append("source_url must be https")

    checked_at = date.fromisoformat(card["source_checked_at"])
    if (today - checked_at).days > MAX_AGE_DAYS:
        errors.append("source snapshot is older than policy allows")

    for field in ("input", "cached_input", "output"):
        value = card.get(field)
        if value is not None and value < 0:
            errors.append(f"{field} cannot be negative")

    if card.get("currency") != "USD":
        errors.append("this SDK release expects USD rows")

    return errors


if __name__ == "__main__":
    failures = []
    for card in load_cards("rate_cards.json"):
        for error in validate_card(card, date(2026, 8, 20)):
            failures.append(f"{card.get('public_model', '<unknown>')}: {error}")

    if failures:
        raise SystemExit("\n".join(failures))

This is intentionally boring. It does not scrape provider pages during every CI run. It validates the checked-in contract. A separate refresh job can update snapshots, but release CI should answer a narrower question: is the SDK about to publish with complete, dated, internally consistent facts?

Runtime Enforcement

The SDK should also carry pricing metadata into each route decision. That does not mean the SDK calculates the final invoice. It means the SDK leaves enough evidence for the gateway ledger to reproduce why a route was selected.

import os
from dataclasses import dataclass, asdict
from openai import OpenAI

client = OpenAI(
    api_key=os.environ.get("AIWAVE_API_KEY", "YOUR_API_KEY_HERE"),
    base_url="https://aiwave.live/v1",
)


@dataclass(frozen=True)
class RouteDecision:
    model: str
    task_kind: str
    source_checked_at: str
    policy_version: str
    max_tokens: int
    fallback_allowed: bool


def choose_route(task_kind: str) -> RouteDecision:
    if task_kind == "long_context_code_review":
        return RouteDecision(
            model="kimi-k3",
            task_kind=task_kind,
            source_checked_at="2026-08-20",
            policy_version="sdk-rate-gate-2026-08-20",
            max_tokens=2600,
            fallback_allowed=False,
        )

    return RouteDecision(
        model="deepseek-v4-flash",
        task_kind=task_kind,
        source_checked_at="2026-08-20",
        policy_version="sdk-rate-gate-2026-08-20",
        max_tokens=1200,
        fallback_allowed=True,
    )


route = choose_route("support_summary")
response = client.chat.completions.create(
    model=route.model,
    messages=[{"role": "user", "content": "Summarize the incident timeline for engineering review."}],
    max_tokens=route.max_tokens,
)

print({"response_id": response.id, **asdict(route)})

The printed route metadata is not a billing record by itself. It is the join key. The gateway can attach actual token counts, cache-hit tokens, retry count, status, latency, and final model. When a customer asks why the SDK used a route, support can answer from the same policy version that engineering reviewed.

Handling Provider-Specific Shapes

A strong source-of-truth gate allows provider-specific fields without forcing every model into the same shape.

For DeepSeek, include cache-hit input, cache-miss input, output, context, maximum output, concurrency, and 429 policy. Because DeepSeek documents user_id isolation, also decide whether the SDK passes a privacy-safe tenant identifier through extra_body.

For Kimi K3, include long-context size, cache-hit input, cache-miss input, output, and whether the route is approved for whole-repository or large-document workloads. A long-context route without cache visibility is hard to budget.

For GLM, include cached input and output rows, tool capability, reasoning tier, and any agent/tool rows that could affect invoice behavior. Do not put text-only assumptions on a route that may call vision, web search, image, video, or agent tools.

For Qwen, include request-length tiers, Batch API policy, thinking-token billing, and tool fees. A Qwen request with thinking enabled can bill differently from a compact text response, even when the public model name is the same.

The contract can support this by adding a billing_dimensions object:

{
  "public_model": "qwen-route-example",
  "provider_family": "Qwen",
  "source_checked_at": "2026-08-20",
  "billing_dimensions": {
    "request_tiers": true,
    "context_caching": true,
    "thinking_tokens_bill_as_output": true,
    "batch_api_has_separate_policy": true,
    "built_in_tool_fees": true
  }
}

The SDK does not need to know every provider detail to make a call. It does need to know which details exist, because those details drive warnings, usage ledgers, and release review.

Release Workflow

The release workflow should be simple.

Step one: refresh public source snapshots. Record the provider URL, the AIWave URL when using AIWave rows, retrieval date, currency, unit, and any notes about schedule or account scope.

Step two: review diffs like code. A price change, context change, output cap change, or concurrency change deserves the same review discipline as an API signature change.

Step three: run CI validation. The release should fail if required fields are missing, dates are stale, values are negative, model names do not appear in the SDK allowlist, or the article/docs layer quotes a different row than the JSON contract.

Step four: publish docs and SDK together. The docs should render from the same checked-in JSON, not from hand-edited tables.

Step five: log runtime route decisions. Every production request should carry model, policy version, source date, token counts, cache fields, retry count, fallback reason, and status. Store usage metadata, not prompt bodies or real customer identifiers.

What to Show Customers

Customer-facing docs do not need every internal control. They need enough to trust the route.

Show the model family, public model ID, supported request shape, context window, output cap, pricing source date, cache categories, and known limits. Explain whether the row is a direct provider row or an AIWave unified row. Link to the source page. Show a small calculator or formula that uses input, cached input, and output separately.

Avoid universal claims. A route can be predictable without being below every official provider row. A model can be strong for one task and poor for another. A cache row can reduce repeated-context cost only when the workload actually reuses context. Precision is more credible than broad marketing language.

Conclusion

Chinese AI model APIs are moving quickly. That is good for developers, but it makes stale pricing dangerous. A production SDK should not ship with numbers that only live in a blog post, screenshot, or private spreadsheet.

A pricing source-of-truth gate turns rate cards into a release artifact. It forces every model route to carry a source URL, retrieval date, billing dimensions, and policy version. It gives CI something concrete to reject. It gives runtime logs a join key. It gives support and finance the same evidence engineering used at release time.

Before your next agent SDK release, make the pricing contract part of the build.

Build Customer-Facing Model Cards for Chinese AI APIs

Mattias chaw — Wed, 19 Aug 2026 13:15:18 +0000

Build Customer-Facing Model Cards for Chinese AI APIs

Most teams start an AI gateway with a simple model list. The first version usually answers one question: which string do I put in the model field? That is enough for a prototype, but it is not enough for a production buyer, platform engineer, or support team.

A production model route has more than a name. It has a provider, a model version, a context window, an output ceiling, cache behavior, rate limits, tool charges, retention policy, and a dated rate card. If those details live only in a private spreadsheet, developers make routing decisions from stale screenshots and finance teams receive invoices they cannot reproduce.

The fix is a customer-facing model card endpoint.

This article shows how I would design one for an OpenAI-compatible gateway that serves Chinese model families such as DeepSeek, Kimi, GLM, and Qwen through a unified API like AIWave. The goal is not to advertise one universal winner. The goal is to make each route explain itself before an application sends traffic.

Source Snapshot for August 19, 2026

The model card should be generated from dated sources. For this article I checked the following public pages on August 19, 2026:

Source	Current signal to expose
DeepSeek Models and Pricing	V4 Flash and V4 Pro list OpenAI and Anthropic base URLs, 1M context, 384K maximum output, cache-hit and cache-miss input rows, output rows, peak/off-peak windows, and concurrency limits.
Kimi K3 Pricing	Kimi K3 lists token billing per 1M tokens, cache-miss input at $3.00/M, cache-hit input at $0.30/M, and output at $15.00/M.
Z.AI Pricing	GLM-5.3, GLM-5.2, and GLM-5.1 list $1.40/M input, $0.26/M cached input, and $4.40/M output.
QwenCloud Qwen3.7 Plus and Qwen3.7 Flash	Qwen3.7 Plus and Flash expose 1M context, cache rows, token rates, tool support, TPM, and RPM.
AIWave Pricing	AIWave currently publishes all-day DeepSeek V4 Flash rates of $0.638/M input, $1.914/M output, and $0.0203/M cache hit; V4 Pro rates of $1.914/M input, $5.742/M output, and $0.0638/M cache hit.

Do not turn this table into hardcoded product truth. Put retrieved_at, source_url, and rate_card_date into your model-card data so a future audit can see exactly what the API believed at the time.

What a Model Card Should Answer

A useful model card answers five operational questions.

First, what request shape is supported? Developers need to know whether the route accepts Chat Completions, Responses API, Anthropic format, images, tools, structured output, prefix completion, or batch jobs. A route that supports the same prompt text may still reject the same tool schema.

Second, what billing dimensions can change the invoice? DeepSeek separates cache-hit input, cache-miss input, output, and peak windows. Kimi K3 separates cache-hit input from cache-miss input and output. Qwen exposes context tiers and cache modes on model pages. GLM lists text model rows alongside vision, audio, image, video, tool, and agent categories. A single price_per_token field hides too much.

Third, what limits should an admission controller enforce? DeepSeek publishes concurrency limits for V4 Flash and V4 Pro. QwenCloud model pages publish TPM and RPM. Long-context routes also need maximum input, maximum output, and thinking-token rules. If the card omits limits, application teams discover them through production errors.

Fourth, what policy applies to logs and retention? A gateway should not place prompts, completions, real customer identifiers, API keys, or payment identifiers into public model metadata. The model card can expose retention class, region, and usage-ledger fields without leaking private data.

Fifth, how should a buyer compare routes? A buyer needs model version, source date, currency, cache behavior, and sample calculation method. They do not need unsupported benchmark claims.

A Minimal Model Card Schema

Start with a data structure that separates identity, capabilities, billing, and policy. Keep it small enough that docs, SDKs, and dashboards can consume the same JSON.

from dataclasses import dataclass
from datetime import datetime


@dataclass(frozen=True)
class PriceRow:
    unit: str
    input_usd: float | None
    cached_input_usd: float | None
    output_usd: float | None
    notes: str


@dataclass(frozen=True)
class ModelCard:
    provider: str
    public_model: str
    upstream_model: str
    model_version: str
    source_url: str
    retrieved_at: datetime
    context_tokens: int | None
    max_output_tokens: int | None
    supports_tools: bool
    supports_json: bool
    supports_vision: bool
    price: PriceRow
    policy_region: str
    retention_class: str

The important detail is that the card can say "unknown" by using None. An unknown value is better than a guessed value. It forces a reviewer to decide whether the route is ready for public docs, internal preview, or temporary suppression.

Store Route Data as Dated JSON

In production I prefer a checked-in snapshot for docs plus a runtime refresh path for dashboards. The checked-in snapshot keeps SDK tests stable. The runtime path warns teams when a provider page changes.

{
  "provider": "AIWave",
  "public_model": "deepseek-v4-flash",
  "upstream_model": "deepseek-v4-flash",
  "model_version": "DeepSeek-V4-Flash-0731",
  "source_url": "https://aiwave.live/pricing",
  "retrieved_at": "2026-08-19T13:30:00Z",
  "context_tokens": 1000000,
  "max_output_tokens": 384000,
  "supports_tools": true,
  "supports_json": true,
  "supports_vision": false,
  "price": {
    "unit": "1M tokens",
    "input_usd": 0.638,
    "cached_input_usd": 0.0203,
    "output_usd": 1.914,
    "notes": "AIWave all-day public rate card checked on 2026-08-19"
  },
  "policy_region": "Singapore",
  "retention_class": "usage-metadata-only"
}

This format gives every downstream system the same vocabulary. Docs can render the card. SDKs can validate model names. Finance can join usage events to a dated rate. Support can see which policy was advertised when a customer opened a ticket.

Serve the Cards Without Exposing Secrets

The endpoint should be public, cacheable, and boring. It should never include internal keys, upstream credentials, private customer routes, negotiated discounts, or hidden fallback priorities. If an enterprise account has custom terms, expose those terms only in authenticated account settings.

Here is a small FastAPI example:

from datetime import datetime, timezone
from fastapi import FastAPI

app = FastAPI()

MODEL_CARDS = [
    {
        "provider": "AIWave",
        "public_model": "deepseek-v4-flash",
        "upstream_model": "deepseek-v4-flash",
        "model_version": "DeepSeek-V4-Flash-0731",
        "source_url": "https://aiwave.live/pricing",
        "retrieved_at": "2026-08-19T13:30:00Z",
        "context_tokens": 1000000,
        "max_output_tokens": 384000,
        "capabilities": {
            "json": True,
            "tools": True,
            "vision": False,
            "streaming": True
        },
        "pricing": {
            "unit": "1M tokens",
            "input_usd": 0.638,
            "cached_input_usd": 0.0203,
            "output_usd": 1.914
        },
        "policy": {
            "region": "Singapore",
            "retention_class": "usage-metadata-only"
        }
    }
]


@app.get("/v1/model-cards")
def list_model_cards():
    return {
        "generated_at": datetime.now(timezone.utc).isoformat(),
        "cards": MODEL_CARDS,
    }

This endpoint is intentionally separate from /v1/models. The model-list endpoint can remain compatible with OpenAI-style clients. The model-card endpoint can carry richer procurement and operations metadata without breaking SDK assumptions.

Add Validation Before Publishing

The most useful model-card system is the one that refuses bad cards before customers see them. A simple validation pass should catch missing source URLs, invalid dates, negative prices, unsupported public claims, and secret-looking strings.

import re
from urllib.parse import urlparse

SECRET_PATTERN = re.compile(r"\bsk-[A-Za-z0-9_-]{20,}\b")


def validate_card(card: dict) -> list[str]:
    errors = []
    url = card.get("source_url", "")
    if urlparse(url).scheme not in {"https"}:
        errors.append("source_url must be https")

    if SECRET_PATTERN.search(str(card)):
        errors.append("card contains a secret-looking token")

    pricing = card.get("pricing", {})
    for field in ("input_usd", "cached_input_usd", "output_usd"):
        value = pricing.get(field)
        if value is not None and value < 0:
            errors.append(f"{field} cannot be negative")

    if not card.get("retrieved_at"):
        errors.append("retrieved_at is required")

    if not card.get("public_model"):
        errors.append("public_model is required")

    return errors

Run this check in CI and again in the publishing script. The second check matters because model cards are often generated from provider docs or admin dashboards, not typed by hand.

Use Cards in the Client Experience

The model card should show up before a developer burns tokens. The docs page can render a compact table. The SDK can expose a client.model_cards.list() helper. The dashboard can attach a route explanation to each usage row.

A support workflow can also use the same data. When a customer asks why a request was blocked, support can answer with the public model card and the private admission event:

Field	Public card	Private event
Model	`deepseek-v4-flash`	`deepseek-v4-flash`
Context limit	`1M`	request had `1,120,000` input tokens
Max output	`384K`	request asked for `500,000` output tokens
Pricing date	`2026-08-19`	usage event joined to same date
Policy	`usage-metadata-only`	no prompt body stored

This separation is useful. The card explains the rule. The private event explains the individual decision. Neither needs a real API key or prompt body.

Keep Pricing Math Reproducible

When pricing is cache-aware, the model card should not only display rows. It should describe how to calculate from usage. Here is a small cost function for a route with input, cached input, and output rows:

def estimate_cost_usd(input_tokens, cached_input_tokens, output_tokens, price):
    billable_input = max(input_tokens - cached_input_tokens, 0)
    return (
        billable_input / 1_000_000 * price["input_usd"]
        + cached_input_tokens / 1_000_000 * price["cached_input_usd"]
        + output_tokens / 1_000_000 * price["output_usd"]
    )


price = {
    "input_usd": 0.638,
    "cached_input_usd": 0.0203,
    "output_usd": 1.914,
}

print(round(estimate_cost_usd(700_000, 280_000, 300_000, price), 6))

This is not an invoice. It is a pre-dispatch estimate that uses the same rows a reviewer can see. The actual invoice should come from the provider or gateway ledger, but the estimate makes surprises much rarer.

Operational Rollout Checklist

Ship model cards in stages.

Start with a public endpoint for stable routes only. Include source URLs, retrieval dates, model versions, capabilities, context, output limits, and pricing rows. Leave experimental routes out until they have enough metadata.

Next, wire the same card data into your SDK tests. If a model appears in /v1/models but lacks a card, fail the docs build or mark the route as internal preview.

Then, connect the usage ledger. Each request should store public_model, provider, rate_card_date, input_tokens, cached_input_tokens, output_tokens, status, and route_reason. Store metadata, not customer content.

Finally, add a change review. When a provider changes a model version, rate row, context limit, tool fee, or cache definition, create a new dated snapshot. Do not silently rewrite the old one. Historical usage needs historical terms.

Conclusion

OpenAI-compatible APIs reduce integration work, but they do not remove operational responsibility. A Chinese model gateway still has to explain which route is being used, what it supports, how it is billed, what policy applies, and which dated source backs the claim.

A customer-facing model card turns that explanation into an API contract. It helps Tier 1 and Tier 2 developers compare DeepSeek, Kimi, GLM, Qwen, and AIWave routes without depending on stale screenshots or private spreadsheets. More importantly, it gives support, finance, and platform teams the same evidence trail.

Before you publish the next model name, publish the card behind it.

Why Unified Billing Matters When Upstream Pricing Is Volatile

Mattias chaw — Wed, 19 Aug 2026 11:01:43 +0000

Why Unified Billing Matters When Upstream Pricing Is Volatile

Show how a single all-day rate changes forecasting, while acknowledging that a gateway can differ from the benchmark list rate. Add a budget calculator and audit trail design. This draft is for review and uses dated examples; verify provider documentation before publishing.

The production problem

A model price is not a complete cost model. The application also has a model version, a provider route, a cache field, a timestamp, a retry policy, and a workload mix. After DeepSeek introduced peak/off-peak rates on 2026-08-17, those dimensions became visible to more developers. The safe response is to make every assumption explicit.

A small ledger

Record provider, model, rate_card_date, peak_class when known, input_tokens, cache_hit_tokens, output_tokens, status, latency_ms, and request_id. Keep prompt and completion bodies out of the ledger. Compute blended cost as (uncached_input * input_rate) + (cache_hit * cache_rate) + (output * output_rate). Store the native rate and currency before converting for a dashboard.

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://aiwave.live/v1",
    api_key=os.environ["AIWAVE_API_KEY"],
)
response = client.chat.completions.create(
    model="deepseek-v4-flash",
    messages=[{"role": "user", "content": "Classify this support ticket."}],
)
print(response.usage)

The key is deliberately read from an environment variable. A draft should never contain a real credential.

What to measure next

Run the same acceptance set through two model choices. Compare success, latency, output quality, cache share, and blended cost. A cheaper-looking row can lose once retries or long outputs dominate. A unified rate can be worth paying for when a team cannot schedule jobs around provider clock windows, but that is a workflow hypothesis to test with a cohort rather than a slogan.

Operational checklist

Pin model names; set a maximum output budget; use bounded retries; log the reason for a fallback; alert before a workspace budget is exhausted; review provider terms; and re-fetch the current rate card before a monthly forecast.

A worked review

Start with a small representative sample rather than a synthetic benchmark. Count repeated system context separately from new user input. If the provider reports only a combined cache field, mark the missing distinction in the report. When routing between models, preserve the request schema and compare the response against an acceptance test.

Conclusion

The durable engineering practice is not memorizing one price. It is keeping a dated rate ledger, exposing the reason for a route, and making a budget decision reproducible.

Review note

Re-run this calculation after a provider changes a model version, cache definition, billing currency, or peak schedule. Record the source URL and retrieval time so another engineer can reproduce the result.

Review note

Building with Kimi K3 and GLM-5 in Production

Mattias chaw — Wed, 19 Aug 2026 11:01:42 +0000

Building with Kimi K3 and GLM-5 in Production

Compare two model adapters behind one client contract. Cover model pinning, structured output, fallbacks, rate verification, and a rollout checklist without unsupported benchmark claims. This draft is for review and uses dated examples; verify provider documentation before publishing.

The production problem

A small ledger

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://aiwave.live/v1",
    api_key=os.environ["AIWAVE_API_KEY"],
)
response = client.chat.completions.create(
    model="deepseek-v4-flash",
    messages=[{"role": "user", "content": "Classify this support ticket."}],
)
print(response.usage)

The key is deliberately read from an environment variable. A draft should never contain a real credential.

What to measure next

Operational checklist

A worked review

Conclusion

The durable engineering practice is not memorizing one price. It is keeping a dated rate ledger, exposing the reason for a route, and making a budget decision reproducible.

Review note

Chinese AI API Cost Governance for Startups

Mattias chaw — Wed, 19 Aug 2026 11:01:40 +0000

Chinese AI API Cost Governance for Startups

Design a ledger with workspace, provider, model, rate version, input, cache, output, status, and budget alert fields. Include a worked blended-cost example and procurement questions. This draft is for review and uses dated examples; verify provider documentation before publishing.

The production problem

A small ledger

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://aiwave.live/v1",
    api_key=os.environ["AIWAVE_API_KEY"],
)
response = client.chat.completions.create(
    model="deepseek-v4-flash",
    messages=[{"role": "user", "content": "Classify this support ticket."}],
)
print(response.usage)

The key is deliberately read from an environment variable. A draft should never contain a real credential.

What to measure next

Operational checklist

A worked review

Conclusion

The durable engineering practice is not memorizing one price. It is keeping a dated rate ledger, exposing the reason for a route, and making a budget decision reproducible.

Review note

Multi-Model Routing with an OpenAI-Compatible API

Mattias chaw — Wed, 19 Aug 2026 11:01:39 +0000

Multi-Model Routing with an OpenAI-Compatible API

Build a small policy router that selects DeepSeek, Kimi, GLM, or Qwen from a task class and budget. Include timeout, retry, acceptance tests, and a safe environment-variable key. This draft is for review and uses dated examples; verify provider documentation before publishing.

The production problem

A small ledger

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://aiwave.live/v1",
    api_key=os.environ["AIWAVE_API_KEY"],
)
response = client.chat.completions.create(
    model="deepseek-v4-flash",
    messages=[{"role": "user", "content": "Classify this support ticket."}],
)
print(response.usage)

The key is deliberately read from an environment variable. A draft should never contain a real credential.

What to measure next

Operational checklist

A worked review

Conclusion

The durable engineering practice is not memorizing one price. It is keeping a dated rate ledger, exposing the reason for a route, and making a budget decision reproducible.

Review note

DeepSeek V4 Cache Economics After Peak and Off-Peak Pricing

Mattias chaw — Wed, 19 Aug 2026 11:01:38 +0000

DeepSeek V4 Cache Economics After Peak and Off-Peak Pricing

Explain how peak windows and cache-hit accounting affect a production ledger. Use the dated project baseline and a formula, then show a Python usage aggregator. Emphasize exact model, timestamp, and cache semantics. This draft is for review and uses dated examples; verify provider documentation before publishing.

The production problem

A small ledger

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://aiwave.live/v1",
    api_key=os.environ["AIWAVE_API_KEY"],
)
response = client.chat.completions.create(
    model="deepseek-v4-flash",
    messages=[{"role": "user", "content": "Classify this support ticket."}],
)
print(response.usage)

The key is deliberately read from an environment variable. A draft should never contain a real credential.

What to measure next

Operational checklist

A worked review

Conclusion

The durable engineering practice is not memorizing one price. It is keeping a dated rate ledger, exposing the reason for a route, and making a budget decision reproducible.

Review note

DeepSeek's New Peak-Pricing Model: What Developers Need to Know

Mattias chaw — Tue, 18 Aug 2026 12:46:45 +0000

DeepSeek's New Peak-Pricing Model: What Developers Need to Know

On August 17, 2026, I opened a cost dashboard and found a new variable in the wrong place: the clock. DeepSeek V4 Flash and V4 Pro now have Beijing-time peak windows in the dated rate card. The input and output rows are higher during 09:00–12:00 and 14:00–18:00 Beijing time, while cache-hit input is its own row. A service that looked stable when measured by requests can move substantially when measured by request time and output tokens.

This is a field note, not a claim that one provider fits every workload. I am sharing the approach I used to make the change measurable: keep the official rows dated, split input/output/cache in the ledger, and put a provider boundary around the OpenAI-compatible client.

The rate change in plain numbers

The official DeepSeek snapshot used for this post is dated 2026-08-17 and linked from the DeepSeek Models & Pricing page.

Route	Cache-hit off-peak	Cache-hit peak	Cache-miss input off-peak	Cache-miss input peak	Output off-peak	Output peak
V4 Flash / `deepseek-chat`	$0.007/M	$0.014/M	$0.22/M	$0.44/M	$0.66/M	$1.32/M
V4 Pro / `deepseek-reasoner`	$0.022/M	$0.044/M	$0.66/M	$1.32/M	$1.98/M	$3.96/M

AIWave publishes a separate unified route price: Flash $0.638 input and $1.914 output, Pro $1.914 input and $5.742 output, with cache-hit rows of $0.0203 and $0.0638. The useful distinction is operational. Direct DeepSeek asks me to reason about a Beijing clock window; the AIWave route gives my budget one all-day rate while keeping a single OpenAI-compatible workflow.

What changed in my ledger

I used to store requests, input_tokens, and output_tokens. That was not enough. I added provider, model, cache_hit_tokens, rate_date, peak_window, and fallback_attempt. The event now answers three questions: which route handled the request, which rate row applied, and whether the answer was validated.

The token mix matters. With 70% input and 30% output and no cache hits, Flash is $0.352/M blended off-peak and $0.704/M at peak. Pro is $1.056/M off-peak and $2.112/M at peak. If half of a workload lands in peak windows, a 10M-token Flash workload is $5.28 under the direct dated rows; the comparable Pro example is $15.84. These are planning calculations, not extra published rate rows.

Cache is separate. If a 10M-token job has 7M input tokens and 40% of that input hits a reusable prefix, only 2.8M tokens use the cache-hit row. Output still uses the output row. Treating the whole request as cached is the fastest way to produce a misleading budget.

The small calculator I actually want in review

from dataclasses import dataclass

@dataclass
class Rate:
    cache_hit_off: float
    cache_hit_peak: float
    cache_miss_off: float
    cache_miss_peak: float
    output_off: float
    output_peak: float

FLASH = Rate(.007, .014, .22, .44, .66, 1.32)

def cost(m_in, m_out, peak_share, cache_share, rate):
    hit = m_in * cache_share
    miss = m_in - hit
    cache_rate = rate.cache_hit_off + peak_share * (rate.cache_hit_peak - rate.cache_hit_off)
    miss_rate = rate.cache_miss_off + peak_share * (rate.cache_miss_peak - rate.cache_miss_off)
    out_rate = rate.output_off + peak_share * (rate.output_peak - rate.output_off)
    return hit * cache_rate + miss * miss_rate + m_out * out_rate

print(cost(7, 3, peak_share=.50, cache_share=.40, rate=FLASH))

The arguments are millions of tokens. Keeping the rate object explicit means a reviewer can compare a dated snapshot with the next one. I also run the same replay set through a second route so the cost change does not get confused with a prompt or workload change.

Moving from a single endpoint to a route list

The migration was mostly configuration. Business code calls one adapter, while the adapter chooses a provider and model. The key is loaded from the environment rather than placed in the draft or a shell command.

import os
from openai import OpenAI

ROUTES = [
    ("aiwave", "https://api.aiwave.live/v1", "deepseek-v4-flash"),
    ("deepseek", "https://api.deepseek.com", "deepseek-v4-flash"),
]

def make_client(base_url):
    return OpenAI(api_key=os.environ.get("AIWAVE_API_KEY"), base_url=base_url)

for provider, base_url, model in ROUTES:
    print(provider, model, bool(make_client(base_url)))

I do not retry validation errors across every route. Timeouts and upstream availability errors can be retriable; malformed tool schemas need a code fix. Every fallback gets a request ID, an attempt number, and a bounded deadline.

def bounded_routes(request, routes, deadline_s=18):
    seen = set()
    for route in routes:
        if route in seen:
            continue
        seen.add(route)
        try:
            return send(route, request, timeout=deadline_s), route
        except (TimeoutError, UpstreamUnavailable):
            continue
    raise RuntimeError("all approved routes failed")

The route list is not an excuse to ignore model capability. Flash handles routine classification, extraction, and short summaries in my policy. Pro is reserved for long reasoning chains, difficult code review, and tasks where validation failure is expensive. I keep a second Chinese model route for capacity and quality experiments rather than assuming that a single fallback is enough.

Scheduling and cache habits

Batch indexing and offline evaluation can be shifted outside the two Beijing peak windows. Interactive traffic normally cannot. I therefore keep an interactive budget and a batch budget, and I report peak share separately. That makes an increase explainable: perhaps the product added a US morning cohort, or perhaps a queue stopped draining overnight.

Cache prefixes are versioned. Stable policy text goes before volatile user content, and the event stores prompt_version. I only count a hit when the provider usage object reports hit tokens. This is less exciting than a clever prompt trick, but it survives a finance review.

A provider-neutral smoke test

Before increasing traffic, I replayed redacted requests against direct DeepSeek and the AIWave route. I compared JSON structure, tool-call validity, latency, output tokens, and task assertions. A 200 response is not equivalence. If a model produces a different tool schema or truncates a long answer, that belongs in the migration report.

#!/usr/bin/env bash
set -euo pipefail
DATE="2026-08-17"
EXPECTED="0.44/1.32"
echo "checking dated Flash peak row ${EXPECTED} on ${DATE}"

The code is intentionally dull. It gives the team a visible place to change the expected date and values, and it prevents a stale spreadsheet from being the only alarm.

The operating lesson

The biggest change was not a single dollar number. It was admitting that time, cache behavior, and provider choice belong in the same control plane. A team can schedule eligible batch work, select Flash for bounded tasks, select Pro when deeper reasoning is worth it, and use a multi-provider adapter when capacity or policy requires it.

If you are making the same change, start with a replay set and an event ledger. Then compare direct DeepSeek, a gateway such as OpenRouter, and a unified route such as AIWave using your own prompts and quality checks. Keep the API contract, model catalog, and RAG examples close to the implementation review.

The durable result is a system that can explain its bill. That is more useful than a single headline rate, especially when the clock becomes part of pricing.

The review questions I added to our runbook

First, we ask whether the application can delay work. Search indexing, document chunking, and offline evaluation are usually better candidates than interactive chat. Second, we ask whether the cache prefix is actually stable. A prefix that changes every request will not deliver the hit rate assumed in a spreadsheet. Third, we ask what happens after a fallback: do we preserve the same model semantics, or do we need a quality check before returning the answer?

For the migration itself, I use a shadow phase before a canary. Shadow requests are redacted and measured, not returned to users. The canary has a route-level budget and a stop condition for latency, validation failures, or unexpected output growth. This is important because a route that looks cheaper per million tokens can create extra calls if the application needs a repair pass.

One small operational habit helped: every monthly export includes the rate date next to the dollar total. A total without its rate date is hard to reproduce after a pricing change. A total with the date, provider, model, and token split can be reconciled against the event ledger and discussed with finance.

I also put a small alert on output-to-input ratio. The dated card has a three-to-one output/input relationship, so a product feature that starts generating longer answers can move spend even when input volume is flat. The alert does not block traffic; it creates a review item with the affected route, task class, and rate date. That keeps the response operational rather than turning a budget surprise into a blame exercise.
I keep that review visible for every release.

Build an Admission Controller for Chinese AI API Gateways

Mattias chaw — Mon, 17 Aug 2026 13:07:42 +0000

Build an Admission Controller for Chinese AI API Gateways

OpenAI-compatible gateways make Chinese model adoption easier, but the hardest production bugs often happen before the request reaches the provider. A coding agent sends a one-million-token repository snapshot without a cache key. A support workflow enables web search on every retry. A customer batch job moves from 31K to 33K input tokens and crosses a context billing tier. A fallback path quietly switches from a text model to a multimodal route. The API shape still looks familiar, but the operational profile has changed.

That is why a serious gateway needs an admission controller.

An admission controller is a small policy layer that runs before dispatch. It decides whether a request can be sent now, should be reshaped, should be queued, should use a different route, or should be rejected with a precise explanation. Kubernetes uses admission control to protect clusters. Multi-model AI gateways need the same idea for token budgets, context tiers, cache requirements, output caps, tool fees, regional policy, and provider capacity.

This article shows a practical admission controller for Tier 1 and Tier 2 engineering teams building with DeepSeek, Kimi K3, GLM, Qwen, ERNIE, or a unified endpoint such as AIWave. The examples use placeholders only and do not require real credentials.

Source Snapshot for August 17, 2026

The policy should be built from dated sources, not from memory. These are the public pages I would check before shipping today's rules:

Provider	Source	Admission-control signal
DeepSeek	Models & Pricing and Rate Limit & Isolation	V4 Flash and V4 Pro list 1M context, cache-hit and cache-miss input rates, output rates, peak/off-peak windows, and account-level concurrency limits.
Kimi	Kimi K3 Pricing	Kimi K3 publishes a 1,048,576-token context window with separate cache-hit, cache-miss, and output pricing.
Z.AI	GLM Pricing	GLM-5.2 and GLM-5.1 list input, cached input, output, tool, vision, image, video, audio, and agent pricing categories.
QwenCloud	Pricing, Qwen3.7 Plus, and Qwen3.7 Flash	QwenCloud documents context-tier billing, Batch API behavior, context caching, thinking-token billing, tool fees, 1M context routes, TPM, and RPM.
Baidu Qianfan	Model Billing	ERNIE routes should be treated as provider-specific billing and availability policy, then normalized behind the gateway.
AIWave	Docs, Models, and Pricing	A unified API should expose model metadata and route decisions without forcing application teams to learn every upstream billing shape.

Do not turn this table into a promise that never changes. Store it as a dated policy input and make the controller report which snapshot it used.

What the Controller Protects

A basic request router asks "which model should handle this prompt?" An admission controller asks a different set of questions first:

Check	Example decision
Context tier	A 990K-token Qwen request is allowed only on routes that explicitly support 1M context.
Cache key	A repeated long-context coding request must include a stable cache key or use a shorter route.
Output cap	A high-output Kimi K3 request needs an approved completion-token budget.
Concurrency	A DeepSeek V4 Pro route may queue when the account-level concurrency pool is near its limit.
Tool use	A GLM or Qwen tool call route must include a tool budget and a retry limit.
Region	A GDPR-sensitive workload should use an approved gateway region and logging policy.
Fallback	A fallback must not silently move a tenant into a different billing class.

The goal is not to block useful work. The goal is to stop requests whose production cost, latency, or compliance profile is unknown.

Step 1: Define an Admission Request

Normalize the caller's request into a small internal object. This makes policy evaluation independent of whether the upstream provider uses OpenAI Chat Completions, Responses API, Anthropic format, or a custom payload.

from dataclasses import dataclass, field


@dataclass(frozen=True)
class AdmissionRequest:
    tenant_id_hash: str
    region: str
    workload: str
    requested_model: str
    prompt_tokens: int
    max_output_tokens: int
    has_cache_key: bool
    cache_expected: bool
    uses_images: bool = False
    uses_video: bool = False
    uses_web_search: bool = False
    uses_code_interpreter: bool = False
    requested_fallbacks: list[str] = field(default_factory=list)

Use a hash or internal account identifier for the tenant. Do not put email addresses, names, API keys, or payment identifiers in policy logs.

Step 2: Keep Provider Rules as Data

The controller should not hardcode provider quirks inside a long if chain. Keep rules in dated data so pricing, context, and capacity changes are reviewable.

{
  "snapshot_date": "2026-08-17",
  "routes": {
    "deepseek-v4-pro": {
      "provider": "deepseek",
      "max_context_tokens": 1000000,
      "max_output_tokens": 384000,
      "requires_cache_key_above_tokens": 200000,
      "account_concurrency_limit": 500,
      "supports_tools": true,
      "supports_images": false,
      "supports_video": false
    },
    "qwen3.7-flash": {
      "provider": "qwencloud",
      "max_context_tokens": 1000000,
      "max_output_tokens": 131000,
      "requires_cache_key_above_tokens": 128000,
      "tpm_limit": 5000000,
      "rpm_limit": 15000,
      "supports_tools": true,
      "supports_images": true,
      "supports_video": true
    },
    "glm-5.1": {
      "provider": "zai",
      "max_context_tokens": 128000,
      "max_output_tokens": 32000,
      "requires_cache_key_above_tokens": 64000,
      "supports_tools": true,
      "supports_images": false,
      "supports_video": false
    }
  }
}

These are policy fields, not a full provider catalog. Add only the fields that affect dispatch. Keep the exact pricing rows in a separate snapshot if finance needs full reconstruction.

Step 3: Return Decisions, Not Boolean Flags

The controller should explain every outcome. A plain true or false forces the caller to guess what happened.

from dataclasses import dataclass


@dataclass(frozen=True)
class AdmissionDecision:
    action: str
    route: str | None
    reason: str
    snapshot_date: str
    estimated_prompt_tokens: int
    max_output_tokens: int
    warnings: list[str]


def deny(req: AdmissionRequest, snapshot_date: str, reason: str) -> AdmissionDecision:
    return AdmissionDecision(
        action="deny",
        route=None,
        reason=reason,
        snapshot_date=snapshot_date,
        estimated_prompt_tokens=req.prompt_tokens,
        max_output_tokens=req.max_output_tokens,
        warnings=[],
    )

Use stable actions such as:

Action	Meaning
`allow`	Dispatch immediately on the selected route.
`reshape`	Send after reducing output cap, disabling a tool, or requiring cache.
`queue`	Wait because the route is capacity constrained.
`fallback`	Use an approved alternate route with matching policy.
`deny`	Reject with a developer-readable reason.

This matters for SDK ergonomics. A developer can handle queue differently from deny, and reshape differently from fallback.

Step 4: Evaluate Policy Before Dispatch

Here is a compact evaluator. It is intentionally strict about long context and tool use because those are common sources of hidden production variance.

def admit(req: AdmissionRequest, policy: dict, live_capacity: dict) -> AdmissionDecision:
    snapshot_date = policy["snapshot_date"]
    routes = policy["routes"]
    route = routes.get(req.requested_model)
    if route is None:
        return deny(req, snapshot_date, "requested model is not in the approved route catalog")

    if req.prompt_tokens > route["max_context_tokens"]:
        return deny(req, snapshot_date, "prompt exceeds route context window")

    if req.max_output_tokens > route["max_output_tokens"]:
        return AdmissionDecision(
            action="reshape",
            route=req.requested_model,
            reason="requested output cap exceeds route policy",
            snapshot_date=snapshot_date,
            estimated_prompt_tokens=req.prompt_tokens,
            max_output_tokens=route["max_output_tokens"],
            warnings=["max_output_tokens was reduced by admission policy"],
        )

    cache_threshold = route.get("requires_cache_key_above_tokens", 0)
    if req.prompt_tokens >= cache_threshold and not req.has_cache_key:
        return deny(req, snapshot_date, "long-context request requires a stable cache key")

    if req.uses_images and not route.get("supports_images", False):
        return deny(req, snapshot_date, "image input is not approved for this route")

    if req.uses_video and not route.get("supports_video", False):
        return deny(req, snapshot_date, "video input is not approved for this route")

    if (req.uses_web_search or req.uses_code_interpreter) and not route.get("supports_tools", False):
        return deny(req, snapshot_date, "tool use is not approved for this route")

    provider = route["provider"]
    active = live_capacity.get(provider, {}).get("active_requests", 0)
    limit = route.get("account_concurrency_limit")
    if limit is not None and active >= int(limit * 0.9):
        return AdmissionDecision(
            action="queue",
            route=req.requested_model,
            reason="provider concurrency pool is near the admission threshold",
            snapshot_date=snapshot_date,
            estimated_prompt_tokens=req.prompt_tokens,
            max_output_tokens=req.max_output_tokens,
            warnings=[],
        )

    return AdmissionDecision(
        action="allow",
        route=req.requested_model,
        reason="request satisfies route admission policy",
        snapshot_date=snapshot_date,
        estimated_prompt_tokens=req.prompt_tokens,
        max_output_tokens=req.max_output_tokens,
        warnings=[],
    )

The 90 percent threshold is an example, not a universal constant. In a real gateway, set it by route class. Interactive coding agents need a different queue policy than offline evaluation jobs.

Step 5: Make Fallbacks Explicit

Fallback is where many gateways lose control. A fallback that changes context window, modality, tool pricing, or output cap should be treated as a new admission request.

def choose_fallback(req: AdmissionRequest, policy: dict, live_capacity: dict) -> AdmissionDecision:
    first = admit(req, policy, live_capacity)
    if first.action in {"allow", "reshape", "queue"}:
        return first

    for fallback_model in req.requested_fallbacks:
        candidate = AdmissionRequest(
            tenant_id_hash=req.tenant_id_hash,
            region=req.region,
            workload=req.workload,
            requested_model=fallback_model,
            prompt_tokens=req.prompt_tokens,
            max_output_tokens=req.max_output_tokens,
            has_cache_key=req.has_cache_key,
            cache_expected=req.cache_expected,
            uses_images=req.uses_images,
            uses_video=req.uses_video,
            uses_web_search=req.uses_web_search,
            uses_code_interpreter=req.uses_code_interpreter,
            requested_fallbacks=[],
        )
        decision = admit(candidate, policy, live_capacity)
        if decision.action in {"allow", "reshape", "queue"}:
            return AdmissionDecision(
                action="fallback",
                route=decision.route,
                reason=f"primary denied: {first.reason}; fallback accepted",
                snapshot_date=decision.snapshot_date,
                estimated_prompt_tokens=decision.estimated_prompt_tokens,
                max_output_tokens=decision.max_output_tokens,
                warnings=decision.warnings,
            )

    return first

This is especially important when moving between providers. A DeepSeek V4 route may expose a 1M context window and account-level concurrency signals. A Qwen route may expose context tiers, TPM, RPM, thinking tokens, multimodal input, and tool charges. A GLM route may add separate categories for text, vision, tools, image generation, video, audio, and agent products. Treating all of them as identical text completion routes is an operations bug.

Step 6: Surface Admission Metadata to Callers

For OpenAI-compatible responses, keep the standard response shape intact and add a namespaced object.

{
  "id": "chatcmpl_placeholder",
  "object": "chat.completion",
  "model": "qwen3.7-flash",
  "usage": {
    "prompt_tokens": 240000,
    "completion_tokens": 1800,
    "total_tokens": 241800
  },
  "aiwave": {
    "admission": {
      "action": "allow",
      "snapshot_date": "2026-08-17",
      "route": "qwen3.7-flash",
      "reason": "request satisfies route admission policy",
      "cache_required": true,
      "cache_key_present": true,
      "tool_budget_applied": false
    }
  }
}

When the controller rejects a request, return a clear application error:

{
  "error": {
    "type": "admission_denied",
    "message": "Long-context request requires a stable cache key.",
    "param": "cache_key",
    "code": "cache_key_required"
  },
  "aiwave": {
    "snapshot_date": "2026-08-17",
    "requested_model": "kimi-k3",
    "estimated_prompt_tokens": 720000,
    "max_output_tokens": 64000
  }
}

That error is much more useful than a later surprise in the invoice, a provider-side 429, or a timeout that hides the true cause.

Step 7: Observe the Controller Itself

Once deployed, watch admission outcomes as a first-class product metric:

Metric	Why it matters
Admission denials by reason	Shows whether docs, SDK defaults, or customer prompts need correction.
Reshaped output tokens by route	Reveals where completion budgets are too loose.
Queued requests by provider	Warns before 429s become user-visible failures.
Fallbacks by primary route	Detects hidden dependence on secondary providers.
Long-context requests without cache keys	Finds workflows that need SDK-level cache support.
Tool-enabled retries	Stops retry loops from compounding tool charges.

Review these metrics separately for interactive, batch, support, coding, and research workloads. A support chatbot should not share the same output and retry policy as a repository-scale coding agent.

Practical Release Checklist

Before enabling a new Chinese model route in production, ask:

Check	Pass condition
Pricing source	Official provider pages were opened and dated.
Context window	The route has a tested max input and max output policy.
Cache policy	Long-context requests require a stable cache key.
Tool policy	Tool-enabled routes have explicit per-request limits.
Capacity policy	Concurrency, TPM, or RPM limits are represented in data.
Fallback policy	Fallbacks are separately admitted, not silently substituted.
Response metadata	SDK users can see the snapshot and admission reason.
Log hygiene	No personal data, secrets, or raw prompts are written to admission logs.

This pattern is not limited to one provider. DeepSeek's concurrency and peak windows, Kimi's long-context cache economics, GLM's broad modality and tool catalog, QwenCloud's context tiers and tool fees, and ERNIE's provider-specific billing all point to the same engineering conclusion: request compatibility is not the same as production compatibility.

Final Takeaway

OpenAI-compatible APIs are valuable because they reduce integration work. They do not remove the need for policy.

If your gateway accepts every request that matches the SDK method signature, it is letting the provider market define your runtime behavior. An admission controller gives that control back to the platform team. It turns context length, cache readiness, output caps, tool usage, concurrency, and fallback behavior into explicit decisions that developers can understand before a request leaves the system.

For teams evaluating Chinese AI models through a unified API, that is the difference between a demo route and a production route.

Build a Pricing Change Runbook for OpenAI-Compatible Chinese AI SDKs

Mattias chaw — Sun, 16 Aug 2026 13:12:40 +0000

Build a Pricing Change Runbook for OpenAI-Compatible Chinese AI SDKs

Chinese model APIs are now changing quickly enough that SDK maintainers need a release process for pricing, not just a README table. DeepSeek's official pricing page says new V4 peak and off-peak prices take effect on August 16, 2026 at 16:00 UTC. Kimi K3 publishes separate cache-hit and cache-miss input rates for long-context API use. Z.AI lists GLM cached-input and output rates across text, vision, tools, and agents. QwenCloud documents pay-as-you-go billing by modality, context tier behavior, Batch API options, context caching, thinking-token billing, and tool fees.

That is a lot of change for a normal application. It is even more sensitive for an OpenAI-compatible SDK or gateway because the caller expects a stable interface while the provider market underneath it keeps moving. The SDK may accept the same chat.completions.create() call, but the bill behind that call can change by model version, cache state, UTC window, output cap, context length, and fallback path.

This article shows a practical pricing-change runbook for Tier 1 and Tier 2 engineering teams that expose Chinese AI models through a unified API. The pattern is deliberately operational: keep append-only pricing snapshots, detect source drift in CI, make cost assumptions visible in SDK responses, and roll out new route weights only after the dated snapshot is acknowledged. The examples use neutral OpenAI-compatible shapes and placeholders only. They work with direct provider integrations or with a unified endpoint such as AIWave's model and chat APIs.

Why Pricing Needs a Runbook

Most SDKs start with a static model catalog:

Model	Provider	Input	Output	Context
deepseek-v4-flash	DeepSeek	configured in code	configured in code	1M
deepseek-v4-pro	DeepSeek	configured in code	configured in code	1M
kimi-k3	Moonshot Kimi	configured in code	configured in code	1M
glm-5.1	Z.AI	configured in code	configured in code	provider page
qwen route	QwenCloud	configured in code	configured in code	tiered

That table is convenient, but it hides the actual production problem. The model catalog is not the pricing contract. A production contract also needs:

Dimension	Why it matters
Effective date	A request made before a price change should be explained with the old snapshot.
Source URL	Finance and SRE need to know which provider page was used.
Cache state	Cache-hit input and cache-miss input are separate billing dimensions.
Time window	DeepSeek's current docs introduce peak and off-peak billing windows.
Context tier	QwenCloud documents long-context tier behavior and separate modality rules.
Tool usage	Z.AI lists separate tool and modality costs.
Output cap	Completion tokens often dominate cost when input is cached.

If those dimensions live only in a spreadsheet, the SDK can ship stale behavior. If they live inside the routing layer, every request can carry the same assumptions used by the estimator, the router, and the later invoice review.

Current Source Snapshot for August 16, 2026

Use dated facts, not memory. Before changing route weights today, the pages to inspect are:

Provider	Source checked	Operational note
DeepSeek	Models & Pricing	V4 Flash and V4 Pro list 1M context, cache-hit input, cache-miss input, output rates, concurrency limits, and an August 16, 2026 effective time for peak and off-peak billing.
DeepSeek	Rate Limit	Account-level concurrency limits affect production route capacity, not only unit price.
Kimi	Kimi K3 Pricing	Kimi K3 lists $3.00 per 1M cache-miss input tokens, $0.30 per 1M cache-hit input tokens, and $15.00 per 1M output tokens.
Z.AI	Pricing	GLM-5.2 and GLM-5.1 list $1.40 per 1M input tokens, $0.26 per 1M cached input tokens, and $4.40 per 1M output tokens.
QwenCloud	Pricing	The page documents pay-as-you-go billing, separate input and output text billing, long-context tiers, Batch API behavior, context caching, thinking-token billing, failed-call billing behavior, and tool fees.
AIWave	Models and Pricing	A unified gateway should expose model ownership, route names, and billing fields without forcing the caller to learn every upstream shape.

The exact table should be maintained in your own repository as data, not copied into SDK code by hand. The point of the runbook is to make a source change reviewable.

Step 1: Store Append-Only Pricing Snapshots

Create one JSON file per effective date. Do not overwrite an old file when a provider updates a page. Historical requests need historical assumptions.

{
  "snapshot_date": "2026-08-16",
  "currency": "USD",
  "sources": [
    {
      "provider": "deepseek",
      "url": "https://api-docs.deepseek.com/quick_start/pricing/",
      "checked_at": "2026-08-16T21:00:00+08:00"
    },
    {
      "provider": "kimi",
      "url": "https://www.kimi.com/resources/kimi-k3-pricing",
      "checked_at": "2026-08-16T21:00:00+08:00"
    }
  ],
  "models": {
    "deepseek-v4-flash": {
      "provider": "deepseek",
      "effective_from": "2026-08-16T16:00:00Z",
      "input_cache_hit_per_1m": 0.0028,
      "input_cache_miss_per_1m": 0.14,
      "output_per_1m": 0.28,
      "notes": "Check peak and off-peak windows before route rollout."
    },
    "kimi-k3": {
      "provider": "kimi",
      "effective_from": "2026-08-12T00:00:00Z",
      "input_cache_hit_per_1m": 0.30,
      "input_cache_miss_per_1m": 3.00,
      "output_per_1m": 15.00,
      "notes": "Long-context use benefits from stable cache keys."
    }
  }
}

The notes field is not decorative. It lets reviewers capture constraints that cannot be represented as a single number. For example, DeepSeek's current V4 page includes peak and off-peak behavior. QwenCloud's page includes context tiers and billing categories beyond text. Z.AI includes tool and modality costs. Those rules should be linked to route policy, not hidden in comments.

Step 2: Fail CI When Pricing Inputs Drift

The CI job does not need to scrape every number perfectly. It needs to detect that a source changed and force a human review before route weights or SDK estimates are released.

import hashlib
import json
import os
from pathlib import Path
from urllib.request import Request, urlopen

SNAPSHOT = Path("pricing/snapshots/2026-08-16.json")


def fetch_text(url: str) -> str:
    request = Request(url, headers={"User-Agent": "PricingSnapshotBot/1.0"})
    with urlopen(request, timeout=20) as response:
        return response.read().decode("utf-8", errors="replace")


def stable_hash(text: str) -> str:
    # Hash the visible pricing page enough to detect changes. Store the hash
    # beside the reviewed snapshot after approval.
    compact = " ".join(text.split())
    return hashlib.sha256(compact.encode("utf-8")).hexdigest()


def main() -> int:
    snapshot = json.loads(SNAPSHOT.read_text(encoding="utf-8"))
    current = {}
    for source in snapshot["sources"]:
        current[source["provider"]] = stable_hash(fetch_text(source["url"]))

    expected_path = SNAPSHOT.with_suffix(".hashes.json")
    if not expected_path.exists():
        print(json.dumps(current, indent=2))
        print("Write these hashes after the pricing snapshot is reviewed.")
        return 1

    expected = json.loads(expected_path.read_text(encoding="utf-8"))
    if current != expected:
        print("Pricing source drift detected.")
        print(json.dumps({"expected": expected, "current": current}, indent=2))
        return 1
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

This is intentionally conservative. A changed provider page does not always mean a changed price. It may be a layout update, a new FAQ, or a marketing page adjustment. CI should not auto-publish new numbers. It should stop the release and ask for a dated review.

Step 3: Put the Snapshot ID in SDK Responses

Developers do not need a lecture on every provider price sheet. They do need to know which snapshot powered a request estimate.

{
  "id": "chatcmpl_...",
  "object": "chat.completion",
  "model": "deepseek-v4-flash",
  "usage": {
    "prompt_tokens": 18320,
    "completion_tokens": 740,
    "total_tokens": 19060
  },
  "aiwave": {
    "route": "coding-execution",
    "pricing_snapshot": "2026-08-16",
    "cache_status": "miss",
    "estimated_cost_usd": 0.00277,
    "source_urls": [
      "https://api-docs.deepseek.com/quick_start/pricing/"
    ]
  }
}

The extra object should be optional and namespaced so it does not break OpenAI-compatible clients. When present, it gives product, finance, and SRE teams the same debugging vocabulary: model, route, cache status, output tokens, snapshot date, and estimated cost.

Step 4: Estimate With Cache and Time Windows

Here is a minimal estimator. It is not a billing system. It is the small piece of logic that prevents a route rollout from ignoring cache state or effective date.

from __future__ import annotations

from dataclasses import dataclass
from datetime import datetime, timezone


@dataclass(frozen=True)
class ModelPrice:
    input_cache_hit_per_1m: float
    input_cache_miss_per_1m: float
    output_per_1m: float
    effective_from: datetime


def is_peak_utc(now: datetime) -> bool:
    hour = now.astimezone(timezone.utc).hour
    return 1 <= hour < 4 or 6 <= hour < 10


def estimate_cost(
    price: ModelPrice,
    prompt_tokens: int,
    completion_tokens: int,
    cache_hit: bool,
    now: datetime,
    peak_multiplier: float = 1.0,
    off_peak_multiplier: float = 0.5,
) -> float:
    if now < price.effective_from:
        raise ValueError("pricing snapshot is not yet effective")

    input_rate = (
        price.input_cache_hit_per_1m
        if cache_hit
        else price.input_cache_miss_per_1m
    )
    output_rate = price.output_per_1m
    time_multiplier = peak_multiplier if is_peak_utc(now) else off_peak_multiplier

    input_cost = prompt_tokens / 1_000_000 * input_rate * time_multiplier
    output_cost = completion_tokens / 1_000_000 * output_rate * time_multiplier
    return round(input_cost + output_cost, 8)

Keep the time-window policy explicit. If a provider page says a new billing window is active, the estimator should require an effective timestamp. If a route has no such rule, set both multipliers to 1.0 and document why.

Step 5: Block Stale Route Rollouts

The route policy should reference a pricing snapshot. That makes stale rollouts visible during code review.

route: coding-execution
target_regions:
  - US
  - GB
  - DE
  - JP
pricing_snapshot: "2026-08-16"
primary:
  model: deepseek-v4-flash
  max_output_tokens: 4096
  require_cache_key: true
fallbacks:
  - model: glm-5.1
    max_output_tokens: 4096
  - model: qwen-compatible-route
    max_output_tokens: 2048
rollout:
  canary_percent: 5
  halt_if_estimate_delta_pct_exceeds: 15

When the SDK starts, validate the policy against the pricing data:

import json
from pathlib import Path


def load_route_policy(path: str) -> dict:
    # Use a YAML parser in production. JSON keeps this example dependency-light.
    return json.loads(Path(path).read_text(encoding="utf-8"))


def validate_policy(policy: dict, pricing_snapshot: dict) -> None:
    expected = pricing_snapshot["snapshot_date"]
    actual = policy.get("pricing_snapshot")
    if actual != expected:
        raise RuntimeError(
            f"route policy uses pricing snapshot {actual}, expected {expected}"
        )

    model_names = set(pricing_snapshot["models"])
    route_models = [policy["primary"]["model"]]
    route_models.extend(item["model"] for item in policy.get("fallbacks", []))

    missing = [model for model in route_models if model not in model_names]
    if missing:
        raise RuntimeError(f"route policy references unpriced models: {missing}")

The important part is not the parser. The important part is that route changes and price changes become the same release surface.

Step 6: Make the Rollout Observable

A pricing change is not complete when the pull request merges. Watch it in production. For a unified endpoint, emit one event per request with the fields required for later review:

Field	Example	Reason
`tenant_id_hash`	`acct_4d9...`	Attribute cost without exposing customer identity.
`route`	`coding-execution`	Compare behavior by workload.
`model`	`deepseek-v4-flash`	Join to pricing snapshot.
`pricing_snapshot`	`2026-08-16`	Reconstruct dated assumptions.
`cache_status`	`hit` or `miss`	Explain input cost differences.
`prompt_tokens`	`18320`	Estimate input cost.
`completion_tokens`	`740`	Estimate output cost.
`estimated_cost_usd`	`0.00277`	Compare with later provider usage.
`fallback_count`	`0`	Detect hidden retry cost.
`provider_status`	`200`	Separate billing drift from request failures.

For the first day after a price change, review four charts every few hours:

Chart	Stop condition
Estimate per 1K requests by route	More than the approved delta.
Cache-hit ratio by route	Unexpected cache misses after SDK release.
Output tokens by model	Completion caps not being enforced.
Fallback rate by provider	Retry path consuming the budget.

This is where a unified API can help. If each app team calls every provider directly, each team has to rebuild source tracking, snapshot storage, and cache-aware estimation. If the gateway exposes consistent route metadata, SDK teams can focus on product behavior while still giving finance and SRE enough evidence to review the bill.

A Practical Release Checklist

Use this checklist before changing model weights, SDK defaults, or published pricing examples:

Check	Owner	Pass condition
Source review	Engineering	Official pricing URLs opened and dated.
Snapshot update	SDK maintainer	New append-only JSON file committed.
Hash update	Release manager	CI hashes match reviewed pages.
Route policy update	Platform team	Route references the new snapshot ID.
Runtime metadata	SDK maintainer	Responses expose optional pricing snapshot metadata.
Canary	SRE	Estimate delta stays inside the approved bound.
Invoice review	Finance or platform owner	Provider usage matches estimates within tolerance.

Do not tie this runbook only to DeepSeek. DeepSeek's August 16 change is the obvious trigger today, but the same pattern applies to Kimi's cache economics, GLM tool and modality costs, and QwenCloud's context tiers. Any model market where the API remains compatible while billing rules diverge needs a dated pricing control plane.

Final Takeaway

OpenAI-compatible routing makes Chinese AI models easier to adopt, but compatibility at the request layer does not remove billing complexity. The teams that handle this well will not be the teams with the longest spreadsheet. They will be the teams that treat pricing as versioned production data.

For an SDK maintainer, that means four habits: source pricing from official pages, store append-only snapshots, bind route policies to snapshot IDs, and expose enough metadata for customers to understand what happened. With that runbook in place, DeepSeek, Kimi, GLM, Qwen, and unified gateways such as AIWave can be evaluated as production routes instead of one-off experiments.

Build a Billing Reconciliation Harness for Chinese AI Model Routers

Mattias chaw — Sat, 15 Aug 2026 13:09:52 +0000

Build a Billing Reconciliation Harness for Chinese AI Model Routers

Chinese AI APIs are moving fast enough that a static cost spreadsheet is now a liability. DeepSeek has versioned V4 Flash and V4 Pro pricing, Kimi K3 exposes separate cache-hit and cache-miss input rates for 1M-token contexts, Z.AI publishes GLM cached-input rates and tool fees, and QwenCloud uses context tiers for long prompts. If your application routes requests across these models, the invoice risk is not only "which model did we call?" It is "which dated price sheet did the router believe, which cache state did the provider apply, and which tokens actually landed on the bill?"

That is why production teams should treat pricing as a reconciled data feed rather than a blog-post constant. A router can choose a model in milliseconds, but finance, SRE, and product teams need a daily way to prove that request-level cost estimates match provider billing within a tolerable margin. This article shows a practical pattern: build a billing reconciliation harness that records dated pricing snapshots, estimates each request before dispatch, imports provider usage later, and flags drift before it becomes a customer-facing surprise.

The examples use a neutral OpenAI-compatible gateway shape, so you can run the same pattern with direct provider APIs or a unified endpoint such as AIWave. Code samples use environment variables and placeholders only.

Why Billing Reconciliation Belongs in the Router

Most teams start with a simple route table:

Workload	Primary model	Fallback model	Cost control
Coding agent planning	DeepSeek V4 Pro	GLM-5.1	Cap output tokens
Coding agent execution	DeepSeek V4 Flash	qwen3.7-plus	Prefer cached context
Long repository analysis	Kimi K3	Qwen long-context model	Track cache-hit rate
Structured extraction	GLM-5.1	DeepSeek V4 Flash	Validate JSON retries

That table is useful, but it hides three billing realities.

First, cache-hit input and cache-miss input are different products. Kimi K3, DeepSeek, and GLM pricing all make repeated context materially different from fresh context. A single "input tokens" column cannot explain the bill.

Second, some price sheets now need effective dates. DeepSeek's official documentation and recent market reporting point to dated V4 pricing changes, including peak and off-peak behavior. If you overwrite yesterday's price row, you can no longer explain yesterday's invoice.

Third, long-context tiers can change the marginal rate. QwenCloud documents per-model text pricing where long input windows can move into higher tiers. A route that is economical at 80K input tokens might not remain economical at 600K.

The router is the right place to capture the expected billing dimensions because it already knows the model, route, tenant, prompt size, completion cap, cache key, and fallback path. Reconciliation then becomes a deterministic join instead of an incident review.

Pricing Snapshot Shape

Do not store pricing as one mutable JSON object per provider. Store append-only snapshots with effective_from, source_url, and a version hash. As of August 15, 2026, the public pages worth checking before a production rollout include:

Provider	Dated pricing fields to snapshot	Operational concern
DeepSeek API docs	V4 Flash and V4 Pro cache-hit input, cache-miss input, output, context length, effective pricing windows	Peak/off-peak changes and model-version drift
Kimi K3 pricing	Cache-hit input, cache-miss input, output, 1M-token context	Long-context cache economics
Z.AI pricing	GLM input, cached input, output, storage and tool fees	Tool-call and cached-input attribution
QwenCloud pricing	Model-specific input tiers, output rates, marketplace references	Context-tier transitions

Your snapshot does not need to mirror every provider page. It needs enough structure to reproduce the estimate that your router gave a request at the time it was sent.

{
  "provider": "deepseek",
  "model": "deepseek-v4-pro",
  "effective_from": "2026-08-15T00:00:00Z",
  "effective_until": "2026-08-16T16:00:00Z",
  "currency": "USD",
  "unit": "1M_tokens",
  "input_cache_hit": 0.003625,
  "input_cache_miss": 0.435,
  "output": 0.87,
  "source_url": "https://api-docs.deepseek.com/quick_start/pricing/",
  "checked_at": "2026-08-15T12:00:00Z"
}

For Qwen-style context tiers, model the input side as bands rather than a scalar.

{
  "provider": "qwencloud",
  "model": "qwen3.7-plus",
  "effective_from": "2026-08-15T00:00:00Z",
  "currency": "USD",
  "unit": "1M_tokens",
  "input_tiers": [
    {"max_input_tokens": 256000, "rate": 0.40},
    {"max_input_tokens": 1000000, "rate": 1.20}
  ],
  "output": 1.60,
  "source_url": "https://docs.qwencloud.com/developer-guides/getting-started/pricing",
  "checked_at": "2026-08-15T12:00:00Z"
}

The point is not to claim that these values will remain stable. The point is to make instability auditable.

Request Ledger

Every routed request should write a compact billing ledger row before returning to the caller. Keep it small enough for hot-path writes, but rich enough for reconciliation.

import os
import time
import uuid
from dataclasses import dataclass, asdict

@dataclass
class BillingLedgerRow:
    request_id: str
    tenant_id: str
    route_name: str
    provider: str
    model: str
    pricing_snapshot_id: str
    prompt_tokens: int
    cached_prompt_tokens: int
    completion_tokens: int
    estimated_cost_usd: float
    cache_key_version: str
    fallback_from: str | None
    created_at_ms: int

def make_ledger_row(
    *,
    tenant_id: str,
    route_name: str,
    provider: str,
    model: str,
    pricing_snapshot_id: str,
    prompt_tokens: int,
    cached_prompt_tokens: int,
    completion_tokens: int,
    estimated_cost_usd: float,
    cache_key_version: str,
    fallback_from: str | None = None,
) -> dict:
    return asdict(BillingLedgerRow(
        request_id=str(uuid.uuid4()),
        tenant_id=tenant_id,
        route_name=route_name,
        provider=provider,
        model=model,
        pricing_snapshot_id=pricing_snapshot_id,
        prompt_tokens=prompt_tokens,
        cached_prompt_tokens=cached_prompt_tokens,
        completion_tokens=completion_tokens,
        estimated_cost_usd=round(estimated_cost_usd, 8),
        cache_key_version=cache_key_version,
        fallback_from=fallback_from,
        created_at_ms=int(time.time() * 1000),
    ))

AIWAVE_API_KEY = os.environ.get("AIWAVE_API_KEY", "YOUR_API_KEY_HERE")

Do not log prompts in this ledger. Reconciliation needs token counts, provider identifiers, route decisions, and price snapshot identifiers. It does not need raw customer content.

Estimation Function

A reconciler should use the same estimator that the router used, or at least the same rules. That prevents "the dashboard says one thing, the router says another" drift.

def estimate_token_cost(snapshot: dict, prompt_tokens: int, cached_tokens: int, output_tokens: int) -> float:
    cached_tokens = min(max(cached_tokens, 0), prompt_tokens)
    uncached_tokens = prompt_tokens - cached_tokens

    if "input_tiers" in snapshot:
        input_rate = rate_for_tier(snapshot["input_tiers"], prompt_tokens)
        input_cost = (prompt_tokens / 1_000_000) * input_rate
    else:
        input_cost = (
            (cached_tokens / 1_000_000) * snapshot.get("input_cache_hit", 0)
            + (uncached_tokens / 1_000_000) * snapshot["input_cache_miss"]
        )

    output_cost = (output_tokens / 1_000_000) * snapshot["output"]
    return input_cost + output_cost

def rate_for_tier(tiers: list[dict], prompt_tokens: int) -> float:
    for tier in tiers:
        if prompt_tokens <= tier["max_input_tokens"]:
            return tier["rate"]
    return tiers[-1]["rate"]

This is intentionally boring code. Billing logic should be clear enough for engineering, finance, and support to read together.

Reconciliation Job

Run reconciliation after provider usage data is available. Some providers expose usage exports by API key; others require invoice exports, dashboard downloads, or gateway-level aggregation. Normalize them into provider usage rows.

from collections import defaultdict

def reconcile(ledger_rows: list[dict], provider_rows: list[dict], tolerance: float = 0.03) -> list[dict]:
    expected = defaultdict(float)
    observed = defaultdict(float)

    for row in ledger_rows:
        key = (row["provider"], row["model"], row["route_name"])
        expected[key] += row["estimated_cost_usd"]

    for row in provider_rows:
        key = (row["provider"], row["model"], row["route_name"])
        observed[key] += row["billed_cost_usd"]

    findings = []
    for key, expected_cost in expected.items():
        billed_cost = observed.get(key, 0.0)
        if expected_cost == 0:
            continue
        delta_ratio = (billed_cost - expected_cost) / expected_cost
        if abs(delta_ratio) > tolerance:
            findings.append({
                "provider": key[0],
                "model": key[1],
                "route_name": key[2],
                "expected_cost_usd": round(expected_cost, 4),
                "billed_cost_usd": round(billed_cost, 4),
                "delta_ratio": round(delta_ratio, 4),
            })
    return findings

Start with a 3 percent tolerance, then tune by provider. Small differences can come from tokenization updates, rounding, retries, streaming cutoffs, and provider-side cache attribution. Large differences usually mean one of five things:

The router used a stale pricing snapshot.
The provider counted fewer cached tokens than expected.
A fallback route changed the model but not the ledger row.
Long-context tiering was missed.
Tool-call or web-search charges were not included.

That list is more useful than a single blended "cost went up" alert.

Alert Routing

Do not page a human every time a provider bill differs by pennies. Route reconciliation findings by blast radius.

Finding	Owner	Action
One tenant, one route, small delta	Support or growth engineer	Add explanation to customer usage view
One model, many tenants	Platform engineer	Check pricing snapshot and route table
One provider, all routes	SRE and finance	Compare provider export with billing dashboard
Cache-hit collapse after deploy	Application owner	Review prompt template and cache key version
Output-token spike	Product owner	Inspect max token caps and task type

For Tier 1 and Tier 2 customers, the valuable experience is not a promise that prices never move. It is a system that can explain changes with dates, provider sources, and request-level evidence.

Dashboard Metrics That Matter

Once the harness exists, expose a small set of operational metrics:

Estimated vs billed cost by provider, model, route, and tenant.
Cache-hit token share by route and prompt-template version.
Effective price snapshot age.
Percentage of requests with unknown provider usage.
Fallback cost multiplier.
Output-token share of total spend.
Long-context tier distribution.

The two best early warning metrics are snapshot age and cache-hit token share. Snapshot age catches pricing drift before a bill arrives. Cache-hit share catches prompt-template changes that quietly turn repeated context into fresh input.

Production Rollout Checklist

Before turning reconciliation into customer-facing reporting, run it privately for at least one billing cycle.

Store append-only pricing snapshots with source URLs and checked dates.
Record pricing_snapshot_id on every routed request.
Keep prompt content out of billing rows.
Track cached and uncached input tokens separately.
Preserve fallback origin and final model.
Normalize provider exports into a stable internal schema.
Alert on route-level deltas, not only account-level totals.
Review official pricing pages before planned route-table changes.

This is also a good place to document how your gateway handles failed calls. Some providers do not bill failed requests; others may have provider-specific edge cases around timeouts, streaming interruptions, or tool calls. Your ledger should capture enough status data to separate "request estimated" from "request completed."

Closing

Chinese AI model access is becoming a routing problem, not a single-provider integration. DeepSeek, Kimi, GLM, and Qwen each expose different economics around cache, context, output, tools, and effective dates. A billing reconciliation harness turns that volatility into an engineering control: every route decision gets tied to a dated price snapshot, every invoice gets compared against request-level estimates, and every drift alert points to a fixable cause.

If you are building a multi-model API layer for production agents, start with this harness before the first customer asks why this week's usage looks different from last week's. It is easier to explain a dated ledger than to reverse-engineer a bill from logs after the route table has already changed.

Build a Runtime Cost Circuit Breaker for Chinese AI API Routers

Mattias chaw — Fri, 14 Aug 2026 13:12:16 +0000

Build a Runtime Cost Circuit Breaker for Chinese AI API Routers

Most AI cost controls are too late.

A dashboard that updates every hour is useful for finance review. A daily usage export is useful for reconciliation. A spreadsheet with current provider rates is useful during planning. None of those controls can stop a coding agent that just entered a retry loop, a summarizer that is sending the same 400K-token context on every turn, or a customer workflow that switched from a compact execution model to a long-context reasoning model without changing its output cap.

For production teams in the United States, the United Kingdom, Canada, Germany, the Netherlands, Japan, Singapore, Korea, Australia, and other Tier 1/2 markets, the safer pattern is a runtime cost circuit breaker. The router estimates each request before it leaves your application, records the actual usage after the response returns, and opens the circuit when a route, tenant, workflow, or model family starts behaving outside policy.

This article shows a practical design for Chinese AI API routers that call DeepSeek, Qwen, GLM, Kimi, or an OpenAI-compatible aggregator such as AIWave. It is not a benchmark and it is not a price-ranking table. It is an engineering control for teams that want model choice without runaway spend.

The examples use placeholders and environment variables only.

Pricing facts checked on August 14, 2026

Runtime breakers need dated pricing inputs. A route policy without a checked date is not an operational fact; it is an old assumption.

The public source pages below were checked on August 14, 2026:

Provider family	Public pricing shape	Breaker implication
DeepSeek V4 Flash	DeepSeek lists `deepseek-v4-flash` at $0.0028 per 1M cache-hit input tokens, $0.14 per 1M cache-miss input tokens, and $0.28 per 1M output tokens. It also lists a 1M context window and a future peak/off-peak schedule taking effect on August 16, 2026 at 16:00 UTC.	The breaker needs current and scheduled rate tables, not one permanent constant.
DeepSeek V4 Pro	DeepSeek lists `deepseek-v4-pro` at $0.003625 cache-hit input, $0.435 cache-miss input, and $0.87 output per 1M tokens, with a 1M context window.	Planning routes should get tighter tenant and workflow caps than execution routes.
Kimi K3	Kimi lists `kimi-k3` at $0.30 cache-hit input, $3.00 cache-miss input, and $15.00 output per 1M tokens, with a 1,048,576-token context window.	Long-context routes need prefix reuse checks and strict output budgets.
Z.AI GLM	Z.AI lists GLM-5.2 and GLM-5.1 at $1.40 input, $0.26 cached input, and $4.40 output per 1M tokens. GLM-5 is listed at $1.00 input, $0.20 cached input, and $3.20 output.	Cached input and output growth should be tracked separately.
QwenCloud	QwenCloud documents pay-as-you-go text billing per million tokens, tiered pricing by request size, Batch API at 50% of real-time rates, model-specific cache discounts, and thinking tokens billed as output.	The breaker must store model-specific rules, input bands, and thinking mode behavior.

Sources:

These prices are source facts for an example control plane. Your application should check the pricing source that it actually bills through, then store the checked date next to the route policy.

What the breaker protects

A runtime breaker protects four failure modes that dashboards usually catch after damage is done.

The first failure mode is request expansion. A prompt template change can turn a 12K-token request into a 120K-token request. The model may still handle it, especially with 1M-context routes, but the unit economics are now different.

The second failure mode is output expansion. Agent loops, verbose tool summaries, and weak stop conditions can push generated tokens far beyond the product value of the response. Output is often the expensive side of the call, so it deserves a separate cap.

The third failure mode is cache collapse. A long-context workflow may look sustainable when repeated prefixes hit cache. A small change in prompt ordering, per-user metadata, or retrieved document order can turn cache-hit input into cache-miss input.

The fourth failure mode is route drift. A router may move traffic from a compact model to a reasoning model, from a Flash route to a Pro route, or from one aggregator route to another. The request still succeeds, but its cost envelope changes.

The breaker should not decide whether a model is good. It should decide whether a request is allowed to spend within the policy that product, finance, and engineering already approved.

Define the policy at the workflow level

Do not start with a global dollar cap. Global caps are blunt and create messy incidents: one noisy tenant can block unrelated customers, or one expensive admin workflow can consume the budget intended for user-facing traffic.

Start with workflow policy:

Workflow	Normal route	Allowed fallback	Preflight ceiling	Rolling window
Support reply draft	`deepseek-v4-flash`	GLM or Qwen execution route	$0.005 per request	$20 per tenant per day
Code review plan	`deepseek-v4-pro`	Kimi K3 only with approval flag	$0.05 per request	$100 per tenant per day
Contract summary	GLM or Qwen long-context route	DeepSeek Pro only for escalation	$0.03 per request	$60 per tenant per day
Retrieval answer	Qwen or DeepSeek Flash	Same family compact route	$0.01 per request	$30 per tenant per day

The exact numbers above are examples. The important part is the shape:

one workflow name;
one primary route;
explicit fallback routes;
one preflight ceiling;
one rolling tenant window;
one escalation path when the breaker opens.

The router can then reject, downgrade, trim context, or require human approval before an expensive call happens.

Store prices as versioned route metadata

Keep prices out of prompt code. Store them as route metadata with a source URL and checked date.

from dataclasses import dataclass
from decimal import Decimal
from typing import Optional


@dataclass(frozen=True)
class PriceCard:
    route: str
    model: str
    source_url: str
    checked_date: str
    input_per_mtok: Decimal
    output_per_mtok: Decimal
    cached_input_per_mtok: Optional[Decimal] = None
    effective_from_utc: Optional[str] = None


PRICE_CARDS = {
    "deepseek_flash_current": PriceCard(
        route="deepseek_flash_current",
        model="deepseek-v4-flash",
        source_url="https://api-docs.deepseek.com/quick_start/pricing/",
        checked_date="2026-08-14",
        input_per_mtok=Decimal("0.14"),
        cached_input_per_mtok=Decimal("0.0028"),
        output_per_mtok=Decimal("0.28"),
    ),
    "kimi_k3_current": PriceCard(
        route="kimi_k3_current",
        model="kimi-k3",
        source_url="https://www.kimi.com/resources/kimi-k3-pricing",
        checked_date="2026-08-14",
        input_per_mtok=Decimal("3.00"),
        cached_input_per_mtok=Decimal("0.30"),
        output_per_mtok=Decimal("15.00"),
    ),
}

Use Decimal, not float, for price math. Also keep scheduled price cards when a provider has announced a future change. DeepSeek's page, for example, lists a new peak/off-peak schedule effective August 16, 2026 at 16:00 UTC. A production router should be able to load the correct card by request time.

Estimate before sending the request

Preflight estimation does not need perfect token accounting to be useful. It needs to be conservative enough to block obvious mistakes.

from decimal import Decimal


def estimate_cost_usd(
    card: PriceCard,
    input_tokens: int,
    max_output_tokens: int,
    cached_input_tokens: int = 0,
) -> Decimal:
    uncached = max(input_tokens - cached_input_tokens, 0)
    cached_rate = card.cached_input_per_mtok or card.input_per_mtok
    input_cost = Decimal(uncached) * card.input_per_mtok / Decimal(1_000_000)
    cache_cost = Decimal(cached_input_tokens) * cached_rate / Decimal(1_000_000)
    output_cost = Decimal(max_output_tokens) * card.output_per_mtok / Decimal(1_000_000)
    return input_cost + cache_cost + output_cost


def should_allow_request(policy, estimate: Decimal, tenant_spend_today: Decimal) -> bool:
    if estimate > policy.preflight_ceiling_usd:
        return False
    if tenant_spend_today + estimate > policy.daily_tenant_ceiling_usd:
        return False
    return True

Use the configured max_tokens or max_completion_tokens for the output side. Do not estimate with the average output from yesterday. If the request allows the model to generate 64K tokens, the breaker should evaluate the request as if that output could happen.

For long-context workflows, estimate both cache-hit and cache-miss scenarios. If the route only works economically when cache hits are high, make that an explicit policy:

def require_cache_health(observed_hit_ratio: Decimal, required_hit_ratio: Decimal) -> None:
    if observed_hit_ratio < required_hit_ratio:
        raise RuntimeError("cache_hit_ratio_below_route_policy")

That error is not a provider failure. It is a release control doing its job.

Route with downgrade states

Opening the breaker should not always mean dropping the request. A good router has downgrade states.

The first state is trim. Remove optional retrieved documents, reduce conversation history, or switch from full files to relevant snippets.

The second state is compact. Use a smaller execution route for tasks that do not need deep planning.

The third state is defer. Move the work to an async queue, especially when a provider has real-time versus batch pricing differences.

The fourth state is approval. Ask for a human flag before a high-value workflow uses a long-context or reasoning route.

Here is a small router shape:

import os
from openai import OpenAI


client = OpenAI(
    api_key=os.environ.get("AIWAVE_API_KEY", "YOUR_API_KEY_HERE"),
    base_url="https://aiwave.live/v1",
)


def complete_with_breaker(workflow, messages, token_estimate, tenant_state):
    route = choose_primary_route(workflow)
    card = load_price_card(route)
    policy = load_workflow_policy(workflow)

    estimate = estimate_cost_usd(
        card=card,
        input_tokens=token_estimate.input_tokens,
        cached_input_tokens=token_estimate.cached_input_tokens,
        max_output_tokens=policy.max_output_tokens,
    )

    if not should_allow_request(policy, estimate, tenant_state.spend_today):
        route, messages = downgrade_route(workflow, route, messages, tenant_state)
        card = load_price_card(route)

    response = client.chat.completions.create(
        model=card.model,
        messages=messages,
        max_tokens=policy.max_output_tokens,
        temperature=policy.temperature,
    )

    record_actual_usage(
        workflow=workflow,
        route=route,
        model=card.model,
        usage=response.usage,
        price_card=card,
    )
    return response

The code assumes you have local helpers such as choose_primary_route, load_price_card, and record_actual_usage. Keep those boring. The business value is in the policy and the event log, not in clever routing code.

Record actual usage after every call

Preflight protects the front door. Actual usage closes the accounting loop.

Record at least:

Field	Reason
`tenant_id`	Per-customer ceilings and anomaly detection
`workflow`	Product-level budget ownership
`route`	Router policy review
`model`	Provider reconciliation
`source_url` and `checked_date`	Auditability for pricing assumptions
`input_tokens`	Prompt expansion detection
`cached_input_tokens`	Cache health
`output_tokens`	Runaway generation detection
`estimated_usd`	Preflight behavior
`actual_usd`	Reconciliation
`downgrade_reason`	Product and support review

This is also where an aggregator such as AIWave can be useful. If your application uses one OpenAI-compatible endpoint for 25+ Chinese models, your own router can stay focused on workflow policy while the platform handles provider access and model catalog exposure. You still need your own breaker because only your application knows which tenant, workflow, and customer action created the request.

Alert on slope, not just totals

A daily ceiling is necessary, but it is too slow by itself. Alert on slope:

spend per tenant over the last 5 minutes;
output tokens per request over the last 20 calls;
cache-hit ratio by route over the last 50 calls;
downgrade count by workflow over the last hour;
request estimate versus actual cost variance.

Slope alerts catch bad deploys quickly. If a new prompt version doubles output tokens, you want the breaker to open before the daily cap is consumed. If cache-hit ratio drops from 80% to 10%, you want a route-level alert before finance asks why the long-context bill changed.

Keep policy separate from provider preference

Production AI routing has two different decisions:

Which model is appropriate for this workflow?
Is this specific request allowed to spend under current policy?

Do not merge those decisions into one if-statement. Model choice may depend on quality, latency, context length, tool support, region policy, and customer tier. Spend permission depends on estimated tokens, current price cards, tenant windows, and route health.

When those decisions are separate, the team can change pricing policy without rewriting model selection, and can change model selection without bypassing cost controls.

A release checklist for the breaker

Before turning on a new Chinese AI model route, require:

a dated price card with source URL;
a workflow policy with preflight and rolling-window ceilings;
a max output cap;
a cache-hit policy for long-context workflows;
a downgrade path;
an actual-usage ledger;
slope alerts;
a rollback flag that can disable the route without redeploying application code.

This checklist is intentionally small. It fits into a pull request. It gives engineering, product, finance, and security one shared object to review.

Final pattern

Chinese AI APIs are now varied enough that a single static provider table is fragile. DeepSeek exposes cache-hit and cache-miss pricing with scheduled peak/off-peak changes. Kimi K3 makes 1M-token context practical, but the output side needs guardrails. GLM and Qwen routes have their own cached input, tool, thinking, and tiered billing rules. Aggregators such as AIWave make access and switching simpler, but the application still needs workflow-aware controls.

The runtime cost circuit breaker is that control.

Estimate before the request. Record after the response. Open the circuit on route, tenant, workflow, and cache anomalies. Downgrade instead of failing when product value allows it. Store source dates next to every price card.

That is how a team can use multiple Chinese AI model families in production without turning every model launch into a billing surprise.