Build a Route Readiness Scorecard for Chinese AI APIs

#python #ai #api #devops

Changing an OpenAI SDK base URL is easy. Deciding that a new model route is ready for production is harder.

That distinction matters for teams in the United States, the United Kingdom, Germany, the Netherlands, Japan, Singapore, and other Tier 1/2 markets. A developer can point an OpenAI-compatible client at a gateway such as AIWave and test DeepSeek, Qwen, GLM, Kimi, ERNIE, or MiniMax models behind one API surface. The first smoke test may pass in minutes. The production question is not whether one prompt returned a plausible answer. The question is whether the route is ready to carry customer traffic with clear pricing evidence, data policy, cache assumptions, usage logs, and rollback behavior.

The pattern I use is a route readiness scorecard. It is a small review artifact that sits between "the API call works" and "this model can serve paid workflows."

This article shows how to build one. The code examples use environment variables for credentials and do not include real keys.

Why a scorecard beats a model picker

Most model pickers collapse too many decisions into one field:

{
  "support_summary_model": "glm-5.2"
}

That is not enough for a production SaaS route. You also need to know:

whether the endpoint shape matches your client;
whether the model supports the response features your parser depends on;
whether the request may contain personal data;
whether your customer region policy allows the route;
whether the rate card was checked recently;
whether cached input is expected and visible;
whether retries or fallbacks can change cost;
whether a rollback can happen without redeploying the app.

The scorecard makes those decisions explicit. It also creates a review surface for engineering, security, product, and finance. A team can disagree with a score, but at least they are disagreeing about named risks rather than a vague preference for one model family.

Pricing shape checked on August 10, 2026

The pricing values below were checked against public provider pages on August 10, 2026. Treat them as source data for examples, not permanent constants. Account-level pricing, promotions, route availability, taxes, and gateway billing can differ, so production systems should read the pricing source they actually bill through before release.

Provider family	Representative current pricing shape	What the scorecard should capture
QwenCloud `qwen3.7-flash`	Up to 32K input: $0.03 input and $0.13 output per 1M tokens; 32K to 256K: $0.10 input and $0.40 output; 256K to 1M: $0.20 input and $0.80 output.	Input-token band, context ceiling, output cap, and tool fees.
QwenCloud `qwen3.7-plus`	Up to 256K input: $0.40 input and $1.60 output per 1M tokens; 256K to 1M: $1.20 input and $4.80 output.	Whether the workflow can cross a tier boundary.
Z.AI GLM-5.2 / GLM-5.1	$1.40 input, $0.26 cached input, and $4.40 output per 1M tokens.	Cached input ratio, output sensitivity, and cache storage policy.
Kimi K3	$3.00 cache-miss input, $0.30 cache-hit input, and $15.00 output per 1M tokens with a 1,048,576-token context window.	Cache-hit rate, output-token cap, and long-context route reason.

These rows show why "tokens total" is a weak metric. Qwen long-context cost depends on the input band. GLM cost depends heavily on whether stable prefixes become cached input. Kimi K3 can be compelling for a high-value long-context task, but output growth must be controlled. A gateway can simplify access, but it does not remove the need for route-specific accounting.

The scorecard fields

Start with a plain data object. Store it next to the route config and review it like application code.

route_id: support_reply_draft_v2
checked_on: 2026-08-10
gateway:
  base_url: https://aiwave.live/v1
  endpoint_family: openai_chat_completions
candidate_model:
  family: glm
  model: glm-5.2
workflow:
  task_class: support_draft
  customer_visible: true
  max_input_tokens: 24000
  max_output_tokens: 900
  requires_streaming: false
  requires_json_object: true
data_policy:
  personal_data_possible: true
  allowed_regions:
    - United States
    - United Kingdom
    - Germany
    - Netherlands
  retention_class: metadata_only
pricing:
  source_url: https://docs.z.ai/guides/overview/pricing
  input_usd_per_1m: "1.40"
  cached_input_usd_per_1m: "0.26"
  output_usd_per_1m: "4.40"
operations:
  feature_flag: ai_route_support_reply_glm52
  rollback_route: support_reply_previous_provider
  owner: platform-ai
  review_after_days: 14

Use strings for money values so they round-trip cleanly into Decimal. Do not store real API keys, payment identifiers, customer emails, raw prompts, or private customer names in the scorecard.

Score the route before traffic moves

A route is ready only when it passes the minimum bar for integration, policy, cost, observability, and operations. The scoring function can be simple.

from dataclasses import dataclass
from decimal import Decimal


@dataclass(frozen=True)
class RouteScorecard:
    route_id: str
    endpoint_verified: bool
    behavior_probe_passed: bool
    data_policy_approved: bool
    pricing_checked_on: str
    input_usd_per_1m: Decimal
    output_usd_per_1m: Decimal
    max_input_tokens: int
    max_output_tokens: int
    max_request_usd: Decimal
    ledger_enabled: bool
    rollback_enabled: bool


def estimate_ceiling_usd(card: RouteScorecard) -> Decimal:
    input_cost = (
        Decimal(card.max_input_tokens)
        * card.input_usd_per_1m
        / Decimal(1_000_000)
    )
    output_cost = (
        Decimal(card.max_output_tokens)
        * card.output_usd_per_1m
        / Decimal(1_000_000)
    )
    return input_cost + output_cost


def readiness(card: RouteScorecard) -> dict:
    blockers = []
    if not card.endpoint_verified:
        blockers.append("endpoint_not_verified")
    if not card.behavior_probe_passed:
        blockers.append("behavior_probe_failed")
    if not card.data_policy_approved:
        blockers.append("data_policy_not_approved")
    if estimate_ceiling_usd(card) > card.max_request_usd:
        blockers.append("request_budget_exceeded")
    if not card.ledger_enabled:
        blockers.append("usage_ledger_missing")
    if not card.rollback_enabled:
        blockers.append("rollback_missing")

    return {
        "route_id": card.route_id,
        "ready": not blockers,
        "blockers": blockers,
        "estimated_ceiling_usd": str(estimate_ceiling_usd(card)),
        "pricing_checked_on": card.pricing_checked_on,
    }

This does not need to be a complex optimizer. The purpose is to stop a risky route from becoming the default path just because one demo worked.

Run behavior probes with the same client shape

OpenAI-compatible APIs reduce SDK churn, but compatibility is still something to verify. Run small probes for the exact behavior your workflow needs.

import os
from openai import OpenAI


client = OpenAI(
    api_key=os.environ["AIWAVE_API_KEY"],
    base_url=os.environ.get("AIWAVE_BASE_URL", "https://aiwave.live/v1"),
)


def probe_json_route(model: str) -> dict:
    response = client.chat.completions.create(
        model=model,
        messages=[
            {"role": "system", "content": "Return valid JSON only."},
            {"role": "user", "content": '{"status":"ready","risk":"low"}'},
        ],
        response_format={"type": "json_object"},
        temperature=0.1,
        max_tokens=80,
    )

    content = response.choices[0].message.content or ""
    return {
        "model": model,
        "has_content": bool(content.strip()),
        "has_usage": response.usage is not None,
        "content_preview": content[:120],
    }

Keep probes boring. They are not benchmarks. They should catch rejected parameters, missing usage, empty responses, streaming incompatibilities, JSON-shape failures, or account-level model access issues.

For a support drafting workflow, run a basic chat probe and a JSON probe. For a coding agent, add a tool-call probe and a streaming probe. For a long-context route, add a cache-sensitive probe using a stable prefix and log whether cached tokens are visible in usage metadata.

Add policy before model selection

Do not let a router choose the model before it knows the customer and data policy. A route that is acceptable for a non-sensitive internal evaluation may be wrong for a customer workflow containing personal data.

from dataclasses import dataclass


@dataclass(frozen=True)
class CustomerPolicy:
    region: str
    allows_personal_data_route: bool
    allowed_model_families: set[str]
    max_request_usd: Decimal


def approve_policy(
    policy: CustomerPolicy,
    model_family: str,
    has_personal_data: bool,
    estimated_usd: Decimal,
) -> tuple[bool, str]:
    if model_family not in policy.allowed_model_families:
        return False, "model_family_blocked"
    if has_personal_data and not policy.allows_personal_data_route:
        return False, "data_policy_blocked"
    if estimated_usd > policy.max_request_usd:
        return False, "budget_blocked"
    return True, "approved"

This is the part Tier 1/2 buyers care about during review. The answer should not be "the provider is OpenAI-compatible." The answer should be "this workflow uses this route, under this customer policy, with this retention class, this budget, this checked pricing source, and this rollback flag."

Store usage metadata, not raw prompts

The route readiness scorecard should require a ledger row for every accepted request and every rejected request. Rejections are useful because they show product demand that policy or budget did not allow.

from datetime import datetime, timezone
import hashlib
import json


def hash_tenant(tenant_id: str) -> str:
    return hashlib.sha256(tenant_id.encode("utf-8")).hexdigest()[:16]


def ledger_row(
    tenant_id: str,
    route_id: str,
    model: str,
    policy_result: str,
    estimated_usd: Decimal,
    pricing_source: str,
    pricing_checked_on: str,
) -> dict:
    return {
        "timestamp": datetime.now(timezone.utc).isoformat(),
        "tenant_hash": hash_tenant(tenant_id),
        "route_id": route_id,
        "model": model,
        "policy_result": policy_result,
        "estimated_usd": str(estimated_usd),
        "pricing_source": pricing_source,
        "pricing_checked_on": pricing_checked_on,
        "retention_class": "metadata_only",
    }


print(json.dumps(
    ledger_row(
        tenant_id="customer-123",
        route_id="support_reply_draft_v2",
        model="glm-5.2",
        policy_result="approved",
        estimated_usd=Decimal("0.042"),
        pricing_source="https://docs.z.ai/guides/overview/pricing",
        pricing_checked_on="2026-08-10",
    ),
    indent=2,
))

A metadata-first ledger helps engineering debug route behavior without collecting more customer content than needed. Add request ID, retry count, fallback route, prompt-token estimate, returned usage, cached tokens, output tokens, and validation result in production.

A practical scoring rubric

Use a small numeric rubric only after the hard blockers pass.

Category	Question	Score
Endpoint	Does the route work with the actual OpenAI-compatible client path?	0 to 2
Behavior	Did probes pass for streaming, JSON, tools, and usage fields required by the workflow?	0 to 3
Pricing	Was the rate source checked in the same release window and stored with the route?	0 to 2
Cost ceiling	Is the worst-case request estimate inside policy?	0 to 3
Data policy	Is the customer region and personal-data policy explicit?	0 to 3
Observability	Are accepted and rejected attempts logged with metadata?	0 to 2
Rollback	Can the route be disabled without a deployment?	0 to 2

Set the launch threshold according to workflow risk. A background summarizer for internal documents may need a lower score than a customer-visible support reply generator. A route that may process personal data should have policy approval, metadata-only retention, and rollback as hard requirements, not optional points.

How this applies to AIWave

AIWave is useful when a team wants OpenAI-compatible access to Chinese model families through one gateway rather than maintaining separate provider integrations. That is an integration advantage. The route readiness scorecard turns the integration into an operating practice.

For example, a team could keep the same OpenAI SDK shape, set base_url to https://aiwave.live/v1, and test routes for Qwen, GLM, Kimi, DeepSeek, and ERNIE. The scorecard then decides which routes can serve which workflows:

Qwen routes for bounded application tasks with visible input-token bands.
GLM routes for structured reasoning where cached input can be measured.
Kimi routes for high-value long-context work with explicit output caps.
DeepSeek routes for cache-aware planning and execution splits.
ERNIE routes for workloads where the model's behavior passes the same probes as any other candidate.

The point is not to crown one model. The point is to make every production route explain itself.

Release rule

Here is the release rule I recommend:

A Chinese AI API route is not ready for production until the endpoint probe, behavior probe, policy check, dated pricing check, usage ledger, and rollback flag all pass in the same release window.

That rule is intentionally strict. It saves time later. When a customer asks why a workflow used a given model, why a request was blocked, or why a bill changed after a model migration, the answer is already in the scorecard and ledger.

OpenAI-compatible gateways make experimentation faster. Route readiness makes experimentation deployable.

Sources checked on August 10, 2026: