Build an Admission Controller for Chinese AI API Gateways

OpenAI-compatible gateways make Chinese model adoption easier, but the hardest production bugs often happen before the request reaches the provider. A coding agent sends a one-million-token repository snapshot without a cache key. A support workflow enables web search on every retry. A customer batch job moves from 31K to 33K input tokens and crosses a context billing tier. A fallback path quietly switches from a text model to a multimodal route. The API shape still looks familiar, but the operational profile has changed.

That is why a serious gateway needs an admission controller.

An admission controller is a small policy layer that runs before dispatch. It decides whether a request can be sent now, should be reshaped, should be queued, should use a different route, or should be rejected with a precise explanation. Kubernetes uses admission control to protect clusters. Multi-model AI gateways need the same idea for token budgets, context tiers, cache requirements, output caps, tool fees, regional policy, and provider capacity.

This article shows a practical admission controller for Tier 1 and Tier 2 engineering teams building with DeepSeek, Kimi K3, GLM, Qwen, ERNIE, or a unified endpoint such as AIWave. The examples use placeholders only and do not require real credentials.

Source Snapshot for August 17, 2026

The policy should be built from dated sources, not from memory. These are the public pages I would check before shipping today's rules:

Provider	Source	Admission-control signal
DeepSeek	Models & Pricing and Rate Limit & Isolation	V4 Flash and V4 Pro list 1M context, cache-hit and cache-miss input rates, output rates, peak/off-peak windows, and account-level concurrency limits.
Kimi	Kimi K3 Pricing	Kimi K3 publishes a 1,048,576-token context window with separate cache-hit, cache-miss, and output pricing.
Z.AI	GLM Pricing	GLM-5.2 and GLM-5.1 list input, cached input, output, tool, vision, image, video, audio, and agent pricing categories.
QwenCloud	Pricing, Qwen3.7 Plus, and Qwen3.7 Flash	QwenCloud documents context-tier billing, Batch API behavior, context caching, thinking-token billing, tool fees, 1M context routes, TPM, and RPM.
Baidu Qianfan	Model Billing	ERNIE routes should be treated as provider-specific billing and availability policy, then normalized behind the gateway.
AIWave	Docs, Models, and Pricing	A unified API should expose model metadata and route decisions without forcing application teams to learn every upstream billing shape.

Do not turn this table into a promise that never changes. Store it as a dated policy input and make the controller report which snapshot it used.

What the Controller Protects

A basic request router asks "which model should handle this prompt?" An admission controller asks a different set of questions first:

Check	Example decision
Context tier	A 990K-token Qwen request is allowed only on routes that explicitly support 1M context.
Cache key	A repeated long-context coding request must include a stable cache key or use a shorter route.
Output cap	A high-output Kimi K3 request needs an approved completion-token budget.
Concurrency	A DeepSeek V4 Pro route may queue when the account-level concurrency pool is near its limit.
Tool use	A GLM or Qwen tool call route must include a tool budget and a retry limit.
Region	A GDPR-sensitive workload should use an approved gateway region and logging policy.
Fallback	A fallback must not silently move a tenant into a different billing class.

The goal is not to block useful work. The goal is to stop requests whose production cost, latency, or compliance profile is unknown.

Step 1: Define an Admission Request

Normalize the caller's request into a small internal object. This makes policy evaluation independent of whether the upstream provider uses OpenAI Chat Completions, Responses API, Anthropic format, or a custom payload.

from dataclasses import dataclass, field


@dataclass(frozen=True)
class AdmissionRequest:
    tenant_id_hash: str
    region: str
    workload: str
    requested_model: str
    prompt_tokens: int
    max_output_tokens: int
    has_cache_key: bool
    cache_expected: bool
    uses_images: bool = False
    uses_video: bool = False
    uses_web_search: bool = False
    uses_code_interpreter: bool = False
    requested_fallbacks: list[str] = field(default_factory=list)

Use a hash or internal account identifier for the tenant. Do not put email addresses, names, API keys, or payment identifiers in policy logs.

Step 2: Keep Provider Rules as Data

The controller should not hardcode provider quirks inside a long if chain. Keep rules in dated data so pricing, context, and capacity changes are reviewable.

{
  "snapshot_date": "2026-08-17",
  "routes": {
    "deepseek-v4-pro": {
      "provider": "deepseek",
      "max_context_tokens": 1000000,
      "max_output_tokens": 384000,
      "requires_cache_key_above_tokens": 200000,
      "account_concurrency_limit": 500,
      "supports_tools": true,
      "supports_images": false,
      "supports_video": false
    },
    "qwen3.7-flash": {
      "provider": "qwencloud",
      "max_context_tokens": 1000000,
      "max_output_tokens": 131000,
      "requires_cache_key_above_tokens": 128000,
      "tpm_limit": 5000000,
      "rpm_limit": 15000,
      "supports_tools": true,
      "supports_images": true,
      "supports_video": true
    },
    "glm-5.1": {
      "provider": "zai",
      "max_context_tokens": 128000,
      "max_output_tokens": 32000,
      "requires_cache_key_above_tokens": 64000,
      "supports_tools": true,
      "supports_images": false,
      "supports_video": false
    }
  }
}

These are policy fields, not a full provider catalog. Add only the fields that affect dispatch. Keep the exact pricing rows in a separate snapshot if finance needs full reconstruction.

Step 3: Return Decisions, Not Boolean Flags

The controller should explain every outcome. A plain true or false forces the caller to guess what happened.

from dataclasses import dataclass


@dataclass(frozen=True)
class AdmissionDecision:
    action: str
    route: str | None
    reason: str
    snapshot_date: str
    estimated_prompt_tokens: int
    max_output_tokens: int
    warnings: list[str]


def deny(req: AdmissionRequest, snapshot_date: str, reason: str) -> AdmissionDecision:
    return AdmissionDecision(
        action="deny",
        route=None,
        reason=reason,
        snapshot_date=snapshot_date,
        estimated_prompt_tokens=req.prompt_tokens,
        max_output_tokens=req.max_output_tokens,
        warnings=[],
    )

Use stable actions such as:

Action	Meaning
allow	Dispatch immediately on the selected route.
reshape	Send after reducing output cap, disabling a tool, or requiring cache.
queue	Wait because the route is capacity constrained.
fallback	Use an approved alternate route with matching policy.
deny	Reject with a developer-readable reason.

This matters for SDK ergonomics. A developer can handle queue differently from deny, and reshape differently from fallback.

Step 4: Evaluate Policy Before Dispatch

Here is a compact evaluator. It is intentionally strict about long context and tool use because those are common sources of hidden production variance.

def admit(req: AdmissionRequest, policy: dict, live_capacity: dict) -> AdmissionDecision:
    snapshot_date = policy["snapshot_date"]
    routes = policy["routes"]
    route = routes.get(req.requested_model)
    if route is None:
        return deny(req, snapshot_date, "requested model is not in the approved route catalog")

    if req.prompt_tokens > route["max_context_tokens"]:
        return deny(req, snapshot_date, "prompt exceeds route context window")

    if req.max_output_tokens > route["max_output_tokens"]:
        return AdmissionDecision(
            action="reshape",
            route=req.requested_model,
            reason="requested output cap exceeds route policy",
            snapshot_date=snapshot_date,
            estimated_prompt_tokens=req.prompt_tokens,
            max_output_tokens=route["max_output_tokens"],
            warnings=["max_output_tokens was reduced by admission policy"],
        )

    cache_threshold = route.get("requires_cache_key_above_tokens", 0)
    if req.prompt_tokens >= cache_threshold and not req.has_cache_key:
        return deny(req, snapshot_date, "long-context request requires a stable cache key")

    if req.uses_images and not route.get("supports_images", False):
        return deny(req, snapshot_date, "image input is not approved for this route")

    if req.uses_video and not route.get("supports_video", False):
        return deny(req, snapshot_date, "video input is not approved for this route")

    if (req.uses_web_search or req.uses_code_interpreter) and not route.get("supports_tools", False):
        return deny(req, snapshot_date, "tool use is not approved for this route")

    provider = route["provider"]
    active = live_capacity.get(provider, {}).get("active_requests", 0)
    limit = route.get("account_concurrency_limit")
    if limit is not None and active >= int(limit * 0.9):
        return AdmissionDecision(
            action="queue",
            route=req.requested_model,
            reason="provider concurrency pool is near the admission threshold",
            snapshot_date=snapshot_date,
            estimated_prompt_tokens=req.prompt_tokens,
            max_output_tokens=req.max_output_tokens,
            warnings=[],
        )

    return AdmissionDecision(
        action="allow",
        route=req.requested_model,
        reason="request satisfies route admission policy",
        snapshot_date=snapshot_date,
        estimated_prompt_tokens=req.prompt_tokens,
        max_output_tokens=req.max_output_tokens,
        warnings=[],
    )

The 90 percent threshold is an example, not a universal constant. In a real gateway, set it by route class. Interactive coding agents need a different queue policy than offline evaluation jobs.

Step 5: Make Fallbacks Explicit

Fallback is where many gateways lose control. A fallback that changes context window, modality, tool pricing, or output cap should be treated as a new admission request.

def choose_fallback(req: AdmissionRequest, policy: dict, live_capacity: dict) -> AdmissionDecision:
    first = admit(req, policy, live_capacity)
    if first.action in {"allow", "reshape", "queue"}:
        return first

    for fallback_model in req.requested_fallbacks:
        candidate = AdmissionRequest(
            tenant_id_hash=req.tenant_id_hash,
            region=req.region,
            workload=req.workload,
            requested_model=fallback_model,
            prompt_tokens=req.prompt_tokens,
            max_output_tokens=req.max_output_tokens,
            has_cache_key=req.has_cache_key,
            cache_expected=req.cache_expected,
            uses_images=req.uses_images,
            uses_video=req.uses_video,
            uses_web_search=req.uses_web_search,
            uses_code_interpreter=req.uses_code_interpreter,
            requested_fallbacks=[],
        )
        decision = admit(candidate, policy, live_capacity)
        if decision.action in {"allow", "reshape", "queue"}:
            return AdmissionDecision(
                action="fallback",
                route=decision.route,
                reason=f"primary denied: {first.reason}; fallback accepted",
                snapshot_date=decision.snapshot_date,
                estimated_prompt_tokens=decision.estimated_prompt_tokens,
                max_output_tokens=decision.max_output_tokens,
                warnings=decision.warnings,
            )

    return first

This is especially important when moving between providers. A DeepSeek V4 route may expose a 1M context window and account-level concurrency signals. A Qwen route may expose context tiers, TPM, RPM, thinking tokens, multimodal input, and tool charges. A GLM route may add separate categories for text, vision, tools, image generation, video, audio, and agent products. Treating all of them as identical text completion routes is an operations bug.

Step 6: Surface Admission Metadata to Callers

For OpenAI-compatible responses, keep the standard response shape intact and add a namespaced object.

{
  "id": "chatcmpl_placeholder",
  "object": "chat.completion",
  "model": "qwen3.7-flash",
  "usage": {
    "prompt_tokens": 240000,
    "completion_tokens": 1800,
    "total_tokens": 241800
  },
  "aiwave": {
    "admission": {
      "action": "allow",
      "snapshot_date": "2026-08-17",
      "route": "qwen3.7-flash",
      "reason": "request satisfies route admission policy",
      "cache_required": true,
      "cache_key_present": true,
      "tool_budget_applied": false
    }
  }
}

When the controller rejects a request, return a clear application error:

{
  "error": {
    "type": "admission_denied",
    "message": "Long-context request requires a stable cache key.",
    "param": "cache_key",
    "code": "cache_key_required"
  },
  "aiwave": {
    "snapshot_date": "2026-08-17",
    "requested_model": "kimi-k3",
    "estimated_prompt_tokens": 720000,
    "max_output_tokens": 64000
  }
}

That error is much more useful than a later surprise in the invoice, a provider-side 429, or a timeout that hides the true cause.

Step 7: Observe the Controller Itself

Once deployed, watch admission outcomes as a first-class product metric:

Metric	Why it matters
Admission denials by reason	Shows whether docs, SDK defaults, or customer prompts need correction.
Reshaped output tokens by route	Reveals where completion budgets are too loose.
Queued requests by provider	Warns before 429s become user-visible failures.
Fallbacks by primary route	Detects hidden dependence on secondary providers.
Long-context requests without cache keys	Finds workflows that need SDK-level cache support.
Tool-enabled retries	Stops retry loops from compounding tool charges.

Review these metrics separately for interactive, batch, support, coding, and research workloads. A support chatbot should not share the same output and retry policy as a repository-scale coding agent.

Practical Release Checklist

Before enabling a new Chinese model route in production, ask:

Check	Pass condition
Pricing source	Official provider pages were opened and dated.
Context window	The route has a tested max input and max output policy.
Cache policy	Long-context requests require a stable cache key.
Tool policy	Tool-enabled routes have explicit per-request limits.
Capacity policy	Concurrency, TPM, or RPM limits are represented in data.
Fallback policy	Fallbacks are separately admitted, not silently substituted.
Response metadata	SDK users can see the snapshot and admission reason.
Log hygiene	No personal data, secrets, or raw prompts are written to admission logs.

This pattern is not limited to one provider. DeepSeek's concurrency and peak windows, Kimi's long-context cache economics, GLM's broad modality and tool catalog, QwenCloud's context tiers and tool fees, and ERNIE's provider-specific billing all point to the same engineering conclusion: request compatibility is not the same as production compatibility.

Final Takeaway

OpenAI-compatible APIs are valuable because they reduce integration work. They do not remove the need for policy.

If your gateway accepts every request that matches the SDK method signature, it is letting the provider market define your runtime behavior. An admission controller gives that control back to the platform team. It turns context length, cache readiness, output caps, tool usage, concurrency, and fallback behavior into explicit decisions that developers can understand before a request leaves the system.

For teams evaluating Chinese AI models through a unified API, that is the difference between a demo route and a production route.