Build an Error Taxonomy for Multi-Model AI API Gateways

Most AI gateway failures are not mysterious. They become expensive because every provider names them differently, every SDK retries them differently, and every team debates them after the incident instead of classifying them before dispatch.

A multi-model gateway that routes across DeepSeek, Qwen, GLM, Kimi, and other OpenAI-compatible surfaces needs more than try again later. It needs a shared error taxonomy that tells the SDK whether to retry, reroute, cap output, ask for user action, or stop immediately. It also needs to attach dated pricing metadata, because a retry policy is partly a cost policy.

This article shows how to build that taxonomy as a small production primitive. The goal is not to hide provider behavior. The goal is to normalize it enough that engineers, support, and finance can explain what happened without reading raw provider responses one by one.

All source and pricing checks below were refreshed on August 22, 2026. AIWave's public pricing pages currently list DeepSeek V4 Flash at $0.638 input, $1.914 output, and $0.0203 cache-hit input per 1M tokens; DeepSeek V4 Pro at $1.914 input, $5.742 output, and $0.0638 cache-hit input per 1M tokens. AIWave keeps those all-day rows separate from official DeepSeek peak and off-peak rows on its predictable-pricing page, so the claim should be operational predictability and unified routing, not a universal price-floor claim. QwenCloud's pricing docs describe pay-as-you-go billing, context caching, Batch API behavior, tool fees, and failed-call billing behavior. Z.AI lists GLM rows with separate cached-input pricing, and Kimi K3 public material lists long-context pricing with cache-hit and cache-miss input rows.

Why Error Names Are Not Enough

If your gateway only stores HTTP status codes, you will lose context fast. A 429 can mean account-level concurrency, tenant-level throttling, provider capacity, or a bursty internal worker queue. A 400 can mean a malformed message, an unsupported tool schema, a context overflow, or a model that does not support a requested mode. A timeout can mean provider latency, a broken stream, a user-aborted request, or a local network path.

Those cases should not have the same behavior. Some deserve a fast retry. Some should reroute to another model family. Some should stop because repeating the call only adds latency and possibly cost. Some should become user-facing errors because the request needs to be changed.

A taxonomy gives every failure a stable internal meaning:

Provider symptom	Gateway class	Default action
Authentication failure	auth_config	Stop and alert operators.
Unsupported model or mode	capability_mismatch	Stop or reroute only if policy allows.
Context window exceeded	context_overflow	Summarize, trim, or ask caller to reduce context.
Rate limit or concurrency limit	capacity_limit	Retry with jitter or reroute by policy.
Provider 5xx	provider_unavailable	Retry within budget, then reroute or fail closed.
Stream ended early	stream_interrupted	Retry only if idempotent and output is discardable.
Tool schema rejected	tool_schema_error	Stop and return developer-facing diagnostics.
Billing or quota denial	commercial_block	Stop and notify account owner.

The important part is not the exact names. The important part is that they are few, stable, and actionable.

The Three Policies Behind Every Retry

Retries look like reliability logic, but they are also product and finance logic. A retry can improve success rate. It can also create duplicate work, longer waits, and avoidable token spend. A useful taxonomy therefore separates three policies.

First, classify the error. This answers what happened in gateway terms. Second, decide whether the request is retryable. This depends on idempotency, task type, stream state, and provider signal. Third, decide whether the retry budget still allows another call. That budget should use dated rate-card rows, not guesses.

For example, a non-streaming JSON extraction call may be safe to retry once after a provider 5xx. A streaming coding assistant that already emitted partial instructions may not be safe to replay unless the client can discard the partial response. A tool-call planning request may be safe to reroute from a Flash model to a Pro model only if the policy says quality matters more than cost for that task class.

The retry decision should return structured metadata:

{
  "error_class": "capacity_limit",
  "retryable": true,
  "reroute_allowed": true,
  "retry_after_ms": 1800,
  "max_attempts": 2,
  "cost_budget_usd": 0.02,
  "policy_version": "error-taxonomy-2026-08-22",
  "rate_card_source_date": "2026-08-22"
}

That metadata can be written beside the final usage record. Later, when a customer asks why a request took 11 seconds or why a fallback model was used, you have the answer.

Rate Cards Belong in Error Handling

Do not let the retry layer treat every model as equal. A retry against a short-output Flash route and a retry against a long-context route are different budget events. This becomes especially important when providers expose separate fields for fresh input, cached input, output, tool calls, context tiers, or failed-call behavior.

At minimum, store these fields for the route that failed:

Field	Purpose
model	The selected provider-facing model name.
route_family	Internal family such as DeepSeek, Qwen, GLM, or Kimi.
input_per_m	Fresh input price per 1M tokens for this route.
cached_input_per_m	Cache-hit input price when available.
output_per_m	Output price per 1M tokens.
source_url	The pricing page or internal dated rate-card source.
source_date	The date the row was checked.
failed_call_policy	Whether failed calls can still affect billing or quotas.

The failed_call_policy field is easy to skip and hard to reconstruct later. If a provider documents that some failed calls can affect billing, quota, or tool charges, your retry budget should know that before it fires another request. If a provider does not expose enough detail, store unknown and keep the retry budget conservative.

Python Example: Classify, Budget, Retry

The following Python example shows a compact gateway layer. It classifies provider errors, estimates retry exposure with a dated AIWave DeepSeek row, and returns a decision the caller can log.

import os
import random
import time
from dataclasses import dataclass, asdict
from typing import Literal

from openai import OpenAI

client = OpenAI(
    api_key=os.environ.get("AIWAVE_API_KEY", "YOUR_API_KEY_HERE"),
    base_url="https://aiwave.live/v1",
)

ErrorClass = Literal[
    "auth_config",
    "capability_mismatch",
    "context_overflow",
    "capacity_limit",
    "provider_unavailable",
    "stream_interrupted",
    "tool_schema_error",
    "commercial_block",
    "unknown",
]

@dataclass(frozen=True)
class RateCard:
    model: str
    input_per_m: float
    cached_input_per_m: float
    output_per_m: float
    source_url: str
    source_date: str

@dataclass(frozen=True)
class ErrorDecision:
    error_class: ErrorClass
    retryable: bool
    reroute_allowed: bool
    max_attempts: int
    retry_after_ms: int
    estimated_retry_usd: float
    policy_version: str
    reason: str

AIWAVE_FLASH = RateCard(
    model="deepseek-v4-flash",
    input_per_m=0.638,
    cached_input_per_m=0.0203,
    output_per_m=1.914,
    source_url="https://aiwave.live/pricing",
    source_date="2026-08-22",
)

def classify_error(status_code: int | None, body: str) -> ErrorClass:
    text = body.lower()
    if status_code in {401, 403}:
        return "auth_config"
    if status_code == 402 or "quota" in text or "balance" in text:
        return "commercial_block"
    if status_code == 404 or "model" in text and "not found" in text:
        return "capability_mismatch"
    if "context" in text and ("length" in text or "window" in text):
        return "context_overflow"
    if status_code == 429 or "rate limit" in text or "concurrency" in text:
        return "capacity_limit"
    if status_code and 500 <= status_code < 600:
        return "provider_unavailable"
    if "tool" in text and "schema" in text:
        return "tool_schema_error"
    return "unknown"

def estimate_retry_cost(
    row: RateCard,
    input_tokens: int,
    cached_input_tokens: int,
    output_cap: int,
) -> float:
    fresh_input_tokens = max(input_tokens - cached_input_tokens, 0)
    cost = (
        fresh_input_tokens / 1_000_000 * row.input_per_m
        + cached_input_tokens / 1_000_000 * row.cached_input_per_m
        + output_cap / 1_000_000 * row.output_per_m
    )
    return round(cost, 6)

def decide_error_policy(
    error_class: ErrorClass,
    row: RateCard,
    input_tokens: int,
    cached_input_tokens: int,
    output_cap: int,
    already_streamed: bool,
) -> ErrorDecision:
    estimated_retry_usd = estimate_retry_cost(
        row, input_tokens, cached_input_tokens, output_cap
    )

    if error_class in {"auth_config", "commercial_block", "tool_schema_error"}:
        return ErrorDecision(
            error_class, False, False, 0, 0, 0.0,
            "error-taxonomy-2026-08-22",
            "caller_or_account_action_required",
        )

    if error_class == "context_overflow":
        return ErrorDecision(
            error_class, False, True, 0, 0, 0.0,
            "error-taxonomy-2026-08-22",
            "summarize_or_route_to_approved_long_context_model",
        )

    if already_streamed:
        return ErrorDecision(
            error_class, False, False, 0, 0, estimated_retry_usd,
            "error-taxonomy-2026-08-22",
            "partial_output_already_released",
        )

    if error_class in {"capacity_limit", "provider_unavailable"}:
        return ErrorDecision(
            error_class, estimated_retry_usd <= 0.02, True, 2,
            random.randint(1200, 3200), estimated_retry_usd,
            "error-taxonomy-2026-08-22",
            "bounded_retry_with_jitter",
        )

    return ErrorDecision(
        error_class, False, False, 0, 0, estimated_retry_usd,
        "error-taxonomy-2026-08-22",
        "unclassified_errors_fail_closed",
    )

def chat_with_error_policy(prompt: str):
    input_tokens = max(1, len(prompt) // 4)
    output_cap = 1200
    attempts = 0

    while True:
        attempts += 1
        try:
            return client.chat.completions.create(
                model=AIWAVE_FLASH.model,
                messages=[{"role": "user", "content": prompt}],
                max_tokens=output_cap,
            )
        except Exception as exc:
            status = getattr(exc, "status_code", None)
            body = str(exc)
            error_class = classify_error(status, body)
            decision = decide_error_policy(
                error_class,
                AIWAVE_FLASH,
                input_tokens=input_tokens,
                cached_input_tokens=0,
                output_cap=output_cap,
                already_streamed=False,
            )
            print(asdict(decision))
            if not decision.retryable or attempts >= decision.max_attempts:
                raise
            time.sleep(decision.retry_after_ms / 1000)

This is not a complete gateway. It is the smallest useful loop: classify, estimate, decide, log, then retry only when the policy allows it.

In production, replace rough token estimation with the tokenizer your gateway uses. Record actual usage after the call. Add provider-specific adapters that preserve raw status, raw body, request id, and response headers in private logs while exposing only the normalized class to product code.

Rerouting Is Not a Blind Fallback

Rerouting is often treated as an automatic reliability feature. It should be more deliberate. If a DeepSeek Flash execution step hits capacity, rerouting to another execution-capable model may be fine. If a Pro planning step fails after a long prompt, rerouting to a different family can change quality, latency, privacy posture, and unit economics.

Use explicit reroute rules:

Task class	Reroute allowed?	Notes
Short extraction	Yes	Keep schema strict and output capped.
Coding patch generation	Yes, with replay test coverage	Log model family and policy version.
Legal or compliance review	Review required	Model change may affect evidence handling.
Customer support draft	Yes	Discard partial streamed output before retry.
Tool-call planning	Limited	Tool schema compatibility must be checked first.

The reroute rule should be attached to the original request, not invented after the error. That makes behavior reproducible. It also prevents an incident from becoming a hidden model migration.

What to Show the Customer

Do not expose raw provider errors by default. They may include internal identifiers, confusing provider names, or messages that encourage the wrong user action. Instead, map normalized classes to product-safe responses.

For context_overflow, explain that the request needs less context or a summarization step. For capacity_limit, say the route is temporarily constrained and the request may be retried. For tool_schema_error, return a developer-facing message with the rejected field and the supported schema shape. For commercial_block, ask the account owner to check billing or plan status.

Support staff should have a richer view: normalized class, raw provider status, request id, selected model, fallback model, source date for rate card, retry count, estimated retry cost, actual usage, and policy version. That is enough to answer most tickets without exposing prompt content.

Observability Fields That Matter

Add these fields to every gateway span:

Span field	Example
ai.route.model	deepseek-v4-flash
ai.route.family	deepseek
ai.error.class	capacity_limit
ai.retry.count	1
ai.retry.estimated_usd	0.0042
ai.rate_card.source_date	2026-08-22
ai.policy.version	error-taxonomy-2026-08-22
ai.fallback.used	false
ai.stream.partial_released	false

These fields let you build dashboards that show failure mix by model, retry spend by tenant, capacity incidents by provider family, and policy changes by release. They also let you separate provider instability from client misuse. A spike in tool_schema_error after an SDK release is not the same problem as a spike in provider_unavailable.

Rollout Plan

Start in shadow mode. Classify every error and log the decision you would have made, but do not change retry behavior yet. Run this for a week across Tier 1 and Tier 2 production traffic, especially US, UK, Germany, Japan, Singapore, and similar developer-heavy markets where reliability expectations are high.

Next, enable the taxonomy for one task class. Short extraction or non-streaming support drafts are good candidates because partial output is easier to discard. Keep max attempts low. Add an alert when retry spend crosses a small daily threshold.

Then add reroute rules for selected model families. Require a replay test for every reroute pair. A replay test should include successful calls, context overflow, schema mismatch, provider 5xx, capacity limits, and streaming interruption. Store the policy version beside every replay result.

Finally, make the taxonomy part of release review. Any new model, provider, SDK feature, or pricing row should answer four questions before launch:

Question	Required answer
Which error classes can this route emit?	Provider adapter mapping exists.
Which classes are retryable?	Policy table has task-level rules.
What is the retry cost ceiling?	Dated rate-card row is attached.
Can fallback change behavior?	Replay fixtures cover the fallback pair.

Source Links for Implementation

If you are building this on AIWave, start with the Chat Completions documentation, model catalog, pricing page, and predictable-pricing calculator. The model catalog and pricing pages give the route and rate-card context your error policy needs.

For external source checks, keep current links to the official DeepSeek pricing docs, DeepSeek rate-limit docs, QwenCloud pricing docs, Z.AI pricing docs, and Kimi K3 page. Recheck these pages before changing retry budgets or fallback policies.

Final Checklist

Before you ship an error taxonomy for a multi-model AI gateway, make sure it can do the following:

Requirement	Pass condition
Normalize provider errors	Every raw error maps to a stable class or unknown.
Separate retry from reroute	A retry decision does not automatically change model families.
Track stream state	Partial output changes retry behavior.
Attach dated rate cards	Retry budgets include source date and route price fields.
Preserve raw diagnostics privately	Operators can debug without exposing internals to users.
Version policies	Every decision stores a policy version.
Replay fallback paths	Model-pair behavior is tested before production rollout.

The useful shift is cultural as much as technical. Error handling is not an afterthought at the edge of the SDK. It is a routing surface. Once failures are classified, budgeted, and versioned, the gateway can make boring decisions under pressure: retry when it is safe, reroute when policy allows it, stop when the caller must act, and keep every decision tied to a dated source.