Build a Quota and Rate-Limit Contract for AI API Gateways

When an application team moves agent traffic through an AI API gateway, the happy-path request is usually the easy part. A chat completion that returns 200 proves only that one model route worked once. The production question is different: can the client tell the difference between a missing key, an exhausted balance, a burst limit, an upstream timeout, and a deliberate route block without turning every failure into a support ticket?

That contract matters more for teams using Chinese model routes from the United States, United Kingdom, Germany, Japan, Singapore, Canada, and other Tier 1 or Tier 2 markets. The team may be using an OpenAI-compatible SDK, but the operational surface is wider than a single provider account. There may be several model families, multiple account groups, dated pricing rows, cache-hit fields, retry classes, and regional procurement checks. If the client collapses those details into "API error", the engineering team loses the evidence it needs exactly when a rollout is under pressure.

This post is a practical contract for quota, rate-limit, and route failures. It uses AIWave as the gateway example because AIWave exposes OpenAI-compatible routes, public pricing evidence, docs, status checks, and request-level billing concepts. Disclosure: this article is from AIWave. The implementation pattern is general enough to adapt to any gateway that wants clean SDK behavior instead of mystery failures.

Start with the outcomes, not the status codes

HTTP status codes are useful, but they are not the product contract by themselves. A user action should map to a clear outcome:

Situation	Client outcome	Product action
Missing or invalid key	Stop immediately	Ask the user to check the key source
Valid account with no available quota	Stop before retrying	Link to billing or funding flow
Burst or concurrency limit	Retry with backoff	Preserve request intent and idempotency
Route not enabled for account	Stop and show route policy	Ask for another model or account tier
Upstream timeout	Retry only if safe	Record attempt class and elapsed time
Response validation failure	Do not auto-spend blindly	Save a redacted fixture for review

That table is deliberately written in product language. The SDK can still carry 401, 403, 429, and 5xx, but the UI and runbook need a stable decision field. A good field name is action_class: fix_key, fund_account, retry_later, choose_route, retry_if_idempotent, or inspect_fixture.

The mistake to avoid is retrying quota failures. A retry loop can help a 429 when the request is otherwise valid. It cannot solve an account with insufficient balance. Retrying that request only burns time, makes logs noisy, and hides the real next step from the developer.

Keep 401, quota, and 429 separate

For OpenAI-compatible clients, three failure families deserve separate handling.

First, authentication failures mean the request cannot be trusted. Treat missing, malformed, revoked, or misplaced API keys as a hard stop. Do not echo the key, do not print environment values, and do not save request bodies with secrets attached. The only useful debugging fields are whether the key source was present, which environment variable name was checked, the base URL, and a request ID if the gateway provided one.

Second, quota failures mean the credential was recognized but the account cannot pay for the next call under its current rules. Some systems use 402; others use 403 with a specific machine-readable error code. The client should not assume all 403 responses are quota problems. It should inspect a bounded error code such as insufficient_user_quota before showing a funding path.

Third, rate limits and overload signals mean the request may be valid later. A 429 should carry retry timing when available, but the SDK still needs a cap. The client should stop after a small number of attempts, emit a receipt, and make the queue visible. Long-running agent jobs should not sit in a hidden loop that spends the operator's entire incident budget.

Here is a compact Python classifier:

from dataclasses import dataclass
from typing import Any


@dataclass
class ApiFailure:
    status: int
    code: str | None
    message: str
    request_id: str | None
    retry_after_seconds: int | None = None


def classify_failure(error: ApiFailure) -> str:
    if error.status == 401:
        return "fix_key"

    if error.status == 402:
        return "fund_account"

    if error.status == 403 and error.code == "insufficient_user_quota":
        return "fund_account"

    if error.status == 429:
        return "retry_later"

    if error.status in {408, 500, 502, 503, 504}:
        return "retry_if_idempotent"

    return "inspect_fixture"

Notice what is absent: the classifier never retries every 403, never logs an API key, and never uses a vague exception string as the only contract. That is the difference between an SDK helper and a production boundary.

Add dated pricing evidence to the same receipt

Error handling and pricing look separate until a buyer asks why a run was stopped. Then they become the same conversation.

I fetched AIWave's public pricing JSON on 2026-09-11. The JSON reported checked: 2026-09-10, currency: USD, unit: per_1m_text_tokens, and pricing version 8c7a0c0b30661ccbc13d142cb54d1e4ae445fe774b2c6fa501080db97c7a3e56. Selected rows in that live response included:

Model ID	Provider	Input per 1M	Cache-hit per 1M	Output per 1M	Effective date
deepseek-v4-pro	DeepSeek	1.914 USD	0.0637362 USD	5.742 USD	2026-08-27
deepseek-v4-flash	DeepSeek	0.638 USD	0.0202884 USD	1.914 USD	2026-08-27
glm-5.1	GLM	2.1 USD	0.680001 USD	6.5999997 USD	2026-08-27
kimi-k3	Kimi	4.5 USD	0.9 USD	22.5 USD	2026-08-27
moonshot-v1-128k	Kimi	1.8 USD	not listed	4.5 USD	2026-08-27
qwen3-max	Qwen	1.5621977891181764 USD	not listed	6.248791156472706 USD	2026-08-27

The point is not to paste a price table into every exception. The point is to attach a pricing_snapshot_id, checked_at, and model_price_effective_date to the run receipt. If the account later hits quota, the team can connect the event to the model row and account group that was active when the job started.

AIWave's public pricing JSON also warns that rates are dated base rates and that the effective account group controls the applied multiplier. That is exactly the kind of sentence a production receipt should preserve. A developer evaluating a gateway needs to know whether an estimate came from a current source, an old document, a custom account group, or a manual spreadsheet.

Define the receipt before writing retries

Before implementing retries, define the record that every attempt must leave behind. A useful receipt is small enough to store for every request and explicit enough to debug.

from dataclasses import dataclass, field
from datetime import datetime, timezone


@dataclass
class AttemptReceipt:
    workflow: str
    model: str
    base_url: str
    request_id: str | None
    started_at: str = field(default_factory=lambda: datetime.now(timezone.utc).isoformat())
    action_class: str | None = None
    status: int | None = None
    error_code: str | None = None
    pricing_snapshot_id: str | None = None
    pricing_checked_date: str | None = None
    estimated_input_tokens: int | None = None
    estimated_output_tokens: int | None = None
    attempt_number: int = 1

This object is not a prompt log. It does not need the user's text, tool outputs, customer data, or reusable credentials. It carries enough metadata to answer operational questions:

• Which model route did the workflow ask for?
• Which base URL did the SDK call?
• Which bounded failure class did the gateway return?
• Which pricing snapshot was used for preflight?
• How many attempts happened before the job stopped?
• Did the SDK choose retry, funding, route change, or manual inspection?

That last question is the one most teams miss. A gateway integration should not only report "what happened"; it should report "what the client decided to do next."

Put budget checks before network calls

A quota contract is stronger when the client can reject risky work before calling the gateway. For agent queues, add a preflight stage:

1. Estimate input tokens, output cap, selected tools, and retrieval context.
2. Load the pricing snapshot approved for the run.
3. Calculate a maximum estimated cost for the planned route.
4. Compare it with a workflow budget.
5. Either send the request, ask for narrower context, or choose a different model route.

The estimate will not always match the final bill. Streaming, cache status, retries, and provider-side tokenization can shift the final number. That is acceptable if the receipt distinguishes estimate from observed usage. The failure mode is pretending the estimate is final truth.

Here is a minimal preflight:

@dataclass
class PriceRow:
    input_usd_per_1m: float
    output_usd_per_1m: float


def estimate_usd(input_tokens: int, output_cap: int, price: PriceRow) -> float:
    input_cost = input_tokens * price.input_usd_per_1m / 1_000_000
    output_cost = output_cap * price.output_usd_per_1m / 1_000_000
    return round(input_cost + output_cost, 6)


def should_send(estimated_usd: float, workflow_limit_usd: float) -> bool:
    return estimated_usd <= workflow_limit_usd

The exact budget number is a product decision. Some teams use a per-request cap, some use a queue-level cap, and some use a daily project envelope. What matters is that the SDK records the decision before the network call. If the request never leaves the process, the receipt should say action_class: inspect_fixture or action_class: choose_route, not pretend that the gateway rejected it.

Make retries boring

Retries should be conservative, observable, and dull. That is a compliment. A retry policy that feels clever during a demo can become expensive during an incident.

For 429, use bounded exponential backoff with jitter. Respect Retry-After when present, but still enforce a maximum wait and maximum attempts. For 408, 502, 503, and 504, retry only idempotent jobs or jobs that can safely deduplicate downstream writes. For streaming responses, retrying after partial output is a product decision, not a generic transport rule.

The receipt should record each attempt separately:

attempt 1: 429 retry_later retry_after=3
attempt 2: 429 retry_later retry_after=8
attempt 3: stopped retry_budget_exhausted

That trail is better than a single final exception because it tells the operator the queue hit a policy limit. It also helps buyers distinguish gateway limits from application behavior. If the client kept retrying past its own cap, the gateway is not the only system to inspect.

Do not leak keys while trying to help

Developer experience often fails at the support boundary. A user pastes an error into a ticket, a log bundle includes environment variables, or a screenshot shows too much. The contract should prevent that by design.

Recommended fields:

• key_source_present: true or false
• key_source_name: AIWAVE_API_KEY
• base_url: the configured API base URL
• request_id: gateway-provided ID when available
• model: requested model ID
• action_class: bounded client decision

Fields to avoid:

• Raw API key
• Prompt text
• Response body from user data workflows
• Full customer identifiers
• Full account balance when a coarse billing action is enough

The SDK can still offer useful advice: check that AIWAVE_API_KEY is set, confirm the base URL, verify that the selected model is enabled for the account, or open the pricing page. It does not need to reveal secrets to do that.

A rollout checklist for Tier 1 and Tier 2 teams

Before moving serious traffic to a gateway route, run this checklist:

• Confirm the OpenAI-compatible base URL in code, CI, and deployed environment.
• Run one authenticated model-list or small chat-completion probe.
• Fetch the public pricing source and record its checked date.
• Save the pricing snapshot ID beside the deployment version.
• Test a missing-key fixture and make sure it maps to fix_key.
• Test an exhausted-balance or quota fixture and make sure it maps to fund_account.
• Test a synthetic 429 and make sure retries stop at the configured cap.
• Test a route-disabled fixture and make sure the UI does not suggest funding if funding is not the issue.
• Redact keys, prompts, and account identifiers from receipts.
• Add one dashboard or log query that groups failures by action_class.

That last query is the quiet superpower. After a week, the team can see whether onboarding is blocked by keys, quota, burst limits, route policy, or upstream instability. Without that grouping, every incident looks equally urgent and equally vague.

Where AIWave fits

AIWave's public docs expose the basic integration path through OpenAI-compatible endpoints. The public pricing JSON gives dated USD rows for supported model routes. The status and docs pages provide external evidence that a buyer can inspect before funding or scaling a workload.

The stronger integration pattern is to combine those surfaces:

• Use the quickstart to prove the request shape.
• Use pricing JSON to pin dated route economics.
• Use error docs and SDK code to classify failures.
• Use receipts to connect route, estimate, result, and next action.
• Use status checks as one signal, not as a substitute for your own request logs.

That pattern is not glamorous, but it makes AI API adoption easier to defend. It shows the buyer that the gateway is not asking for blind trust. It is giving the developer a route, a price source, a failure contract, and a receipt trail.

Final test: can support answer in one message?

A useful quota and rate-limit contract passes a simple support test. When a developer asks "why did my agent stop?", support should be able to answer in one message:

The key was valid. The request stopped before retrying because the account lacked available quota for this route. The run used pricing snapshot 8c7a0c0...e56, checked 2026-09-10. No prompt text or API key was stored in the receipt. Add balance or choose a smaller route, then rerun the job.

That answer is specific without exposing secrets. It separates quota from rate limit. It names the pricing evidence. It gives the next step. That is the standard an AI API gateway should aim for before asking production teams to depend on it.

Sources checked

• AIWave pricing JSON: https://aiwave.live/api/v1/pricing
• AIWave pricing page: https://aiwave.live/pricing
• AIWave quickstart docs: https://aiwave.live/docs/quickstart
• AIWave error docs: https://aiwave.live/docs/errors
• AIWave status page: https://aiwave.live/status