Build a Route Dry-Run Simulator for AI API Releases

Most AI API release reviews happen after the route already answered traffic.

That is backwards.

By the time a route is live, the team may already be dealing with user-facing errors, unexpected output length, missing receipt fields, or a pricing source that changed since the original planning note. A Tier 1 or Tier 2 engineering team should be able to test the route contract before any private workload touches the gateway.

A route dry-run simulator is a small harness for that job. It does not call the model. It does not store prompts. It does not need a reusable production key. It checks whether a proposed AI API route can be explained with the evidence the team expects to rely on later.

The simulator answers five questions:

1. Does the requested model ID resolve to a current route?
2. Which dated pricing row will reviewers use?
3. Which account or key group multiplier applies?
4. What happens to the budget if retries or output tokens move?
5. Which redacted receipt fields must exist after production traffic runs?

I will use AIWave as the concrete example because its public pricing sources can be checked without a private account. During this run on September 22, 2026, https://aiwave.live/api/v1/pricing returned 56 model rows, currency USD, unit per_1m_text_tokens, checked=2026-09-10, updated_at=2026-09-18, and pricing version 83f77abde81ee3a096a672ed959ccc096f5d37a45c177ae8e03229456b5415a5. The live route table at https://aiwave.live/api/pricing returned success=true, 73 route rows, pricing version a42d372ccf0b5dd13ecf71203521f9d2, and group ratios default=1 and vip=0.9.

Those values are not marketing promises. They are dated inputs for a simulator.

Why dry-run before the first private request?

AI API releases often mix three different concerns:

Concern	Typical owner	Failure mode
Route selection	Platform engineering	Alias points to a different model than the product expected
Pricing evidence	Finance or procurement	Workbook uses a row that is no longer the checked source
Runtime receipts	Application engineering	Production logs cannot explain cache, retry, group, or output behavior

The dry-run simulator does not replace any of those owners. It gives them one shared preflight artifact.

That artifact is especially useful when the release uses an OpenAI-compatible interface. The client call may look stable, but the route behind it can still vary by provider, model alias, context mode, batch mode, cache behavior, tool fee, or account group.

The interface is compatible. The evidence still needs a contract.

A dry-run input object

Keep the simulator input synthetic. It should describe a workload shape, not a real user prompt.

{
  "dry_run_id": "support-triage-route-2026-09-22",
  "checked_at": "2026-09-22T13:20:00Z",
  "client_shape": "openai_chat_completions",
  "requested_model": "deepseek-v4-flash",
  "account_group": "default",
  "traffic_shape": {
    "requests": 1000,
    "input_tokens_per_request": 120000,
    "expected_cached_input_tokens_per_request": 0,
    "max_output_tokens_per_request": 900,
    "max_retry_attempts_per_request": 1
  },
  "receipt_contract": [
    "request_id",
    "model_id",
    "prompt_tokens",
    "completion_tokens",
    "cached_input_tokens",
    "retry_attempts",
    "group",
    "charged_amount"
  ]
}

There is no API key in that file. There is no request body. There is no customer identifier. That makes the object safe to attach to a release review while still being concrete enough for engineers to test.

Load dated pricing and live routes

The simulator needs two different sources:

• a dated pricing snapshot for review math
• a live route table for current route resolution

from __future__ import annotations

import json
import urllib.request


def load_json(url: str) -> dict:
    with urllib.request.urlopen(url, timeout=20) as response:
        return json.loads(response.read().decode("utf-8"))


pricing = load_json("https://aiwave.live/api/v1/pricing")
routes = load_json("https://aiwave.live/api/pricing")

assert pricing["currency"] == "USD"
assert pricing["unit"] == "per_1m_text_tokens"
assert routes["success"] is True

Do not hide these source versions. Print or store the checked, updated_at, and pricing_version values in the simulator result. A dry run without source versions is just another estimate that will be hard to audit later.

Resolve the model deliberately

A model ID can fail in two different ways:

• it is absent from the pricing snapshot
• it is absent from the live route table

Those are not the same failure. The first means the team cannot price the planned route from the dated source. The second means the route may not be callable in the current gateway table.

def find_pricing_row(pricing: dict, model_id: str) -> dict | None:
    return next((row for row in pricing["models"] if row["id"] == model_id), None)


def live_route_names(routes: dict) -> set[str]:
    names: set[str] = set()
    for row in routes.get("data", []):
        for key in ("id", "model", "model_name"):
            value = row.get(key)
            if isinstance(value, str):
                names.add(value)
    return names


requested_model = "deepseek-v4-flash"
pricing_row = find_pricing_row(pricing, requested_model)
route_exists = requested_model in live_route_names(routes)

if pricing_row is None:
    raise SystemExit("missing_pricing_row")
if not route_exists:
    raise SystemExit("missing_live_route")

In production, you may allow aliases, but they should resolve to a named canonical model before review. Otherwise a later incident will spend time asking which route actually shipped.

Simulate cost without pretending it is billing

The simulator should call its output an estimate or review cost, not an invoice.

def usd_per_million(tokens: int, rate: float) -> float:
    return tokens / 1_000_000 * rate


def estimate_run_cost(shape: dict, row: dict, group_ratio: float) -> float:
    requests = shape["requests"]
    input_tokens = requests * shape["input_tokens_per_request"]
    cached_tokens = requests * shape.get("expected_cached_input_tokens_per_request", 0)
    output_tokens = requests * shape["max_output_tokens_per_request"]

    fresh_tokens = max(input_tokens - cached_tokens, 0)
    input_rate = row["input_usd_per_1m_tokens"]
    cached_rate = row.get("cache_hit_input_usd_per_1m_tokens") or input_rate
    output_rate = row["output_usd_per_1m_tokens"]

    subtotal = (
        usd_per_million(fresh_tokens, input_rate)
        + usd_per_million(cached_tokens, cached_rate)
        + usd_per_million(output_tokens, output_rate)
    )
    return round(subtotal * group_ratio, 6)

The group ratio should come from the live route source or another controlled configuration, not from a comment in the release ticket.

group = "default"
group_ratio = routes["group_ratio"][group]
review_cost = estimate_run_cost(dry_run["traffic_shape"], pricing_row, group_ratio)

That number is useful because it is reproducible. It is still not the final bill. Real usage can change through output length, retries, cache status, tool calls, failed requests, upstream changes, and account configuration.

Add stress bands

A single estimate is too neat. Add small stress bands that represent common production movement.

def clone_shape(shape: dict, **updates: int) -> dict:
    next_shape = dict(shape)
    next_shape.update(updates)
    return next_shape


base = dry_run["traffic_shape"]
scenarios = {
    "planned": base,
    "long_outputs": clone_shape(base, max_output_tokens_per_request=1800),
    "one_retry": clone_shape(
        base,
        requests=base["requests"] * (1 + base["max_retry_attempts_per_request"]),
    ),
    "cache_observed": clone_shape(
        base,
        expected_cached_input_tokens_per_request=60000,
    ),
}

for name, scenario in scenarios.items():
    print(name, estimate_run_cost(scenario, pricing_row, group_ratio))

The point is not to scare people with a dramatic range. The point is to expose which assumption matters.

If long_outputs moves the review cost more than any other scenario, the release should focus on response length controls. If one_retry dominates, the team should inspect timeout and idempotency behavior. If cache_observed changes the result but the route cannot produce cache receipt fields, the release should not depend on cache savings yet.

Fail closed on missing receipt fields

The dry run should also check that the planned receipt contract is strong enough for later analysis.

REQUIRED_FIELDS = {
    "request_id",
    "model_id",
    "prompt_tokens",
    "completion_tokens",
    "retry_attempts",
    "group",
    "charged_amount",
}


def validate_receipt_contract(fields: list[str]) -> list[str]:
    present = set(fields)
    return sorted(REQUIRED_FIELDS - present)


missing = validate_receipt_contract(dry_run["receipt_contract"])
if missing:
    raise SystemExit({"missing_receipt_fields": missing})

Notice that cached_input_tokens is not in the minimum set above. Some routes may not support cache evidence. That is fine as long as the simulator records the unknown and the forecast does not rely on cache behavior.

For a cache-dependent release, promote that field to required:

if dry_run["traffic_shape"]["expected_cached_input_tokens_per_request"] > 0:
    REQUIRED_FIELDS.add("cached_input_tokens")

The simulator should preserve unknowns instead of filling them with guesses. Unknown is a valid release result. It is much better than a confident spreadsheet that cannot be reconciled later.

Produce a review result

The result should be small enough for a pull request, release note, or procurement answer.

{
  "dry_run_id": "support-triage-route-2026-09-22",
  "verdict": "review_required",
  "requested_model": "deepseek-v4-flash",
  "pricing_source": {
    "checked": "2026-09-10",
    "updated_at": "2026-09-18",
    "pricing_version": "83f77abde81ee3a096a672ed959ccc096f5d37a45c177ae8e03229456b5415a5"
  },
  "route_source": {
    "pricing_version": "a42d372ccf0b5dd13ecf71203521f9d2",
    "group_ratio": {"default": 1, "vip": 0.9}
  },
  "checks": {
    "pricing_row": "present",
    "live_route": "present",
    "receipt_contract": "present",
    "cache_evidence": "not_assumed"
  },
  "review_notes": [
    "output cap is the largest stress driver",
    "final billing depends on actual receipts and selected group"
  ]
}

That object gives each owner a clear next step. Platform engineering can review route resolution. Product engineering can review output caps. Finance can inspect the source versions. Security can check that private content stayed out of the artifact.

Keep public language narrower than internal proof

A dry-run simulator is good internal evidence. It is not a license to make broad claims.

Safe public copy says:

The route was dry-run against a dated pricing snapshot and live route table before release.

Risky public copy says:

The route will always cost less in production.

The first statement is about process and evidence. The second predicts a workload outcome without knowing the future request mix, cache behavior, retries, output length, group selection, or source changes.

For AI API gateways, the practical standard should be boring and strict:

• Route IDs resolve before the release.
• Pricing versions are recorded before the release.
• Group rules are explicit before the release.
• Stress bands are visible before the release.
• Receipt fields are agreed before the release.
• Private prompts and reusable credentials stay out of the artifact.

If the team can do that, the first production request is no longer the first time the route's cost and evidence contract has been tested.