Build Route Decision Records for AI API Gateway Changes

AI API route changes often look smaller than they are.

A pull request may change one model ID. A configuration update may switch a gateway row. A product owner may ask for longer output. A finance reviewer may approve a budget from last week's pricing table. Each change is understandable on its own, but the combined result can alter quality, latency, spend, privacy posture, and rollback behavior.

That is why route changes need a decision record.

An AI API route decision record is a compact release artifact. It says which route is being changed, which dated sources were checked, what behavior is expected, which unknowns remain, and how the team will reverse the change if production evidence disagrees.

This is not a compliance ceremony. It is a practical way for Tier 1 and Tier 2 engineering teams to avoid shipping a route change whose evidence is scattered across chats, spreadsheets, dashboards, and tribal memory.

I will use AIWave as the concrete example because its public pricing and route sources can be checked without a private account. During this run on September 25, 2026, https://aiwave.live/api/v1/pricing returned 56 dated USD model rows, currency USD, unit per_1m_text_tokens, checked=2026-09-10, updated_at=2026-09-18, and pricing version 83f77abde81ee3a096a672ed959ccc096f5d37a45c177ae8e03229456b5415a5. The live route table at https://aiwave.live/api/pricing returned success=true, 73 route rows, pricing version a42d372ccf0b5dd13ecf71203521f9d2, auto_groups=["default"], and group ratios default=1 and vip=0.9.

Those values are dated inputs for a decision record. They are not timeless promises.

Why route changes need a record

OpenAI-compatible APIs make client code portable. They do not make route decisions self-explanatory.

The same SDK call can hide several changes:

Change	What can move
Model ID	Quality, context behavior, tool support, output length
Provider route	Availability, latency, failure mode, receipt fields
Pricing source	Forecast basis, effective date, account group multiplier
Retry policy	User-visible delay, duplicate work, budget variance
Output cap	Product quality, completion tokens, timeout risk
Data class	What crosses the API boundary
Rollback path	Whether the previous route can be restored cleanly

A route decision record does not solve those issues automatically. It puts them in one place so the team can review the actual decision rather than reconstruct it after an incident.

A small decision record schema

Keep the schema boring. Boring survives release pressure.

{
  "decision_id": "rdr-support-summary-2026-09-25",
  "status": "proposed",
  "owner": "platform-ai",
  "checked_at": "2026-09-25T13:20:00Z",
  "client_shape": "openai_chat_completions",
  "change": {
    "from_model": "deepseek-v4-flash",
    "to_model": "glm-5",
    "reason": "evaluate longer reasoning for support summaries"
  },
  "sources": {
    "dated_pricing": {
      "url": "https://aiwave.live/api/v1/pricing",
      "checked": "2026-09-10",
      "updated_at": "2026-09-18",
      "pricing_version": "83f77abde81ee3a096a672ed959ccc096f5d37a45c177ae8e03229456b5415a5"
    },
    "live_routes": {
      "url": "https://aiwave.live/api/pricing",
      "pricing_version": "a42d372ccf0b5dd13ecf71203521f9d2",
      "route_rows": 73,
      "group_ratio": {
        "default": 1,
        "vip": 0.9
      }
    }
  },
  "release_limits": {
    "max_output_tokens": 600,
    "max_retry_attempts": 1,
    "timeout_ms": 45000
  },
  "receipt_contract": [
    "request_id",
    "model_id",
    "prompt_tokens",
    "completion_tokens",
    "retry_attempts",
    "group",
    "charged_amount"
  ],
  "rollback": {
    "previous_model": "deepseek-v4-flash",
    "trigger": "quality, receipt, or budget gate fails"
  }
}

There is no prompt text in that object. There is no reusable API key. There is no customer identifier. The record should be safe to attach to a release ticket while still being precise enough to audit.

Separate the decision from the estimate

A route decision record can include budget math, but it should not pretend to be the final bill.

from __future__ import annotations

from dataclasses import dataclass


@dataclass(frozen=True)
class RateRow:
    input_usd_per_1m: float
    cache_hit_usd_per_1m: float | None
    output_usd_per_1m: float


def usd_per_million(tokens: int, rate: float) -> float:
    return tokens / 1_000_000 * rate


def estimate_review_cost(
    input_tokens: int,
    cached_input_tokens: int,
    output_tokens: int,
    rate: RateRow,
    group_ratio: float,
) -> float:
    fresh_input = max(input_tokens - cached_input_tokens, 0)
    cache_rate = rate.cache_hit_usd_per_1m or rate.input_usd_per_1m
    subtotal = (
        usd_per_million(fresh_input, rate.input_usd_per_1m)
        + usd_per_million(cached_input_tokens, cache_rate)
        + usd_per_million(output_tokens, rate.output_usd_per_1m)
    )
    return round(subtotal * group_ratio, 6)

The estimate is useful because it is reproducible. It is still only a review calculation. Final usage can move because output length, retries, cache behavior, account group, provider route, and source rows can move.

The decision record should therefore store both the estimate and the conditions that would make the estimate invalid.

{
  "estimate_scope": {
    "requests": 1000,
    "input_tokens": 120000000,
    "cached_input_tokens": 0,
    "output_tokens": 1800000,
    "group": "default"
  },
  "estimate_unknowns": [
    "final completion length depends on production prompts",
    "cache behavior is not assumed unless receipts prove it",
    "final charge depends on selected account or key group"
  ]
}

That wording matters. It keeps the decision honest without slowing the team down.

Require source version checks

The record should fail closed when a source version moves unexpectedly.

import json
import urllib.request


def load_json(url: str) -> dict:
    with urllib.request.urlopen(url, timeout=20) as response:
        return json.loads(response.read().decode("utf-8"))


record = json.load(open("route-decision-record.json", encoding="utf-8"))
pricing = load_json(record["sources"]["dated_pricing"]["url"])
routes = load_json(record["sources"]["live_routes"]["url"])

if pricing["pricing_version"] != record["sources"]["dated_pricing"]["pricing_version"]:
    raise SystemExit("dated_pricing_version_moved")

if routes["pricing_version"] != record["sources"]["live_routes"]["pricing_version"]:
    raise SystemExit("live_route_version_moved")

For a deliberate re-approval, update the record and make the diff visible. For an accidental movement, stop and review.

This is especially important when a route has both a public dated rate card and a live route table. Those sources answer different questions:

• The dated pricing JSON is a forecast source.
• The live route table is current route availability and group metadata.

Treating them as one interchangeable table creates confusion later.

Make the acceptance matrix explicit

A useful decision record has a small acceptance matrix.

{
  "acceptance": [
    {
      "gate": "route_exists",
      "method": "live route table contains target model ID",
      "required": true
    },
    {
      "gate": "pricing_row_exists",
      "method": "dated pricing source contains forecast row",
      "required": true
    },
    {
      "gate": "quality_slice_passes",
      "method": "10 redacted fixtures pass reviewer rubric",
      "required": true
    },
    {
      "gate": "receipt_fields_present",
      "method": "request receipt includes model, tokens, group, charge, retry count",
      "required": true
    },
    {
      "gate": "budget_band",
      "method": "review cost remains inside approved variance band",
      "required": true
    }
  ]
}

The point is not to add paperwork. The point is to prevent vague approval.

"Looks good" is hard to audit.

"Ten redacted support-summary fixtures passed the rubric, the pricing version is pinned, and receipt fields include group and retry count" is much easier to defend.

Include rollback before rollout

Rollback should be part of the decision, not an afterthought.

{
  "rollback": {
    "previous_model": "deepseek-v4-flash",
    "previous_record_id": "rdr-support-summary-2026-09-18",
    "allowed_until": "2026-10-02",
    "triggers": [
      "quality_slice_regression",
      "missing_receipt_field",
      "timeout_rate_above_gate",
      "budget_variance_above_gate"
    ],
    "verification_after_rollback": [
      "live route resolves previous model",
      "fixture set passes previous rubric",
      "new receipts show previous model ID"
    ]
  }
}

This small block changes the release conversation. The team does not need to improvise when a route misbehaves. It already knows which signal triggers reversal and which evidence proves reversal worked.

Route decision records and public copy

Internal evidence should be stronger than public language.

Safe public copy says:

This route was reviewed against a dated pricing source and a live route table before release.

Risky public copy says:

This route will always reduce cost or improve quality.

The first statement is about process and evidence. The second predicts an outcome without knowing the future workload, prompts, output length, retries, cache status, account group, provider state, or pricing source.

Good route records help teams move faster because they reduce argument. Each record answers the same questions:

• What changed?
• Which sources were checked?
• Which model IDs and group rules apply?
• Which acceptance gates passed?
• Which unknowns remain?
• Which receipt fields will prove runtime behavior?
• Which rollback trigger reverses the change?

If a team can answer those questions before production traffic arrives, the route change is no longer a leap of faith. It is a bounded decision with dated evidence.