Build a request-group audit trail for AI API billing

Most AI API billing bugs are not dramatic. They usually start with a small mismatch between the account state and the request that actually reached the gateway.

A user is moved to a different billing group. A token was created before that change. A routing layer still has a stale model ID. The application retries on a different route and merges the cost into the first attempt. Finance sees one number, engineering sees another, and nobody can say which request used which rule.

For a small prototype, that is annoying. For a paid product that routes high-volume coding agents across DeepSeek, Qwen, GLM, Kimi, ERNIE, MiniMAX, Doubao, StepFun, and MiMo routes, it becomes a trust problem.

The fix is not a longer pricing page. You need a request-group audit trail: a compact record that says which billing group was effective for the request, which model route handled it, which pricing snapshot was used, how many tokens landed in each bucket, and what happened after retries.

This article shows one implementation pattern. The examples use AIWave because its live pricing endpoint exposes the pieces that make the problem visible, but the pattern works for any OpenAI-compatible gateway.

Why the effective group matters

An account group is a policy. A request group is evidence.

Those two values can drift. For example, a customer may qualify for VIP pricing, but the request still needs to prove which group was used at execution time. If your system only stores the user's current group, a later audit can accidentally rewrite history. If you only store final spend, you cannot separate a valid bill from a stale token, a retry, or a fallback route.

The audit record should answer these questions without reading secrets or prompts:

Question	Field to keep	Why it matters
Which account sent the call?	account_id_hash	Keeps the record joinable without exposing the account
Which token path was used?	token_id_hash	Finds stale token behavior without logging a key
Which billing group applied?	effective_group	Separates account policy from execution evidence
Which model handled it?	model	Prevents family-level names from hiding route changes
Which rate snapshot was used?	pricing_version, rate_checked_at	Makes later reconciliation possible
What was billed?	input_tokens, cache_hit_tokens, output_tokens	Avoids one blended token bucket
Did retry spend occur?	attempt, parent_request_id	Keeps fallback cost visible
What failed?	status_class, provider_error_code	Separates provider failure from client misuse

The most useful audit trail is boring. It does not store prompts. It does not store API keys. It does not need a universal benchmark score. It gives you enough facts to reconstruct the bill.

Use a dated pricing snapshot

I pulled the live AIWave pricing endpoint on August 28, 2026 before writing this post. The endpoint returned success=true, pricing_version=a42d372ccf0b5dd13ecf71203521f9d2, 63 model records, and group ratios of default=3 and vip=1. The same response exposed supported OpenAI-compatible routes and per-model fields such as model_ratio, completion_ratio, cache_ratio, and enable_groups.

Treat those fields as gateway billing evidence, not as permission to invent a public USD table. A published USD price needs a dated source.

For DeepSeek V4 budgeting, the AIWave public rate card checked on August 28, 2026 lists all-day AIWave prices per one million tokens:

AIWave route	Input	Output	Cache-hit input
deepseek-v4-flash	$0.638	$1.914	$0.0203
deepseek-v4-pro	$1.914	$5.742	$0.0638

DeepSeek's own pricing page checked the same day separates cache-hit input, cache-miss input, and output for V4 Flash and V4 Pro, with peak and off-peak rows. QwenCloud documents context-tiered request billing, Batch API discounts, context caching, thinking-token billing, and built-in tool fees. Z.AI's GLM pricing separates input, cached input, cache storage, output, and tool fees. Kimi's public API pricing describes long-context billing and web-search invocation fees.

That variety is the reason the audit record needs a pricing_source and rate_checked_at. "Model cost" is not one number across providers.

A compact event shape

Start with a line-delimited event. It can go to a warehouse, a queue, or a plain JSONL file while the system is young.

{
  "request_id": "req_20260828_001",
  "parent_request_id": null,
  "attempt": 1,
  "account_id_hash": "acct_hash_example",
  "token_id_hash": "tok_hash_example",
  "effective_group": "vip",
  "model": "deepseek-v4-pro",
  "provider_family": "deepseek",
  "pricing_version": "a42d372ccf0b5dd13ecf71203521f9d2",
  "pricing_source": "https://aiwave.live/api/pricing",
  "rate_checked_at": "2026-08-28T13:36:27+08:00",
  "input_tokens": 120000,
  "cache_hit_tokens": 80000,
  "output_tokens": 6000,
  "status_class": "success",
  "provider_error_code": null
}

Two details matter here.

First, the effective group is stored on the request. Do not rely on the user's current group when you audit a call from last week.

Second, retries are first-class rows. If a request tries deepseek-v4-pro, receives a provider timeout, then retries on deepseek-v4-flash, the second attempt should carry the same parent_request_id and a higher attempt number. Merging those attempts into a single cost line hides the operational cause.

Fetch the catalog before routing

You do not need to fetch pricing on every request. You do need to pin a fresh enough snapshot for the routing decision.

import datetime as dt
import json
import os
import urllib.request

PRICING_URL = "https://aiwave.live/api/pricing"


def fetch_pricing_snapshot() -> dict:
    req = urllib.request.Request(
        PRICING_URL,
        headers={"User-Agent": "billing-audit-example"},
    )
    with urllib.request.urlopen(req, timeout=20) as response:
        payload = json.loads(response.read().decode("utf-8"))

    if not payload.get("success"):
        raise RuntimeError("pricing endpoint did not return success=true")

    rows = payload.get("data") or []
    return {
        "checked_at": dt.datetime.now(dt.timezone.utc).isoformat(),
        "pricing_version": payload.get("pricing_version"),
        "group_ratio": payload.get("group_ratio", {}),
        "rows_by_model": {row["model_name"]: row for row in rows},
    }


snapshot = fetch_pricing_snapshot()
route = snapshot["rows_by_model"]["deepseek-v4-pro"]

print(snapshot["pricing_version"])
print(route["model_name"], route.get("enable_groups"))

Store the snapshot version with the routing policy that used it. If a finance review happens later, you can say which catalog existed when the request was admitted.

Wrap the OpenAI-compatible call

The request wrapper should add audit metadata without changing the client contract.

import hashlib
import os
from openai import OpenAI


def digest(value: str) -> str:
    salt = os.environ["AUDIT_HASH_SALT"]
    return hashlib.sha256((salt + value).encode("utf-8")).hexdigest()[:20]


client = OpenAI(
    base_url="https://aiwave.live/v1",
    api_key=os.environ["AIWAVE_API_KEY"],
)


def run_with_audit(
    *,
    request_id: str,
    account_id: str,
    token_id: str,
    effective_group: str,
    model: str,
    pricing_snapshot: dict,
    messages: list[dict],
) -> tuple[str, dict]:
    response = client.chat.completions.create(
        model=model,
        messages=messages,
        temperature=0,
        max_tokens=700,
    )

    usage = response.usage
    event = {
        "request_id": request_id,
        "parent_request_id": None,
        "attempt": 1,
        "account_id_hash": digest(account_id),
        "token_id_hash": digest(token_id),
        "effective_group": effective_group,
        "model": model,
        "pricing_version": pricing_snapshot["pricing_version"],
        "pricing_source": PRICING_URL,
        "rate_checked_at": pricing_snapshot["checked_at"],
        "input_tokens": getattr(usage, "prompt_tokens", 0),
        "cache_hit_tokens": 0,
        "output_tokens": getattr(usage, "completion_tokens", 0),
        "status_class": "success",
        "provider_error_code": None,
    }

    return response.choices[0].message.content, event

This wrapper hashes local identifiers, uses environment variables for credentials, caps output, and records the pricing snapshot. It does not log prompts or the API key.

If your gateway returns cached-token usage directly, populate cache_hit_tokens from the response. If it does not, keep the field and set it to 0 or null consistently. The important part is that the absence of cache evidence is visible.

Estimate with buckets, not averages

A quick estimator can keep billing discussions honest during review. The example below uses the public DeepSeek V4 AIWave rates checked on August 28, 2026. Replace the table with your own dated rate source.

RATES_PER_1M = {
    "deepseek-v4-flash": {
        "input": 0.638,
        "cache_hit": 0.0203,
        "output": 1.914,
        "source": "https://aiwave.live/pricing",
        "checked_at": "2026-08-28",
    },
    "deepseek-v4-pro": {
        "input": 1.914,
        "cache_hit": 0.0638,
        "output": 5.742,
        "source": "https://aiwave.live/pricing",
        "checked_at": "2026-08-28",
    },
}


def estimate_usd(event: dict) -> float:
    rate = RATES_PER_1M[event["model"]]
    uncached = max(event["input_tokens"] - event["cache_hit_tokens"], 0)
    total = (
        uncached / 1_000_000 * rate["input"]
        + event["cache_hit_tokens"] / 1_000_000 * rate["cache_hit"]
        + event["output_tokens"] / 1_000_000 * rate["output"]
    )
    return round(total, 6)

Do not replace the three buckets with one average unless the source itself bills that way. A long-context coding agent with a high cache-hit ratio and a short extraction task with almost no cache reuse are not the same workload.

Review the trail before customer support needs it

The audit trail should be reviewed before an invoice dispute appears. A small daily job can catch the common failure modes:

Check	Query logic	Action
Unknown group	effective_group not in current allowed groups	quarantine the route policy
Stale snapshot	rate_checked_at older than your threshold	refresh pricing before new traffic
Missing cache field	cache_hit_tokens is null for cache-capable routes	fix usage parsing or mark unsupported
Retry cost hidden	successful parent has multiple attempts but one bill row	split attempts in reporting
Family name only	model lacks a versioned route ID	block production promotion
Source mismatch	USD estimate lacks source and checked_at	reject the estimate

This is where the system earns trust. A buyer in the US, UK, Germany, Japan, Singapore, or another Tier 1/2 market is not only comparing prices. They are asking whether your numbers can be explained after a high-volume agent run.

Keep provider breadth out of the bill

AIWave's current public positioning is 60+ AI models from 9 Chinese AI providers. That is useful for evaluation breadth, but it should not leak into billing logic.

Do not count the vendors array from one endpoint and call it the provider total unless the product explicitly says that field is the public provider map. Do not route on a family label such as "Qwen" or "GLM" when the pricing fields sit on model records. Do not treat an upstream provider's price page as the same thing as the gateway invoice.

The request record should stay closer to the machine:

account hash
token hash
effective group
exact model ID
pricing version
dated source
token buckets
attempt result

Those nine lines are enough to keep most billing reviews grounded.

Publish a billing answer, not a pricing slogan

The test for an AI gateway is not whether it can display a large catalog. It is whether a developer can run a real workload, receive an invoice, and understand the path from request to charge.

A request-group audit trail gives that answer. It records the effective group instead of assuming it. It pins the model route instead of naming a family. It keeps cached input, uncached input, and output separate. It saves the pricing version and source date beside the usage row. It shows retries as operational events, not mystery spend.

That is the kind of evidence paid teams need before they move production traffic through any multi-model gateway.