Build a Prompt Prefix Registry for Long-Context AI Gateways

Long-context AI agents do not become predictable just because the model accepts a million tokens.

A coding agent may reuse the same system prompt, repository map, style guide, security policy, tool schema bundle, and task rubric across hundreds of runs. A support agent may reuse the same product manual, escalation policy, and customer-safe answer format. A procurement workflow may reuse the same contract clauses and compliance checklist.

Those repeated blocks are valuable. They are also easy to lose.

If every request assembles the prefix in a slightly different order, inserts timestamps into stable text, changes whitespace, swaps tool schema order, or moves route metadata into the prompt, the gateway cannot reason clearly about cache-hit input. Engineering expects reuse. Finance sees fresh input. Customer support has no evidence for what changed.

A prompt prefix registry fixes the boring part of the problem. It records reusable prompt blocks as named, versioned artifacts. The router can then attach a stable prefix fingerprint to each request, keep cache assumptions out of the prompt body, and reconcile expected cache behavior against actual usage.

This article uses AIWave examples because AIWave exposes OpenAI-compatible routes across Chinese model providers. The pattern is not AIWave-specific. It applies to any gateway routing long-context workloads across DeepSeek, Qwen, GLM, Kimi, ERNIE, MiniMAX, Doubao, StepFun, MiMo, or another provider family.

The Problem Is Prefix Drift

Most teams start with a simple prompt builder:

prompt = system_prompt + "\n\n" + repo_summary + "\n\n" + user_task

That is fine for a prototype. It is not enough for production.

The reusable portion of a request needs its own lifecycle. A stable prefix should answer these questions before the request reaches the model:

Question	Why it matters
Which reusable blocks were included?	Explains what the agent knew before the task
Which exact versions were used?	Makes regression review possible
Was the block eligible for cache planning?	Separates stable context from task-specific context
Which route saw the request?	Cache behavior can differ by provider and model
Which pricing snapshot approved the route?	Keeps billing review tied to a dated source
Did actual usage match the cache assumption?	Finds prompt churn and route switching

Without those fields, a cache miss is just an invoice surprise. With them, it becomes a debuggable event.

Use Dated Pricing Evidence

Before writing this article, I checked AIWave's live pricing endpoint on August 29, 2026. It returned success=true, 63 model records, auto_groups=["default"], and pricing_version=a42d372ccf0b5dd13ecf71203521f9d2. The DeepSeek V4 rows exposed OpenAI-compatible route metadata for deepseek-v4-flash and deepseek-v4-pro, including model_ratio, completion_ratio, cache_ratio, create_cache_ratio, and enabled groups of default, vip, and svip.

That live endpoint is useful gateway evidence. It is not a reason to invent a public USD table. Public price claims need dated source links.

DeepSeek's official pricing page checked on August 29, 2026 lists deepseek-v4-flash and deepseek-v4-pro with cache-hit input, cache-miss input, and output rows. The visible USD table separates peak and off-peak prices. QwenCloud's pricing docs describe pay-as-you-go billing, context-tiered request billing, Batch API discounts, context caching, thinking-token billing, built-in tool fees, and bill-query views. Z.AI's pricing page separates input, cached input, cached input storage, output, and tool-related rows. Kimi's public API pricing page describes long-context billing, context caching, web search, reasoning effort, JSON mode, and OpenAI-compatible API usage, but the fetched page body did not expose every K3 price row, so do not quote K3 amounts without checking the billing console or a current official table.

The important point is structural: modern AI pricing has buckets. A prefix registry helps keep the cache bucket honest.

Registry Shape

The registry does not need to be complex. Start with a table or JSON document that defines reusable blocks.

{
  "prefix_id": "coding-agent-base",
  "version": "2026-08-29.1",
  "owner": "agent-platform",
  "purpose": "Shared instructions for repository analysis and patch generation",
  "cache_eligible": true,
  "content_sha256": "example_prefix_hash",
  "token_estimate": 18400,
  "allowed_routes": [
    "deepseek-v4-flash",
    "deepseek-v4-pro",
    "qwen3.5-27b",
    "glm-5.1"
  ],
  "source_files": [
    "system/coding-agent.md",
    "policy/security-review.md",
    "tools/repo-map-schema.json"
  ],
  "created_at": "2026-08-29T13:05:00Z"
}

Keep the registry separate from the prompt text. The prompt text can live in source control, object storage, or your application database. The registry is the auditable manifest that says which stable blocks were intended to be reused.

For a small gateway, a JSON file committed with the router may be enough. For a larger platform, use a database table with immutable versions and a review workflow.

Request Manifest

Each request should carry a manifest that joins the reusable prefix to the task-specific payload.

{
  "request_id": "req_20260829_001",
  "tenant_id_hash": "tenant_hash_example",
  "model": "deepseek-v4-pro",
  "prefixes": [
    {
      "prefix_id": "coding-agent-base",
      "version": "2026-08-29.1",
      "content_sha256": "example_prefix_hash",
      "cache_eligible": true,
      "token_estimate": 18400
    },
    {
      "prefix_id": "repo-map-aiwave-sdk",
      "version": "2026-08-29.3",
      "content_sha256": "example_repo_hash",
      "cache_eligible": true,
      "token_estimate": 61200
    }
  ],
  "task_tokens_estimate": 2200,
  "max_output_tokens": 1200,
  "pricing_version": "a42d372ccf0b5dd13ecf71203521f9d2",
  "pricing_checked_at": "2026-08-29T13:20:00Z"
}

This manifest gives you an audit trail without logging prompts, API keys, or customer content. You can answer which stable context was expected, which exact route handled the call, and which pricing snapshot approved the dispatch.

Build The Prefix Loader

The loader should produce deterministic text. That means stable ordering, stable separators, and no runtime-only values inside cache-planned blocks.

import hashlib
import json
from dataclasses import dataclass
from pathlib import Path


@dataclass(frozen=True)
class PrefixBlock:
    prefix_id: str
    version: str
    path: Path
    cache_eligible: bool


def normalize_text(text: str) -> str:
    lines = [line.rstrip() for line in text.splitlines()]
    return "\n".join(lines).strip() + "\n"


def digest(text: str) -> str:
    return hashlib.sha256(text.encode("utf-8")).hexdigest()


def load_prefixes(blocks: list[PrefixBlock]) -> tuple[str, list[dict]]:
    parts: list[str] = []
    manifest: list[dict] = []

    for block in sorted(blocks, key=lambda item: (item.prefix_id, item.version)):
        text = normalize_text(block.path.read_text(encoding="utf-8"))
        parts.append(f"<prefix id=\"{block.prefix_id}\" version=\"{block.version}\">\n{text}</prefix>")
        manifest.append({
            "prefix_id": block.prefix_id,
            "version": block.version,
            "content_sha256": digest(text),
            "cache_eligible": block.cache_eligible,
            "bytes": len(text.encode("utf-8")),
        })

    return "\n\n".join(parts), manifest


prefix_text, prefix_manifest = load_prefixes([
    PrefixBlock("coding-agent-base", "2026-08-29.1", Path("system/coding-agent.md"), True),
    PrefixBlock("repo-map-aiwave-sdk", "2026-08-29.3", Path("context/repo-map.md"), True),
])

print(json.dumps(prefix_manifest, indent=2))

The XML-like wrapper is not required. Use whatever format your model routes handle well. The key is determinism. If the same prefix version produces a different hash tomorrow, the registry should fail the build.

Keep Runtime State Out Of Cache-Planned Text

Some values change on every request. They should not be placed inside the stable prefix:

Runtime value	Better location
Request ID	audit manifest
Current timestamp	request metadata
User task	task payload
Effective billing group	gateway event
Pricing version	route manifest
Retry attempt	dispatch event
Trace ID	observability span

This separation feels strict, but it prevents accidental cache churn. A timestamp inside the stable prefix turns every request into a different prefix. A pricing version inside the prompt can do the same. The model does not need to read billing metadata unless the task is explicitly about billing.

Route With Prefix Awareness

The router should select a model after it sees the prefix manifest and task size. It should not route only on the user's requested model.

from dataclasses import dataclass


@dataclass(frozen=True)
class Route:
    model: str
    max_context_tokens: int
    supports_cache_planning: bool
    freshness_minutes: int


def choose_route(
    routes: list[Route],
    *,
    requested_model: str,
    prefix_tokens: int,
    task_tokens: int,
    output_cap: int,
) -> Route:
    required = prefix_tokens + task_tokens + output_cap
    candidates = [
        route for route in routes
        if route.model == requested_model and route.max_context_tokens >= required
    ]
    if not candidates:
        raise ValueError(f"no route can fit {required} tokens for {requested_model}")
    return candidates[0]


route = choose_route(
    [
        Route("deepseek-v4-flash", 1_000_000, True, 30),
        Route("deepseek-v4-pro", 1_000_000, True, 30),
        Route("glm-5.1", 128_000, True, 30),
    ],
    requested_model="deepseek-v4-pro",
    prefix_tokens=79_600,
    task_tokens=2_200,
    output_cap=1_200,
)

This example is intentionally small. A real router may also use latency class, allowed provider families, regional policy, tool support, retry policy, and account group. The prefix registry gives that router one missing input: the size and identity of reusable context.

Call The OpenAI-Compatible Endpoint

Once the route is chosen, the request can use a normal OpenAI-compatible client.

import os
from openai import OpenAI


client = OpenAI(
    base_url="https://aiwave.live/v1",
    api_key=os.environ["AIWAVE_API_KEY"],
)


def run_agent(prefix_text: str, task: str, model: str):
    return client.chat.completions.create(
        model=model,
        temperature=0,
        max_tokens=1200,
        messages=[
            {"role": "system", "content": prefix_text},
            {"role": "user", "content": task},
        ],
    )

The example keeps credentials in environment variables. It does not print the key. It does not log the prompt. In production, attach the request manifest to your tracing and billing events, not to the model-visible prompt unless the model truly needs it.

Reconcile Expected And Actual Usage

The registry earns its keep after the response returns.

def build_reconciliation_event(response, request_manifest: dict) -> dict:
    usage = getattr(response, "usage", None)
    prompt_tokens = getattr(usage, "prompt_tokens", None)
    completion_tokens = getattr(usage, "completion_tokens", None)

    expected_prefix_tokens = sum(
        item["token_estimate"]
        for item in request_manifest["prefixes"]
        if item["cache_eligible"]
    )

    return {
        "request_id": request_manifest["request_id"],
        "model": request_manifest["model"],
        "pricing_version": request_manifest["pricing_version"],
        "prefix_fingerprints": [
            f"{item['prefix_id']}@{item['version']}:{item['content_sha256'][:12]}"
            for item in request_manifest["prefixes"]
        ],
        "expected_cache_eligible_tokens": expected_prefix_tokens,
        "actual_prompt_tokens": prompt_tokens,
        "actual_completion_tokens": completion_tokens,
        "cache_reconciliation_status": "needs_provider_usage_join"
    }

If your provider or gateway returns cached-token usage directly, join it here. If it does not, keep the event anyway. The status makes the evidence gap visible. That is better than silently treating all prompt tokens as either cached or uncached.

Failure Modes To Alert On

The daily check should be small and specific.

Check	Failure signal
Prefix hash drift	Same prefix_id and version now produce a different hash
Prefix order drift	The same block set serializes in a different order
Cache expectation drift	Cache-eligible token estimate changes without a new version
Route mismatch	A prefix approved for one route is sent to another
Pricing drift	Request uses a stale pricing_version beyond policy
Usage gap	Cache-capable route has no cached-token evidence
Prompt logging risk	Raw prefix or user task appears in billing logs

These checks are cheap to run. They also create the evidence a Tier 1/2 buyer expects during procurement: not a broad promise, but a reproducible control.

Rollout Plan

Start with one high-volume workflow. Pick a coding agent, support triage agent, or document analysis agent that reuses large context. Do not start with every route.

First, put the stable blocks under registry control and compute fingerprints in shadow mode. Keep dispatch behavior unchanged. Compare prefix hashes for a week. If hashes drift daily, fix the builder before talking about cache optimization.

Second, attach request manifests to traces. Store prefix IDs, versions, hashes, model route, pricing version, and token estimates. Do not store raw customer prompts.

Third, block only obvious mistakes: missing prefix version, stale pricing snapshot, raw key pattern in prompt logs, or a route that cannot fit the planned context. These checks protect reliability without requiring perfect cache accounting.

Fourth, reconcile usage. Join actual prompt, output, and cached-token fields where available. Segment by workflow, model, account group, and prefix version. If a new prefix version increases fresh input materially, treat it like a production change.

Finally, use the registry in customer-facing support. When a buyer asks why a long-context run cost what it did, you should be able to answer with route, prefix versions, token buckets, pricing source, and request outcome. Do not ask support to reverse-engineer it from one blended usage number.

Source Links To Keep Beside The Registry

Keep source links in metadata, not inside the reusable prompt body.

For gateway routes, keep dated checks for AIWave pricing, AIWave model docs, and the live pricing endpoint at https://aiwave.live/api/pricing.

For provider calibration, keep dated checks for DeepSeek pricing, QwenCloud pricing, Z.AI pricing, and Kimi API pricing. Recheck them before changing a route policy.

Final Checklist

Before you call a long-context route in production, the gateway should know:

Question	Pass condition
Is the reusable context versioned?	Every stable block has an ID, version, and hash
Is the prompt deterministic?	Same versions produce the same serialized prefix
Is runtime state separate?	Request IDs, timestamps, pricing versions, and trace IDs stay out of the stable prefix
Is the route compatible?	Context size and allowed model routes are checked before dispatch
Is pricing dated?	The request stores the pricing snapshot used for admission
Is cache evidence visible?	Expected and actual cache behavior can be reconciled
Are secrets excluded?	Keys, prompts, and customer content are not written to billing logs

A long context window gives an agent room to work. A prompt prefix registry gives the gateway a way to prove what stayed stable, what changed, and which request should have received cache treatment.

That proof is what turns a clever prompt builder into production infrastructure.