Don't Rehearse a Bill on Gift Capacity

You cannot price a paid path by running the same prompt on gift capacity. A free queue has a different wait, a different preemption story, and a different retry surface. Treat it as a scratch pad. Rehearse spend only where the invoice will actually land.

That sounds obvious until a tool-calling agent walks in. One user message looks cheap. Then the model asks for a tool, you stuff the JSON back into the transcript, the context grows, and the "same" job is four prompts with a fatter tail.

On gift capacity you shrug. In production that tail is the bill.

Think of it like timing a delivery van in a bicycle lane. You will get a number. It will be a number about bicycles.

The usual failure is not a mysterious model. It is a staging habit. You point the same prompt text at a free server because the string matches, then you treat latency, retries, and token mix as if they transferred. Matching text is not matching economics. Cold start, boarding delay, mid-loop preemption, and the ratio of input tokens to output tokens all move when the iron and the meter diverge. Your canary starts lying the moment those two diverge.

Label the work before you label the model.

Every job needs a lane. exploratory may ride gift capacity while you are still wrong on purpose. rehearsal must hit the endpoint, the token cap, the tool budget, and the concurrency you will ship. committed is user-facing or tied to a ship clock. If you cannot name the lane, you are already in the wrong one. A prompt without a lane is how a sketch inherits a production retry ceiling, and how a production agent inherits a gift queue.

Here is a small gate you can run locally. It is a classifier, not a FinOps platform. It refuses to send rehearsal or committed work at a gift base URL. It also refuses to pretend a character heuristic is a vendor quote. Read the comments in the file before you trust a single integer it prints.

#!/usr/bin/env python3
"""job_lane.py — refuse to rehearse a bill on gift capacity.

Heuristic only. Ceiling, not a quote. Not a vendor invoice.
"""
from __future__ import annotations

import argparse
import json
import os
import sys
from dataclasses import dataclass
from typing import Literal

Lane = Literal["exploratory", "rehearsal", "committed"]

@dataclass
class JobEnvelope:
    job_id: str
    lane: Lane
    prompt: str
    tool_hops: int = 0
    max_tokens: int = 512
    concurrency: int = 1
    retry_ceiling: int = 1
    base_url: str = ""

    def heuristic_input_tokens(self) -> int:
        # char/4 stand-in. Not a tokenizer. Do not invoice from this.
        return max(1, len(self.prompt) // 4)

    def output_ceiling(self) -> int:
        hops = max(1, self.tool_hops + 1)
        return self.max_tokens * hops

def is_gift_endpoint(base_url: str) -> bool:
    marker = os.environ.get("GIFT_BASE_URL", "").rstrip("/")
    url = (base_url or os.environ.get("LLM_BASE_URL", "")).rstrip("/")
    if not url:
        return False
    if marker and url == marker:
        return True
    lowered = url.lower()
    return "gift" in lowered or "free" in lowered

def gate(job: JobEnvelope) -> dict:
    gift = is_gift_endpoint(job.base_url)
    input_tok = job.heuristic_input_tokens()
    out_ceil = job.output_ceiling()
    ceiling = (input_tok + out_ceil) * max(1, job.retry_ceiling)
    allowed, reason = True, "ok"

    if job.lane in ("rehearsal", "committed") and gift:
        allowed, reason = False, "refusing to rehearse or ship on gift capacity"
    elif job.lane == "committed" and job.retry_ceiling > 2:
        allowed, reason = False, "committed work cannot hide behind a wide retry ceiling"
    elif gift and job.concurrency > 1:
        allowed, reason = False, "concurrency on gift capacity is not a load test"

    return {
        "job_id": job.job_id,
        "lane": job.lane,
        "gift_endpoint": gift,
        "allowed": allowed,
        "reason": reason,
        "heuristic_input_tokens": input_tok,
        "heuristic_output_ceiling": out_ceil,
        "retry_ceiling": job.retry_ceiling,
        "heuristic_token_ceiling": ceiling,
        "note": "ceiling is a tripwire, not a vendor quote",
    }

def simulate_tool_loop(user_msg: str, tool_payloads: list[str]) -> dict:
    """The first message is not the meter. Each tool result is more prompt."""
    transcript = user_msg
    hops = []
    for i, payload in enumerate(tool_payloads, start=1):
        transcript += "\n" + payload
        hops.append({
            "hop": i,
            "transcript_chars": len(transcript),
            "heuristic_tokens": max(1, len(transcript) // 4),
        })
    return {
        "first_message_heuristic_tokens": max(1, len(user_msg) // 4),
        "final_transcript_heuristic_tokens": max(1, len(transcript) // 4),
        "hops": hops,
    }

def main() -> int:
    parser = argparse.ArgumentParser()
    parser.add_argument("--job", required=True)
    parser.add_argument("--simulate-tools", action="store_true")
    args = parser.parse_args()
    raw = json.loads(open(args.job, encoding="utf-8").read())
    job = JobEnvelope(**{k: raw[k] for k in JobEnvelope.__dataclass_fields__ if k in raw})
    result = gate(job)
    if args.simulate_tools:
        tools = raw.get("tool_payloads", [])
        result["tool_loop"] = simulate_tool_loop(job.prompt, tools)
    print(json.dumps(result, indent=2))
    if not result["allowed"]:
        print(result["reason"], file=sys.stderr)
        return 2
    return 0

if __name__ == "__main__":
    raise SystemExit(main())

Save a fixture that pretends to be a rehearsal. Notice the lane and the base URL disagree on purpose. That disagreement is the bug you want CI to catch, not a clever default you want the model to paper over.

{
  "job_id": "agent-rehearsal-014",
  "lane": "rehearsal",
  "prompt": "List open invoices, then draft a dunning note.",
  "tool_hops": 3,
  "max_tokens": 800,
  "concurrency": 4,
  "retry_ceiling": 3,
  "base_url": "https://gift.example.invalid/v1",
  "tool_payloads": [
    "{\"tool\":\"sql\",\"rows\": 42, \"csv\": \"...\"}",
    "{\"tool\":\"crm\",\"notes\": \"prior balance, three emails\"}",
    "{\"tool\":\"policy\",\"tone\": \"firm\"}"
  ]
}

Run the gate twice. First as a reject. Then point base_url at the path you actually pay, drop concurrency to what you will ship, and run it again as a ceiling check. The second run is still not an invoice. It is a tripwire you can fail a build with.

python3 job_lane.py --job rehearsal.json --simulate-tools; echo exit:$?

# expected: exit 2, reason about gift capacity

GIFT_BASE_URL="https://gift.example.invalid/v1" \
LLM_BASE_URL="https://gift.example.invalid/v1" \
python3 job_lane.py --job rehearsal.json; echo exit:$?

The --simulate-tools path is the part most cost dashboards still flatten. Each tool result is not a side channel. It is more prompt. If you only count the first user message, you are counting the first mile of a round trip and calling it the route. Watch the hop list grow. The last hop is where output tokens get expensive, because the model is now reading its own homework plus yours.

That is why gift capacity is the wrong rehearsal. Preemption in the middle of a four-hop tool loop does not just waste time. It often restarts the graph. You already paid in calendar, and on a metered path you may pay in tokens for the partial transcript too. The free path hides that, because you are not looking at an invoice. You are looking at a courtesy.

Queue time still burns calendar. Calendar is what a release train spends. A gift server that boards you in twenty minutes is not a rehearsal of a 400ms p95. It is a different sport with the same jersey. Load tests on that sport will teach finance the wrong lesson, and they will teach you to overfit a queue you will not have on Monday morning.

So split the lanes on purpose, in the job envelope, not in a wiki nobody reads at 2 a.m.

Use gift capacity to be wrong cheaply. Draft the prompt. Throw away the transcript. Confirm the tool schema even parses. Confirm the agent does not need seven hops to ask for the date. Then copy the envelope onto the paid path and run a short rehearsal at production concurrency, with the same max_tokens, the same stop conditions, and the same retry ceiling. Record input tokens, output tokens, tool hops, queue wait, and wall time as separate columns. If you fold them into one duration number, you will optimize the wrong column. Duration can look fine while output tokens quietly eat the month.

For the exploratory lane only, MonkeyCode — an open-source project with free model access and a free server option — can hold draft-and-discard work so you do not burn the paid meter while the schema is still wrong. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Park sketches there, then move rehearsal onto the endpoint you will actually invoice.

A limitation worth saying out loud: this gate does not know your vendor's billing rules. Some meters count killed streams. Some count cached prefixes differently. Some round up. Some bill tool payloads at a different rate than the surrounding prose. The char/4 stand-in will drift on code, on CJK, and on compressed JSON. Use it as a tripwire that fails closed when a lane and an endpoint disagree. Do not use it as a quote you paste into a budget slide.

Who should not put work on gift capacity at all: anything carrying secrets or regulated user data; anything you plan to treat as an availability replica; any load test whose numbers will be shown to finance as a forecast. Gift capacity is not a cluster you own. It is not a capacity reserve. It is a courtesy with a different SLO, which is to say, maybe none. If your agent is the product, the free path is the wrong bet for canaries. If your job is still a sketch, burning the paid meter to find a better system prompt is the other wrong bet.

Keep those two mistakes from sharing a queue. Name the lane. Gate the URL. Count every hop. Then believe the invoice, not the gift.