Gate your AI agents' token cost in CI — before the bill, not after

A multi-agent run doesn't fail loudly when it gets expensive. It just quietly costs more.

One extra delegation loop, a tool that dumps a big blob into shared context, an agent that re-reads the whole history every turn — and the pipeline step that cost a few cents last week costs a few dollars this week. Nothing broke, so nothing warned you. You find out after the invoice, not before the run.

We already gate build time, bundle size, and test coverage in CI, and we fail the PR when they regress. The token/compute cost of the LLM steps themselves is the one budget we still tend to discover on the monthly bill. Here are three habits that have actually helped, then the concrete CI piece I was least sure how to do — and now do on every repo that runs agents.

1. Budget the prompt before the run, not after

Before kicking off an agent, look at what it actually carries into context: system prompt, tool schemas, accumulated task output. If a single agent walks in with 15k tokens of boilerplate it never uses, that's a fixed tax on every turn it takes. Trim it once, save it every run.

A zero-setup way to see the shape of a real session's cost — output vs cached re-reads vs cache writes — without wiring anything up:

npx @wartzar-bee/tokenscope --demo        # sample cost report, no logs needed
npx @wartzar-bee/tokenscope               # your most recent Claude Code session

2. Attribute tokens per source, not per message

"This run used 400k tokens" tells you nothing actionable. "This run used 400k tokens and 60% of them are the same three prior task outputs re-sent on every agent handoff" tells you exactly what to fix — cache it, summarize it, or stop broadcasting it. Attribution is what turns a scary number into a diff.

3. Gate the cost in CI the way you'd gate build time

This is the one that changed how I think about it. A run that suddenly costs 3x more is a regression, the same as a build that suddenly takes 3x longer — it should be able to fail a check, not surprise you in prod.

The tricky part is estimating a PR's cost statically, before you spend it. tokenscope scan does a pre-run token estimate over your agent code:

npx @wartzar-bee/tokenscope scan --dir .   # static, pre-run token estimate

and there's a GitHub Action that wraps it, comments the delta on the files responsible, and can block the PR on a regression. The honest way to adopt it is mode: warn first — it posts the same verdict and flags the breach in the Actions log, but never fails the build, so you can watch what it would block for a couple of weeks before turning enforcement on:

# .github/workflows/agent-cost.yml
name: Agent cost guardrail
on: pull_request
permissions:
  contents: read
  pull-requests: write        # so it can comment the delta on the PR
jobs:
  cost:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0       # needs the base branch to diff against
      - uses: wartzar-bee/ci-guardrail@v1
        with:
          github-token: ${{ secrets.GITHUB_TOKEN }}
          threshold-pct: 20    # flag if predicted tokens grow >20% vs base
          max-usd: 0.50        # hard dollar ceiling per run (0 = off)
          model: sonnet        # price the delta at the right $/1M
          mode: warn           # report only; switch to 'block' when you trust it

When you're ready to enforce, drop mode: warn (it defaults to block) and the PR can't merge until the regression is addressed. A couple of inputs worth knowing:

• min-delta-tokens (default 100) is a noise floor so a tiny base — 10 → 30 tokens, +200% — doesn't fail a build over a fraction of a cent. That's the commonest cost-guardrail false positive, off by default-sane.
• max-tokens / max-usd are absolute ceilings that catch a codebase that's already over budget and creeping up by a small percentage each PR — something a percentage gate alone can never flag.

Both the CLI (@wartzar-bee/tokenscope) and the Action (wartzar-bee/ci-guardrail) are Apache-2.0 and run locally / in your own runner — nothing phones home, it's reading your session logs and code. They're early and I'm actively working on them, so if you gate agent cost some other way, or the estimate is off for your setup, I genuinely want to hear it.

For those of you running agents in a real pipeline — do you gate the cost anywhere, or watch a dashboard and hope? What's caught you out?