Fail CI on Token Overruns with a GitHub Actions Budget Gate

A CI token budget gate fails the build as soon as an agent run exceeds a cap, instead of waiting until the allowance is gone. I implement it as a GitHub Actions step plus a Python checker that reads a JSONL usage log and returns a non-zero exit code on overrun.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. The gate itself is tool-agnostic; I adapt the field names and CLI to whatever agent server I run. MonkeyCode's free tier is only the example allowance in the snippets below.

Why I Gate Token Spend Instead of Metering It

A meter reports. A gate fails the build. That difference matters because a report lands in a log that nobody reads, while a failed build lands in someone's inbox.

I split the setup into two parts:

• A workflow step that runs the agent and captures its usage log
• A script that enforces the budget and returns a non-zero exit code on overrun

Meters are useful after the fact. They tell me what a run cost once I already paid for it. A gate is a quality check: if a pull request's agent task burns 400,000 tokens against a 250,000 cap, CI goes red before that pattern ships into every later run. I treat the gate the same way I treat a test failure or a lint error—loud, blocking, and specific.

Compared with a meter:

• A meter logs spend and stays green even when one PR doubled token use
• A gate fails that PR, so the author sees the overrun in the same inbox as a broken test

The 250,000-token-per-run figure in the workflow below is a starting point, not a rule. I set the real number from measured usage later in this article. Until I have that baseline, a cap is just a guess with a failure mode.

Capture the Agent Usage Log in GitHub Actions

I add a workflow that runs an agent task on every pull request, writes a JSONL usage log, then calls the checker. The monkeycode run command is a placeholder; I replace it with the actual command from the project's documentation. The workflow uses GitHub Actions workflow syntax.

name: agent-budget-gate

on:
  pull_request:

jobs:
  agent-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Run agent task
        id: agent
        run: |
          monkeycode run \
            --task 'generate tests for changed files' \
            --output usage.jsonl

      - name: Enforce token budget
        run: |
          python scripts/check_agent_budget.py usage.jsonl \
            --budget 250000 \
            --allowance 10000000

What each step and flag does

1. actions/checkout@v4 gives the agent the PR's tree.
2. The "Run agent task" step writes usage.jsonl next to the checkout.
3. The "Enforce token budget" step runs the checker with --budget 250000 and --allowance 10000000.

--allowance is for reporting only. It does not fail the job. --budget does. I keep them separate so a dashboard-style percentage can print in the log without changing the fail threshold. If the agent CLI writes the log to a different path, I pass that path to the checker. If it writes a single JSON object instead of JSONL, I convert the file or adapt load_usage() rather than force the format.

Compared with stuffing a one-line jq sum into the workflow, a checked-in script is reviewable, testable locally, and versioned with the repo. When the budget number changes, the YAML diff is obvious.

Enforce the Cap with a Python Budget Checker

The checker reads a JSON Lines file where each line describes one agent task. I adapt the field names to whatever the agent server actually emits.

#!/usr/bin/env python3
'''Enforce a token budget on an agent usage log.

Reads a JSONL file where each line describes one agent task:
{'task': 'generate tests', 'tokens_in': 1200, 'tokens_out': 3400, 'duration_ms': 8200}

Exit code 1 fails the build when the budget is exceeded.
Requires Python 3.9+.
'''
import argparse
import json
import sys
from pathlib import Path


def load_usage(path: Path) -> list[dict]:
    records = []
    with path.open() as fh:
        for line in fh:
            line = line.strip()
            if not line:
                continue
            records.append(json.loads(line))
    return records


def main() -> int:
    parser = argparse.ArgumentParser()
    parser.add_argument('log', type=Path)
    parser.add_argument(
        '--budget',
        type=int,
        required=True,
        help='Max tokens per run before the gate fails',
    )
    parser.add_argument(
        '--allowance',
        type=int,
        default=10_000_000,
        help='Total allowance, for reporting only',
    )
    args = parser.parse_args()

    records = load_usage(args.log)
    total = sum(
        record.get('tokens_in', 0) + record.get('tokens_out', 0)
        for record in records
    )
    count = len(records)
    pct = 100.0 * total / args.allowance

    print(f'Agent tasks: {count}')
    print(f'Tokens consumed: {total}')
    print(f'Share of allowance: {pct:.2f}%')

    if total > args.budget:
        print(f'BUDGET EXCEEDED: {total} > {args.budget}', file=sys.stderr)
        return 1

    print(f'Within budget: {total} <= {args.budget}')
    return 0


if __name__ == '__main__':
    sys.exit(main())

Dry-run the checker before you wire it into CI

I save this as scripts/check_agent_budget.py and run it locally against a captured log:

python scripts/check_agent_budget.py usage.jsonl --budget 250000

Expected behavior:

• Exit 0 and print Within budget: N <= 250000 when the sum of tokens_in and tokens_out stays under the cap.
• Exit 1 and print BUDGET EXCEEDED to stderr when it does not. GitHub Actions fails the job on a non-zero exit.

The script requires Python 3.9+ for list[dict] annotations. Missing fields default to 0 via record.get(...), so a partial log under-counts rather than crashing. That is conservative for availability, not for spend: if the server omits token fields, the gate will not fire. I verify one real log line against the documented schema before I trust the numbers.

Set the Budget from a Week of Real Runs

A budget set from intuition is a guess with a failure mode. I set it from data instead:

1. Run the agent for a week with no gate and collect usage.jsonl.
2. Compute the median tokens per run.
3. Set the budget at 2.5 times the median.

The 2.5× multiplier absorbs spikes without hiding regressions. If three consecutive runs exceed the budget, I review the task prompt before raising the cap. Prompt bloat is the usual culprit: a task description that grows over time drags in more context with every edit.

Worked example from a median

If a week of PR runs lands at a median of 80,000 tokens, 2.5× is 200,000—tighter than the 250,000 starter cap. If the median is 140,000, 2.5× is 350,000, and the starter cap would flap on normal work. I would rather change one number in YAML after a week of logs than debug flaky red builds.

I keep the gate on pull requests, not on a scheduled job that aggregates a week of spend. Per-run failure is what makes prompt bloat and retry loops visible. A weekly total can hide one runaway task inside a quiet average.

What the Gate Catches—and When I Skip It

A token budget gate surfaces problems that are otherwise invisible until the allowance runs out:

• Prompt bloat — a task description that slowly grew and now doubles the context on every call
• Loop amplification — an agent that retries failed steps and multiplies its spend
• Repository growth — more files in context as the codebase expands
• Allowance drift — team usage creeping toward the free tier's 10-million-token ceiling

Each of these is a slow, compounding change. A gate turns them into a single, loud failure.

The gate only sees what the agent reports. If the server does not emit per-task usage, the script cannot help; I check the project's documentation for the actual log format.

The free server is a shared resource. Latency and throughput vary with load, and the 10-million-token allowance is the current published figure, not a guarantee of permanence. I re-check the docs before relying on it in production. A token budget is not a cost budget. If the team later moves to a paid tier, the numbers change; I revisit the cap when pricing changes.

I skip this gate when:

• I run fewer than a handful of agent tasks per week. The gate is overhead.
• I have strict data-residency rules. A hosted server may be off-limits regardless of the allowance.
• I have not measured a baseline. A budget without data is just a guess with a failure mode.

The free tier is open to try. I run the gate against my own repository and see what the agent actually spends before I set the cap. A budget is not a restriction. It is a number that tells me when to look.

Next step: check out one real usage.jsonl from a typical PR, compute the median over a week of ungated runs, drop scripts/check_agent_budget.py into the repo, and add the two workflow steps above with --budget at 2.5× that median. The next overrun will fail the build instead of silently eating the allowance.

MonkeyCode provides free models that can run this workflow.