Catch runaway AI agent costs before merge with a GitHub Action

ronnie0297-stack — Mon, 21 Sep 2026 15:10:20 +0000

AI cost control usually begins after deployment: a provider dashboard reports what has already been spent. But several expensive failure modes are visible in code before a pull request is merged.

Consider a tool-using Vercel AI SDK call:

return streamText({
  model: openai("gpt-5.4-mini"),
  prompt: question,
  tools: { search }
});

Two limits are missing. There is no output-token ceiling, and the tool loop has no stopping condition. Even if the prompt looks harmless, neither the worst-case response size nor the maximum number of steps is explicit.

A bounded version makes both decisions reviewable:

return streamText({
  model: openai("gpt-5.4-mini"),
  prompt: question,
  maxOutputTokens: 800,
  stopWhen: stepCountIs(5),
  tools: { search }
});

Turning the check into a budget gate

Code alone cannot predict a bill. A useful estimate also needs traffic assumptions. Keep those assumptions in the repository so reviewers can challenge them:

{
  "monthlyCallsPerSite": 10000,
  "assumedInputTokens": 1000,
  "assumedOutputTokens": 1000,
  "warnMonthlyCost": 50,
  "failMonthlyCost": 250,
  "failOnUnboundedTools": true
}

Then estimate each call site with a deliberately simple formula:

monthly cost = monthly calls ×
  ((input tokens × input price) + (maximum output tokens × output price))

The estimate is not an invoice prediction. It is a consistent review signal. A changed model, increased output ceiling, or new call site produces a visible change before production.

Running it in GitHub Actions

I built an open-source Action called AICostFence to automate this check:

name: AI cost check
on: pull_request

permissions:
  contents: read
  pull-requests: write

jobs:
  cost-check:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: ronnie0297-stack/aicostfence@v0.1.0
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

It scans locally on the GitHub runner, posts one updated pull-request report, and can fail the check when a configured guard is violated. It does not call an AI model or send source code to a model provider.

The current release intentionally supports a narrow surface: JavaScript/TypeScript projects using the Vercel AI SDK's generateText, streamText, generateObject, or streamObject calls. That makes the analysis deterministic while the early workflow is validated.

Repository: https://github.com/ronnie0297-stack/aicostfence

Marketplace: https://github.com/marketplace/actions/aicostfence

npm: https://www.npmjs.com/package/aicostfence

What cost control would be most useful in your pull requests: model-swap deltas, provider-specific checks, or organization-wide policies?

DEV Community: ronnie0297-stack

Catch runaway AI agent costs before merge with a GitHub Action

Turning the check into a budget gate

Running it in GitHub Actions