DEV Community

Casey Sun
Casey Sun

Posted on

Fail CI When the Agent Treats Free Compute as Inventory

The scene below is a labeled example, not a memoir.
It exists to show a planning failure, not a personal claim.

The coding agent opened a pull request at 02:11 UTC.
Its plan file routed every call through free inference.
A free server hosted the long-running worker process.

The commit message called the change a cost optimization.
Reviewers almost merged it before the morning standup.

The diff did not contain a billing owner.
It also lacked a failover model and an exit date.
The agent had treated complimentary capacity as inventory.

That silent assumption is the actual planning bug.
This article records a refuse list for reviewers.

How free capacity enters the plan

Coding agents copy README snippets into plans.
Many READMEs mention free tiers near the quickstart.
The agent then treats the quickstart as topology.

That is the assumption failure in another form.
The agent did not evaluate blast radius.
It completed a cheap path that satisfied the prompt.

Tool-calling loops then amplify the same mistake.
Retries look free when the route has no meter.
The plan grows a worker to absorb that retry heat.

None of that appears in the unit tests.
Tests pass on a laptop with a short fixture.
The merged service then inherits a complimentary runtime.

The failure is in the plan, not the model

Agents optimize hard for the prompt they received.
A prompt that says ship something cheap produces free defaults.
Those defaults look responsible in a cost-sensitive org.

They still are not a real capacity plan.
Free model access remains a spike surface only.
A free server is a sandbox, not a fleet.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.
MonkeyCode currently offers free model access and a free server option.
Those two facts are useful for labeled experiments.

They are not a reason to accept an agent-authored production topology.
Labeled spikes still pass through the same refuse list.

Red flags in agent-authored plans

Reviewers should scan the plan file first.
Code review of the worker should come second.

Reject the plan when any item below appears unlabeled.
Silence around capacity is not an implicit approval.

  • The agent selects free inference as the only route.
  • The agent places a queue consumer on a free server.
  • The agent stores session state on that same host.
  • The agent omits a paid or self-hosted fallback.
  • The agent omits a named owner for spend.
  • The agent treats retries as infinite free budget.
  • The agent writes autoscaling against an unknown quota.
  • The agent copies secrets into the free runtime.
  • The agent labels the unlabeled setup as production-ready.

Each listed flag is independently sufficient for rejection.
Two flags together should stop the merge.

A reviewer decision table

The table below is a working artifact.
Teams can paste it into the review template.

Signal in the agent plan Default action Better alternative
Free model is the only route Block Named primary plus fallback
Free server runs a worker Block Ephemeral sandbox with TTL
Retries have no budget cap Block Hard token and time ceilings
Secrets reach the free host Block Vault injection on owned runtime
No owner, no exit date Block CODEOWNERS plus a kill date
Eval uses the same free model Block Separate judge on owned capacity
Queue plus free host plus local disk Block Managed queue and owned compute

The default action is block, not comment.
Comments do not prevent the next silent default.

Artifact: policy file, scanner, and fixtures

The following files are labeled examples for operators.
Operators should treat them as unexecuted samples.
Run them only inside a throwaway clone first.

Policy file

Operators can save the deny rules as agent-compute-policy.yml.

# Example (unexecuted): reviewer policy for agent-authored plans
version: 1
mode: refuse_by_default
require:
  plan_file: AGENT_PLAN.md
  owner: CODEOWNERS
  exit_date: true
  environment_label: [sandbox, spike, prod]
deny:
  unlabeled_free_model_default: true
  unlabeled_free_server_worker: true
  secrets_on_free_runtime: true
  unbounded_retries: true
  self_eval_on_same_route: true
allow_spike:
  environment_label: sandbox
  max_ttl_hours: 24
  must_include_fallback: true
  must_include_kill_switch: true
exit_criteria:
  - fallback_route_tested
  - owner_acked
  - secrets_not_copied
  - ttl_enforced
Enter fullscreen mode Exit fullscreen mode

The policy does not name vendors or models.
It names only behaviors that fail review.

Scanner

Operators can save the scanner as scan_agent_plan.py.

#!/usr/bin/env python3
"""Example (unexecuted): fail CI when an agent plan defaults to free capacity."""
from __future__ import annotations

import re
import sys
from pathlib import Path

PLAN = Path("AGENT_PLAN.md")

DENY_PATTERNS = [
    (r"free\s+model", "unlabeled free model default"),
    (r"free\s+server", "unlabeled free server worker"),
    (r"retry\s+forever|infinite\s+retry", "unbounded retries"),
    (r"production-ready", "unearned production claim"),
    (r"self-?host(ed)?\s+on\s+free", "free runtime as home"),
]

REQUIRE_PATTERNS = [
    (r"owner:\s*\S+", "named owner"),
    (r"exit[_-]?date:\s*\d{4}-\d{2}-\d{2}", "exit date"),
    (r"fallback:\s*\S+", "fallback route"),
    (r"env:\s*(sandbox|spike|prod)", "environment label"),
    (r"kill[_-]?switch:\s*\S+", "kill switch"),
]


def main() -> int:
    if not PLAN.exists():
        print("FAIL: AGENT_PLAN.md missing from the change")
        return 2
    text = PLAN.read_text(encoding="utf-8").lower()
    failures = []
    for pat, label in DENY_PATTERNS:
        if re.search(pat, text) and "sandbox" not in text:
            failures.append(f"deny: {label}")
    for pat, label in REQUIRE_PATTERNS:
        if not re.search(pat, text):
            failures.append(f"missing: {label}")
    if "secret" in text and "free server" in text:
        failures.append("deny: secrets mentioned with free server")
    if failures:
        print("REFUSE PLAN")
        for item in failures:
            print(f"- {item}")
        return 1
    print("PLAN GATED: required fields present")
    return 0


if __name__ == "__main__":
    sys.exit(main())
Enter fullscreen mode Exit fullscreen mode

Teams should wire the scanner as a required check.
A missing plan file is already a failure.

The commands below belong in the required check.

# Example (unexecuted)
python3 scan_agent_plan.py
git diff --name-only origin/main...HEAD | grep -E 'AGENT_PLAN.md|agent-compute-policy.yml'
Enter fullscreen mode Exit fullscreen mode

Agents that skip planning should not skip the gate.
Empty diffs still need an explicit plan file.

Plan header

Require this header in every AGENT_PLAN.md.

# AGENT_PLAN
owner: platform-oncall
exit_date: 2026-09-12
env: sandbox
fallback: owned-endpoint-b
kill_switch: FEATURE_AGENT_RUNTIME=off
token_budget: set-by-human
retry_budget: 3
notes: spike only; not a worker host
Enter fullscreen mode Exit fullscreen mode

Humans must fill every budget field by hand.
Agents must not invent quotas or capacity numbers.

Fixtures the scanner should refuse and allow

A failing fixture should trip several deny rules.
Store the failing sample under fixtures/plan_should_fail.md.

# AGENT_PLAN
owner: the-agent
env: production-ready
route: free model only
notes: retry forever on the free server
secrets: copy .env onto the host
Enter fullscreen mode Exit fullscreen mode

A passing fixture uses sandbox labels and an owner.
Store the passing sample under fixtures/plan_should_pass.md.

# AGENT_PLAN
owner: platform-oncall
exit_date: 2026-09-12
env: sandbox
fallback: owned-endpoint-b
kill_switch: FEATURE_AGENT_RUNTIME=off
token_budget: set-by-human
retry_budget: 3
notes: spike only; not a worker host
Enter fullscreen mode Exit fullscreen mode

The tests below pin the scanner's refuse contract.

# Example (unexecuted): pytest contract for scan_agent_plan.py
import subprocess
import sys
from pathlib import Path

SCANNER = Path("scan_agent_plan.py").resolve()


def run_scanner(tmp_path, body: str) -> subprocess.CompletedProcess:
    plan = tmp_path / "AGENT_PLAN.md"
    plan.write_text(body, encoding="utf-8")
    return subprocess.run(
        [sys.executable, str(SCANNER)],
        cwd=tmp_path,
        capture_output=True,
        text=True,
    )


def test_refuses_unlabeled_free_defaults(tmp_path):
    body = Path("fixtures/plan_should_fail.md").read_text(encoding="utf-8")
    result = run_scanner(tmp_path, body)
    assert result.returncode == 1
    assert "REFUSE PLAN" in result.stdout


def test_allows_labeled_sandbox_spike(tmp_path):
    body = Path("fixtures/plan_should_pass.md").read_text(encoding="utf-8")
    result = run_scanner(tmp_path, body)
    assert result.returncode == 0
    assert "PLAN GATED" in result.stdout


def test_missing_plan_fails_closed(tmp_path):
    result = subprocess.run(
        [sys.executable, str(SCANNER)],
        cwd=tmp_path,
        capture_output=True,
        text=True,
    )
    assert result.returncode == 2
Enter fullscreen mode Exit fullscreen mode

GitHub Actions can enforce the gate on pull requests.

# Example (unexecuted)
name: agent-plan-gate
on:
  pull_request:
jobs:
  refuse-free-defaults:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Fail unlabeled free-capacity plans
        run: python3 scan_agent_plan.py
Enter fullscreen mode Exit fullscreen mode

Better alternatives

Free capacity still has a narrow, labeled job.
It belongs in a named sandbox with a clock.

  1. Run the spike on a free model route under a budget file.
  2. Keep the worker on owned compute, even if idle.
  3. Record fallback latency with a single synthetic probe.
  4. Delete the sandbox when the exit date hits.
  5. Promote only the prompt and the tests, not the host.

A free server can host a short-lived demo.
It should not host a queue consumer.
It should not host a webhook with retries.

Complimentary inference belongs only to drafts and spikes.
Drafts do not become the production path by silence.

Exit criteria

Write the exit test before the spike starts.
The merge of the spike requires every line below.

  • The owner field names a human in CODEOWNERS.
  • The environment label is explicit in the plan.
  • The fallback route has answered one synthetic request.
  • Secrets never landed on the free host at all.
  • Retry count is a number, not a hope.
  • The kill switch disables the route in one deploy.
  • The exit date is in the calendar, not the README.

If any line is false, keep the branch closed.
Partial exits are delayed production accidents in disguise.

Who should not use this gate

This refuse list is for teams with agents that write plans.
It is not a general cloud cost guide.

Skip this workflow when the agent cannot open pull requests.
Skip it when humans already choose every runtime by hand.

Skip it when the org forbids complimentary endpoints entirely.
Skip it for air-gapped runtimes with no free tier at all.

Do not use the scanner as a security audit.
It only matches words in the plan file.
It does not prove isolation or secret handling.

Do not use it as a license to keep a free worker alive.
A passing scan is not a capacity contract.

Limitations

Regex gates drift quickly as agents change vocabulary.
Update patterns when a denied plan still merges.

The policy does not measure model quality at all.
It also does not measure free server uptime.
It does not invent quotas, SKUs, or hardware.

Teams must verify current product terms for themselves.
Availability claims can change without a code review.

The almost-merged 02:11 pull request was a planning failure.
Cheap defaults remain defaults under a new label.
Reviewers should refuse the plan, then discuss the spike.

Keep every sandbox labeled in the plan file.
Put the exit date in the same pull request.

Top comments (0)