DEV Community

Taylor Lin
Taylor Lin

Posted on

Admission Control for Free-Tier Coding Agents: A Glossary, a Decision Tree, and Four Worked Leaves

Illustrative scenario, not a case study. You have a backlog of forty tasks and a hosted coding agent that runs on a free tier. Someone suggests sending the whole backlog through it over a weekend. Nothing in this post describes a real deployment, and none of the code below was executed — it is a harness you can adapt, drawn to show where the decision points are.

The useful question is not "how many tokens do we have left?" It is "which tasks deserve to be sent at all?" A free allowance is a budget for experiments, and budgets are spent fastest on tasks you cannot check.

Glossary: six terms worth fixing before you argue

Term Working definition Why it changes the decision
Reversal cost Work needed to undo the change: git revert, a data migration, a credential rotation, a customer email Anything above "revert a commit" disqualifies the task from unsupervised runs
Verification latency Wall-clock time from "agent finished" to "a machine says pass/fail" If it exceeds a few minutes, you will skip it, and an unverified pass is noise
Blast radius Set of files, services, or users a wrong answer can touch Bounded by a file allowlist, not by good intentions
Context dependency How much private or tacit knowledge the task needs that you cannot paste into a prompt High dependency turns into guesswork with confident prose
Deliverable contract The exact command and the exact expected outcome that defines "done" Without one, "done" becomes a human opinion
Task card A small machine-readable record: goal, base commit, allowlist, acceptance command Makes admission checkable instead of conversational

The tree: four questions, asked in order

Ask these in sequence and stop at the first hard stop. The order matters — reversibility first, because it is the cheapest thing to establish and the most expensive thing to get wrong.

  1. Is the change reversible without a data migration or credential rotation? No → Leaf D.
  2. Can a machine decide pass/fail in a couple of minutes with a command you already trust? No → Leaf C.
  3. Can you bound the scope with a file allowlist and one acceptance command? No → shrink the task; if it still will not fit → Leaf C.
  4. Does the task need context you cannot paste (private docs, tribal knowledge, production state)? Yes → Leaf C. Otherwise, if a failing test can be written first → Leaf B, else → Leaf A.

Four leaves. Each one gets a concrete artifact.

Leaf A — admit as-is, with an allowlist guard

The task is reversible, checkable, and bounded. The artifact you need is not a better prompt; it is a guard that fails the run when the diff escapes the agreed scope.

Task card first (illustrative):

# task.yml — illustrative task card, not a real ticket
id: T-1042
goal: add a retry cap to the webhook sender
base: origin/main
allow:
  - "src/webhooks/*.py"
  - "tests/webhooks/*.py"
acceptance: "pytest tests/webhooks -q"
Enter fullscreen mode Exit fullscreen mode

Then the guard (illustrative, unexecuted):

# check_scope.py — illustrative; adapt before trusting it.
import subprocess, sys, fnmatch, yaml  # PyYAML

card = yaml.safe_load(open("task.yml"))
changed = subprocess.check_output(
    ["git", "diff", "--name-only", f"{card['base']}...HEAD"], text=True
).split()

outside = [f for f in changed
           if not any(fnmatch.fnmatch(f, p) for p in card["allow"])]
if outside:
    print("BLOCKED: files outside allowlist:")
    for f in outside:
        print("  -", f)
    sys.exit(1)
print(f"OK: {len(changed)} changed files, all inside allowlist")
Enter fullscreen mode Exit fullscreen mode

Two honest checks of the guard itself: run it on a branch that only touches allowed paths and expect exit code 0; add a stray file such as README.md and expect exit code 1. If both hold, the guard does what the comment claims.

Leaf B — admit with a contract written first

Here you can verify, but only if the check exists before the agent starts. Write the failing test yourself, commit it, then hand over. This converts a subjective review into an exit code.

# tests/webhooks/test_retry_cap.py — illustrative; written before the change,
# expected to fail until the retry cap exists.
import pytest
from src.webhooks.sender import send_with_retries

def test_retry_attempts_are_bounded(monkeypatch):
    calls = {"n": 0}
    def fake_post(*args, **kwargs):
        calls["n"] += 1
        raise TimeoutError("simulated")
    monkeypatch.setattr("src.webhooks.sender._post", fake_post)
    with pytest.raises(TimeoutError):
        send_with_retries("https://example.invalid", max_retries=3)
    assert calls["n"] == 3
Enter fullscreen mode Exit fullscreen mode

example.invalid is a reserved domain, so the test cannot accidentally reach a real endpoint. The assertion is on the number of attempts, not on timing — timing assertions are the fastest way to build a flaky gate.

Leaf C — human-in-the-loop, draft only

The task is real work but fails the machine-check test, usually because "correct" is a design judgment, a visual result, or a policy call. Admit it, but change the output type: the agent produces a draft branch, and a person owns the merge.

A minimal rule set that is easy to enforce in review:

  • The agent's output goes to a branch prefixed draft/ and never to a protected branch.
  • The PR description must quote the acceptance command and its output, or state plainly that no command exists.
  • The reviewer's first question is "what would make this wrong and still compile?" — if there is no answer, the draft was not worth reviewing.

This leaf is where free-tier experimentation earns its keep, because the cost of a wrong answer is a closed PR, not an incident.

Leaf D — do not send

Reversibility fails, or the context boundary fails. Typical members: schema migrations, credential rotation, anything touching billing, infra/ directories, files under secrets/, and content you are not permitted to paste into a third-party service.

Keep the refusal executable rather than cultural:

# deny_paths.sh — illustrative; run before any agent invocation.
if git diff --name-only origin/main...HEAD | grep -qE '^(infra/|\.env|secrets/)'; then
  echo "BLOCKED: diff touches sensitive paths"
  exit 1
fi
echo "OK: no sensitive paths in diff"
Enter fullscreen mode Exit fullscreen mode

A denylist is a backstop, not a policy. If your repository relies on this script as the only protection for sensitive data, fix the repository layout first — the script cannot see what the agent reads before it writes.

Where a free tier fits, and what it does not fix

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The admission tree above needs somewhere to run the guard and the acceptance command, and it needs an agent to produce the diff in the first place. MonkeyCode is an open-source project whose operator-supplied description includes free model access and a free server option; this account's operator also cites an allowance on the order of ten million tokens. Treat all three as product claims rather than measurements — quotas, model availability, and hosting terms change, so confirm the current numbers in the project's own documentation before you plan a weekend around them.

The mechanical fit is narrow and worth stating precisely: put the verification step — check_scope.py, the contract test, the denylist check — on the free server so that agent branches and human branches pass through the same gate. Identical checks in two places is how "it worked on my laptop" gets eliminated. What a free tier does not do is shrink Leaf D, fix flaky tests, or make an unverifiable task verifiable. A larger allowance mostly buys you more attempts at tasks you have not scoped.

Limits and who should skip this

  • Flaky suites break the gate. If pytest tests/webhooks -q fails one run in five on main, verification latency is undefined and Leaf B collapses into Leaf C.
  • The allowlist is only as good as the glob. src/webhooks/*.py does not match src/webhooks/v2/, and fnmatch treats * more loosely than you may expect. Test the guard with the escape case, not just the happy path.
  • Regulated or customer data. If you cannot lawfully paste the context, the task belongs in Leaf D regardless of how clean the diff would be.
  • Performance and migration work. Needs runtime evidence the harness does not produce; send it to a person with a profiler.
  • Teams without a trusted test command. Build that first. An agent on top of an untrusted suite just produces faster unverified diffs.

A verification pass you can repeat

  1. Write three task cards: one that should land in Leaf A, one in Leaf B, one in Leaf D.
  2. Run check_scope.py against a clean branch and confirm exit 0; add an out-of-scope file and confirm exit 1.
  3. Run deny_paths.sh against a branch touching infra/ and confirm it blocks.
  4. Run the acceptance command on a branch with no change and confirm it passes — a gate that fails on main is a gate you will disable.
  5. Only then let an agent produce a diff, and record which leaf the task landed in.

If you try the project, keep the least glamorous piece: the task card and the scope guard. They are the parts that still work when the token budget runs out.

Top comments (0)