A Triage Agent on a Token Budget: A Free-Tier Case Study in Tool-Call Accounting

Last month, a small open-source repo I help maintain crossed 40 new issues in a week. Triage took four hours of manual work: reading, labeling, prioritizing, replying. I wanted an agent to do the first pass. The catch: the project has no cloud credits, no GPU, and no budget line for AI.

Zero dollars was not a slogan. It was the constraint.

Background

The repo receives a mix of bug reports, feature requests, and setup questions. Most issues are short, but a few contain long stack traces and logs. The workload is bursty: quiet for days, then 10 issues overnight.

The obvious path was a hosted agent platform. The obvious path was also paid. I chose a different constraint: MonkeyCode's free model access and its free server option. Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode is an open-source project; at the time of writing, it advertises a 10-million-token free allowance and a free server tier. Exact limits change, so verify them in the project docs before you rely on them.

The repo details are a worked example. The harness and the accounting method are the reproducible parts.

Goal

I wrote three success criteria before writing any agent code:

1. Triage 50 issues per week within the token allowance.
2. Match my manual labels on at least 85% of a 30-issue labeled set.
3. Run unattended on the free server, with a nightly cron job.

The fourth, implicit criterion was the real one: know exactly where every token went.

Implementation

Agent design

The agent has three tools and one guardrail:

• list_issues — fetch open issues, one page at a time.
• get_issue — fetch the full body of a single issue.
• classify — return a label, a confidence score, and a draft reply.
• Guardrail: a maximum of 5 tool calls per issue, enforced in the loop.

Every tool call is logged before the agent is allowed to continue. That logging is the artifact this case study is about.

The budget ledger

Here is the core of the harness. It records one JSON line per tool call:

# tool_budget_ledger.py
import json
from dataclasses import dataclass, asdict

@dataclass
class ToolCall:
    ts: float
    agent_step: int
    tool: str
    input_tokens: int
    output_tokens: int
    status: str  # ok | error | skipped
    latency_ms: int

def append_call(run_id: str, call: ToolCall) -> None:
    path = f"ledgers/{run_id}.jsonl"
    with open(path, "a", encoding="utf-8") as f:
        f.write(json.dumps(asdict(call)) + "\n")

The summarizer turns the ledger into a per-tool cost table:

def summarize(run_id: str) -> dict:
    totals = {}
    with open(f"ledgers/{run_id}.jsonl", encoding="utf-8") as f:
        for line in f:
            c = json.loads(line)
            t = totals.setdefault(
                c["tool"],
                {"calls": 0, "input": 0, "output": 0, "errors": 0},
            )
            t["calls"] += 1
            t["input"] += c["input_tokens"]
            t["output"] += c["output_tokens"]
            t["errors"] += int(c["status"] == "error")
    return totals

The budget guard lives in the loop, not in the wallet:

BUDGET = 10_000_000  # free-tier allowance, operator-reported; verify in docs
RESERVE = 0.8        # stop at 80% to leave headroom

def within_budget(spent: int) -> bool:
    return spent < BUDGET * RESERVE

The guard is deliberately pessimistic. An agent that stops at 80% is annoying. An agent that stops at 100% mid-run leaves 20 issues unlabeled and no tokens to fix it.

Where the tokens actually went

I validated the harness on a synthetic 10-issue pilot before spending real quota. The numbers below are illustrative; the accounting logic is identical to what the harness records against any provider.

Tool	Calls	Input tokens	Output tokens	Share of spend
list_issues	6	4,200	900	6%
get_issue	10	61,400	1,800	74%
classify	10	9,800	2,100	14%
post_comment	4	3,100	800	6%

The finding was not subtle. get_issue consumed 74% of the budget, and the reason is a property of tool-calling agents: the tool result is re-sent to the model on every subsequent step. A 2,000-token issue body costs 2,000 tokens the first time the model sees it, then 2,000 again on the next step, then 2,000 again.

Three fixes came out of that one table:

1. Fetch each issue body exactly once and cache it in the run context.
2. Truncate bodies to the first 800 characters for classification; stack traces rarely change the label.
3. Collapse get_issue and classify into one step so the body is sent exactly once.

After the fixes, the pilot's per-issue cost dropped by roughly half, and the projected weekly spend fit comfortably inside the allowance.

Decision table

The approach is not universal. Here is the table I used to decide whether a free-tier triage agent makes sense for a workload:

Workload	Free-tier fit	Why
Fewer than 200 issues/week, bodies under 1k tokens	Yes	Fits the allowance with headroom
Nightly batch triage	Yes	Cron-friendly, latency-tolerant
Long-document review (50k-token files)	No	One file can consume most of the budget
Real-time responses under 2 seconds	No	Free servers and cold starts do not mix with SLAs
Compliance-sensitive data	No	Verify data handling before sending anything

Lessons learned

1. Measure before optimizing. The ledger showed the cost driver was tool-result resends, not the model call itself. Without the per-tool table, I would have blamed the model and switched providers for no reason.
2. Budget caps belong in the loop. A hard stop at 80% of the allowance turns a potential billing surprise into a scheduled, visible event.
3. Free tiers are a design constraint, not a handicap. The 10M limit forced truncation, caching, and single-pass design. The agent got simpler and cheaper — and the simpler version was easier to test.
4. Log everything, even in a pilot. The synthetic run cost nothing and exposed the 74% problem before it touched real quota.

Who should not use this

Teams with latency SLAs, compliance requirements, or document-heavy workloads should not build on a free server and a token allowance. The harness is still useful there, but as a measurement tool, not as a production platform. Measure first, then decide what to pay for.

Run the experiment

The harness is about 120 lines and fits in one file. If you want to know where your own agent's tokens go, start with the ledger before you start with a bigger model. MonkeyCode's free tier is a reasonable place to run that experiment; the project docs list the current limits. Run the pilot, read the table, and let the numbers decide.