The Token Meter: A Hands-On Lab for AI Patch Economics

Picture a team that connected an AI coding assistant to its repository in January. By March the CI bill had grown, and nobody wanted to explain the new line item in the stand-up. The assistant itself was free. The tokens were not invisible; they were just unmeasured.

The busy season for AI coding hype is behind us. The useful season rewards teams that can answer one plain question before a generated patch lands: what did this cost, and did it pass the gate? Model names and token prices change every few weeks, so this article quotes neither. It builds a repeatable meter instead.

MonkeyCode is an open-source project that offers a free model access tier and a free server option for experiments like this. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The project advertises a 10-million-token free allowance on the model tier; treat that figure as a claim to re-verify on the day you run the lab, not as a contract. The free server is a small machine for low-traffic jobs, which is exactly the right size for the exercise below.

The artifact

This lab builds a four-stage pipeline: a deliberately broken function, a single model call that patches it, a verification gate, and a token ledger. By the end you have a cron job on a free server that produces one verified patch and one cost line every morning. That output is the meter.

The whole session fits in about ninety minutes. Each exercise gives the timing, the expected output, and the failure mode to watch for.

Exercise 1 — the broken function (15 minutes)

Start with a function that looks fine and is not.

# buggy.py
def apply_discount(items, rate):
    total = 0
    for item in items:
        total += item["price"] * item["qty"]
    return total - (total * rate)

Three defects live in those five lines. rate is never validated, so a rate like 1.5 makes the total negative. A negative quantity turns the discount into a surcharge. And a caller who applies the function twice stacks discounts silently. Ask students to fix it by hand first and write down how long it takes. That number is the human baseline the model has to beat.

Exercise 2 — ask once, do not argue (20 minutes)

Conversational back and forth is the fastest way to burn a free allowance. One prompt, one response, one patch file. The script below is an adapter; read the endpoint docs for the free tier before running it, because field names change between providers.

# ask_once.py
import os
import requests

ENDPOINT = os.environ["MC_ENDPOINT"]
HEADERS = {"Authorization": f"Bearer {os.environ['MC_TOKEN']}"}

def ask(prompt: str) -> dict:
    resp = requests.post(ENDPOINT, headers=HEADERS,
                         json={"prompt": prompt, "max_tokens": 512},
                         timeout=120)
    resp.raise_for_status()
    return resp.json()   # expect text plus a usage object

PROMPT = """Fix the bugs in this function and keep the signature
apply_discount(items, rate). Return only the corrected code block."""

with open("buggy.py") as f:
    PROMPT += f"\n```
{% endraw %}
python\n{f.read()}\n
{% raw %}
```"

result = ask(PROMPT)
print(result.get("text", ""))
print("usage:", result.get("usage"))

Save the model answer as patched.py. Then stop. No second prompt, no "please also add comments" follow-up. The point of the exercise is to measure a minimal realistic call, not to negotiate with the model.

Exercise 3 — the verification gate (30 minutes)

The gate decides whether the patch is worth anything. Two commands matter: ruff check patched.py for static sanity and pytest -q test_discount.py -x for behavior. Define the behavior first.

# test_discount.py
import pytest
from patched import apply_discount

def test_discount_basic():
    items = [{"price": 100, "qty": 2}]
    assert apply_discount(items, 0.1) == pytest.approx(180)

def test_rate_bounds():
    items = [{"price": 100, "qty": 1}]
    for bad in (-0.5, 0, 1.5, 2.0):
        with pytest.raises(ValueError):
            apply_discount(items, bad)

def test_negative_quantity():
    items = [{"price": 100, "qty": -2}]
    with pytest.raises(ValueError):
        apply_discount(items, 0.1)

A short runner combines the two signals into one exit code.

# verify_patch.py
import subprocess

def run(cmd):
    proc = subprocess.run(cmd, capture_output=True, text=True)
    return proc.returncode, (proc.stdout + proc.stderr).strip()

ruff_code, ruff_out = run(["ruff", "check", "patched.py"])
test_code, test_out = run(["pytest", "-q", "test_discount.py", "-x"])
print("ruff:", "pass" if ruff_code == 0 else "fail")
print("pytest:", "pass" if test_code == 0 else "fail")

Most groups discover something useful here: the model often fixes the negative-quantity case and leaves rate unchecked, or the other way around. The gate exists precisely because the patch looks confident. Confidence is not a test result.

Exercise 4 — the token ledger (10 minutes)

Now attach a price to the result. Free allowances are not infinite, and a ledger makes that concrete.

# ledger.py
FREE_ALLOWANCE = 10_000_000  # published claim for MonkeyCode free tier, re-verify

def report(result: dict) -> None:
    usage = result.get("usage", {})
    used = usage.get("prompt_tokens", 0) + usage.get("completion_tokens", 0)
    share = used / FREE_ALLOWANCE
    print(f"tokens this run: {used} ({share:.4%} of the free allowance)")
    print(f"estimated runs per budget: {FREE_ALLOWANCE // max(used, 1)}")

On this tiny task the numbers look great and the estimate is meaningless. Long context, multiple files, or an agentic loop can consume a day of allowance in minutes. The ledger is a thermometer, not a prophecy.

Exercise 5 — schedule the meter (15 minutes)

The last step moves the pipeline onto the free server option. A cron entry runs the ask-verify-report cycle once a day and appends the result to a flat file.

# crontab -e
0 7 * * * cd /opt/token-meter && export MC_ENDPOINT=... MC_TOKEN=... && \
  python ask_once.py > patched.py && \
  python verify_patch.py >> ledger.log 2>&1

A low-traffic morning job is a legitimate use of a free server. It is not a production deployment. It is a lab bench with a power cord.

Limitations

Three caveats keep this honest. First, the free allowance and server are marketing claims with variable terms; check the project docs on the day you run the lab and expect them to change. Second, latency and rate limits on free endpoints are not a contract, so anything time-sensitive belongs behind a paid or self-hosted path. Third, do not send proprietary code to any third-party endpoint, free or not; this entire workflow assumes a throwaway function you are allowed to share.

Teams that need a service-level agreement, work with regulated data, or route production CI through external endpoints should skip this pattern. The same goes for anyone who treats the ledger as a reason to stop reviewing. The meter tells you what a patch costs; it does not tell you whether the patch is right. That judgment still needs a human, which is a feature, not a bug.

A final note

The lab is deliberately boring. No leaderboards, no model comparisons, no clever prompts. A broken function, one call, a gate, and a number on a log file. Run it once on MonkeyCode free tier if the claims match your constraints, and keep the ledger either way. If you try it, post your tokens-per-verified-patch number in the comments; that shared baseline is worth more than any benchmark chart.