Measure Rework Before You Add a Teammate to the AI Bench

Free-model benches feel free until rework shows up. A two-person team absorbs sloppy output through slack. A third person turns that slack into a queue.

This playbook adds one measurement step before headcount. It runs for a single week on git history. The cost is a commit convention and a small script.

The bench this assumes

The setup is a shared repo, free model access for drafting, and the free server option for the test loop. Both availability claims here are operator-supplied, not benchmarks. Limits and hardware change without notice. Check the product page on the day you commit a team to them.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

A bench is only shared if work is visible. Output that never lands in commits cannot be measured. So the first move is a naming convention, not a tool.

Step one: make AI work visible in git

Tag every assist-assisted commit with a scope. Use feat(ai):, fix(ai):, test(ai):, or refactor(ai):. Human-only work keeps its normal scope.

Rework gets its own prefixes. A correction inside the same day is fix(ai):. A rollback is revert:. Untagged fixups are invisible to every later measurement.

feat(ai): add pagination to invoice list
fix(ai): correct off-by-one in page cursor
test(ai): cover empty page boundary
revert: feat(ai): add pagination to invoice list

The convention only survives if review enforces it. A trailer does that cheaply. Require one sign-off line per AI-tagged commit in the pull request history.

# Fail the PR when AI-tagged commits arrive without a checker.
set -e
missing=$(git log --format=%B origin/main..HEAD \
  | awk 'BEGIN{RS=""} /commit|AI/ && !/Checked-by:/ {c++} END{print c+0}')
[ "$missing" -eq 0 ] || { echo "$missing AI commit(s) need Checked-by:"; exit 1; }

That gate is crude on purpose. It is readable in a code review and breaks loudly. Nobody needs a parser to trust it.

Step two: run the rework clock

The script below reads one week of history. It counts AI-tagged commits and the subset that were later corrected or reverted. It prints a share, not a verdict.

#!/usr/bin/env python3
# rework_clock.py - share of AI-tagged commits that needed correction in one week.
import datetime as dt
import re
import subprocess

WINDOW_DAYS = 7
AI_TAG = re.compile(r"^(feat|fix|refactor|test)\(ai\):|^ai:", re.I)
REWORK = re.compile(r"^fix\(ai\):|^revert:|^fixup!|^squash!", re.I)


def git(*args):
    out = subprocess.run(["git", *args], capture_output=True, text=True, check=True)
    return out.stdout


since = (dt.datetime.now(dt.timezone.utc) - dt.timedelta(days=WINDOW_DAYS)).isoformat()
rows = git("log", f"--since={since}", "--pretty=%H%x1f%s").splitlines()

ai_total = ai_rework = 0
for row in rows:
    sha, subject = row.split("\x1f", 1)
    if AI_TAG.search(subject):
        ai_total += 1
        if REWORK.search(subject):
            ai_rework += 1

share = "n/a" if ai_total == 0 else f"{ai_rework / ai_total:.0%}"
print(f"ai_commits={ai_total} rework_commits={ai_rework} rework_share={share}")

Run it on a Friday and append the result to the bench log. An illustrative output looks like this, not a measurement of any real bench:

$ python3 rework_clock.py
ai_commits=37 rework_commits=11 rework_share=30%

One number is noise. Four weekly numbers show a direction. Compare the same week against the previous one, never against a different project.

Sometimes the useful signal is smaller than rework. Count touched files per AI commit as a size proxy:

git log --since="7 days ago" --grep="(ai)" --numstat --pretty=format:%h \
  | awk 'NF==3 {f[prev]+=1} NF==1 {prev=$1} END {for (k in f) print k, f[k]}' | sort -k2 -nr | head

Wide commits and a high rework share usually travel together. Fix the brief before adding a person.

Step three: read the number together

The clock exists to support one decision. Use the share as a brake, not a scoreboard.

Rework share	Reading	Next move
Under 15%	The bench absorbs corrections fine	Add the third person and keep the convention
15% to 30%	The brief is thin, not the people	Rewrite the intake brief, re-measure one week
Over 30%	Review is the bottleneck	Freeze headcount, use the free server for the test loop only

Those cutoffs are starting points. A team of two with a shaky test suite will run hot. A team with strong CI will run cold.

The one-page run for the wiki

Paste this block into the team wiki and date it. Three roles, one week, one number.

BENCH REWORK CLOCK - week of YYYY-MM-DD

Roles
  Driver   drafts with the free model tier, tags every commit
  Checker  reviews AI-tagged commits, adds Checked-by: trailer
  Scribe   runs rework_clock.py Friday, posts the number

Monday    Driver confirms the commit convention in the repo README
Daily     Checker blocks untrailered AI commits at review
Friday    Scribe runs the clock, appends the row to the bench log
Friday    All three read the share, pick one next move from the table
Handoff   Scribe posts role names and the week number for the next run

Stop rule
  Two weeks above 30% pauses new bench members until the brief changes.

The handoff line matters more than the script. Rotating scribes keeps the number honest and stops one person from owning the narrative.

Where this breaks

Commit messages are self-reported. A team that squashes pull requests hides its rework history. Measure at the PR level instead, or accept a blind spot.

The clock counts corrections, not reading time. Review effort is the real cost and stays invisible here. Pair it with a simple timer if that cost matters.

Small samples mislead. Twenty commits in a week cannot separate a bad brief from a bad Tuesday. Run four weeks before you act on a percentage.

Skip this approach if you work alone. Skip it if your team has no commit convention and no appetite for one. Skip it if AI output is merged by a bot without human sign-off, because the trailer gate has nothing to check.

A closing note

Cheap drafting is easy to buy and hard to amortise. The rework clock is the cheapest way I know to ask whether the bench is paying for itself. If you want to test the setup described here, start with MonkeyCode's free model access and its free server option, and confirm the current limits on the product page before your first Monday.