Should Your Squad Keep a Free AI Lane When Green Tests Stop Meaning Done? Use This False-Done Gate

Nine of the last twelve AI-touched pull requests on the board were green. Two of them still shipped a path no test executed. That ratio can reverse a spend decision faster than any token invoice.

I'm treating this as a composite squad, not a war story. If your numbers differ, the card still works. Would you keep the free lane, isolate the sensitive work, or exit generation on that surface entirely?

Most teams start in the wrong meeting. They argue free versus paid, or free versus self-hosted, before they can say what "done" meant last week. I don't. I want a field that can fail.

The standup that should have been a stop

Picture Tuesday. A service owner pastes a diff that reads complete. CI is green. The reviewer asks one question: which test hits the new branch?

Silence. Then a follow-up commit. Then another. The original PR was not unfinished in the Git sense. It was unfinished in the evidence sense. That is the leak I score.

Throughput looked fine. Review comments looked polite. The team still could not reproduce the claim without the author in the room. If that is your week, buying a bigger lane will not fix it. It will only make false-done cheaper.

Stop pricing the lane before you define false-done

A free coding lane is a capacity choice. It is also a governance choice. Mixing those two debates is how platform leads buy isolation they did not need, or keep a shared lane they cannot defend.

I use four outcomes, not two:

1. Keep the free lane for that surface.
2. Split: free for scratch, isolated for anything with data, secrets, or production blast radius.
3. Exit generation on that surface until evidence holds.
4. Park the decision with an expiry, not a vibe.

Paid and self-hosted sit inside Split or Exit. They are not a personality. They are a response to a gate you can fail.

This scorecard is a conversation tool, not objective truth. If nobody on the squad can name the variable that would reverse Keep, you are not deciding. You are decorating a preference.

Define the variables before anyone scores

I will not score a lane with undefined nouns. Write these on the wiki first.

• False-done rate (FDR). Share of AI-touched PRs in the sample that merged or were marked ready, then needed a follow-up because a generated path was untested, unreproducible, or unexplained. Count the PR, not the commit spam.
• Evidence lag (EL). Hours from "looks done" until a named owner can reproduce the claim without the author. If the owner is "the team," EL is infinite.
• Boundedness (B). Share of generated branches, handlers, or jobs that a test, a recorded manual reproduction, or a failing fixture actually hits.
• Isolation need (I). Binary for the surface: does policy forbid prompts, code, or tool output on a shared free server? If yes, Keep is already illegal. Do not score taste.
• Switch cost (SC). Hours to stand up an isolated lane (vendor or self-host) and move the evidence template with it. Exclude catalogue shopping. Include review, secrets, and rollback notes.
• Sample window (W). Consecutive calendar days, not "busy week." I default to 14. Shorter windows lie. Longer windows hide the fire.

Tag AI-touched PRs in the description or a label. If you cannot tag them, you cannot compute FDR. That is a process bug, not a model bug.

The 4-gate false-done card

Copy this. Fill it in a doc, not in Slack.

# false-done-gate.v1.yaml
# Conversation tool. Not a procurement score.
surface: payments-webhooks  # one surface per card
owner: em-payments          # named human, not a rotation
window_days: 14
expiry: 2026-10-17          # 28 days from decision, not "next quarter"
sample:
  ai_touched_prs: 12
  false_done_prs: 2
  fdr: 0.167                # 2 / 12
  evidence_lag_hours_p50: 6
  boundedness: 0.75         # 9 / 12 had a hit on the new path
  isolation_need: false     # policy check, not preference
  switch_cost_hours: 16
gates:
  fdr_max: 0.20             # fail closed if FDR > 20%
  el_max_hours: 8           # fail if p50 evidence lag > 8h
  boundedness_min: 0.70     # fail if < 70% of generated paths are hit
  isolation_must_pass: true
outcome: keep               # keep | split | exit | park
exit_criteria:
  - "FDR > 0.20 in any completed window"
  - "EL p50 > 8h for two windows"
  - "isolation_need flips to true"
reversal_question: "Which single field, if it flipped, would kill Keep?"

Hard reading order. Do not average the fields.

1. Isolation gate. If I is true, Keep is forbidden. Split or Exit. Stop scoring FDR as a way to smuggle a policy exception.
2. FDR gate. If FDR is above your max, you do not have a capacity problem. You have an evidence problem. Exit generation on that surface, or Split so the messy work cannot hide in the same queue as production.
3. Boundedness + lag gate. If B is below the floor or EL blows the cap, Keep only if the owner can name a two-week fix. Otherwise Park with expiry.
4. Switch-cost gate. Buy or self-host only when gates 1–3 already demand isolation and SC is lower than the evidence debt you are already paying. Evidence debt, roughly: false_done_prs * (EL hours + re-review hours).

Would a cheaper token price reverse Exit? It should not. Price does not create a test hit.

Worked example, then I break it

Illustrative numbers. Not a benchmark. Not a customer.

A payments-webhooks squad, 14-day window, 12 AI-touched PRs, 2 false-done. FDR = 16.7%. Evidence lag p50 = 6 hours. Boundedness = 75%. Isolation need = false. Switch cost to an isolated lane = 16 hours. Re-review on each false-done ≈ 3 hours.

Evidence debt in the window: 2 * (6 + 3) = 18 hours. Switch cost is 16 hours. Close, right? Not yet. Isolation need is false, FDR is under 20%, lag is under 8, boundedness is over 70%. Keep. The 16-hour migrate would buy a feeling, not a gate.

Now flip one field. Same squad, isolation need becomes true because webhook payloads include customer identifiers that cannot sit on a shared free server. Outcome is Split on the same afternoon. FDR did not move. Policy did.

Flip a different field. Isolation stays false, but false-done PRs are 4 of 12. FDR = 33%. Evidence debt: 4 * 9 = 36 hours. Exit generation on that surface until boundedness recovers. Do not "upgrade" the lane so people can miss tests faster.

Sensitivity I actually walk:

Knob you change	Keep still holds?	What I ask next
FDR 17% → 25%	No	Which paths were unhit? Exit or Split, not spend.
EL 6h → 12h	No	Name an owner. If you cannot, Park.
B 75% → 60%	No	Add reproduction notes before any vendor talk.
I false → true	No	Split today. Free scratch can remain.
SC 16h → 40h	Yes, if I is false	Do not migrate to soothe anxiety.
W 14d → 5d	Invalid	Refuse the card. Sample is too short.

If FDR is the field that reverses you, you were never in a build-versus-buy meeting. You were in a test-design meeting wearing a procurement hat.

Commands to count false-done without inventing dashboards

I want the sample in git, not in memory. Label this as a method you can copy. It is not production telemetry, and it will lie if you skip labels.

# Last 14 days of merged PRs that someone tagged as AI-touched.
# Requires GitHub CLI. Adjust the label to whatever you actually use.
gh pr list --state merged --search "label:ai-touched merged:>=2026-09-05" \
  --json number,title,mergedAt,url,labels > /tmp/ai-prs.json

# Follow-up commits on the same paths in 72h often signal false-done.
# This is a heuristic. Read the PR. Do not let the script merge for you.
git log --since='14 days ago' --pretty=format:'%h %ad %s' --date=short -- '*.py' '*.ts'

A tiny scorer you can run after you classify each PR by hand. Classification is the work. The script only refuses a missing denominator.

# proposal: score a window you already classified
# fdr = false_done / ai_touched; fail closed on empty sample

def false_done_rate(ai_touched: int, false_done: int) -> float:
    if ai_touched <= 0:
        raise ValueError("empty sample; do not score")
    if false_done < 0 or false_done > ai_touched:
        raise ValueError("false_done must sit inside ai_touched")
    return false_done / ai_touched

def gate(fdr: float, fdr_max: float = 0.20) -> str:
    return "fail" if fdr > fdr_max else "pass"

if __name__ == "__main__":
    fdr = false_done_rate(ai_touched=12, false_done=2)
    print({"fdr": round(fdr, 3), "gate": gate(fdr)})

If you will not classify PRs by hand for two weeks, do not run a free lane at production blast radius. You cannot see FDR. You are guessing.

A PR template that makes boundedness visible

I want the evidence in the PR, not in a hallway. Paste this under the diff.

## Evidence (AI-touched)
- [ ] Generated paths listed (file + function / job name)
- [ ] Test or fixture that hits each path, *or* a recorded reproduction
- [ ] Named owner who can re-run that claim without the author
- [ ] Data / secrets in the prompt: none | redacted | blocked (isolation)
- [ ] Follow-up PR expected? no | yes, ticket id

If the checklist is empty and CI is green, that is a false-done candidate. Green is not done. Green is "the tests you already had did not scream."

Where a free model lane still earns Keep

Some squads need a scratch lane to measure FDR before they pay for isolation. That is a real job. It is not a brand preference.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode is an open-source coding tool with free model access and a free server option. I treat those as availability facts, not as a bake-off. I will not invent model names, quotas, hardware, or permanence. If a free shared server cannot hold your prompts, Isolation need is already true. The product does not get a vote.

Use a free lane when all of this is true:

• Isolation need is false for that surface.
• You will tag AI-touched PRs and run the 14-day window.
• The owner can fail Keep.
• Scratch work and production work are not one queue.

If you want a place to run the card without buying capacity first, that free model access and free server option are one way to do the measurement. The card still owns the decision.

Owner, expiry, exit

• Owner. Engineering manager or platform lead for the surface. Not "AI champions." Champions do not revoke lanes.
• Expiry. 28 days, written on the YAML. If the card is not re-scored, the lane Parks. Park is not Keep with worse lighting.
• Exit. Any completed window with FDR above max, EL above cap for two windows, or Isolation need flipping to true. Exit means generation stops on that surface until boundedness recovers. It does not mean a angry all-hands.
• Archive. Keep the YAML. Future you will try to remember why you Split. Future you is a liar.

Who should not use this approach

Do not run a free shared lane, including a free server, if legal already forbids prompts or code leaving your boundary. Do not use this card to launder that exception.

Do not use it if you refuse to label AI-touched PRs. The math needs a denominator.

Do not use it as a model beauty contest. There are no latency tables here on purpose. If your question is "which weights are hottest this week," you are in a different article.

Do not use it for a two-person hobby repo with no reviewer. FDR collapses into self-grade. Be honest and skip the theater.

Limitations, said plainly

FDR depends on how strict you are when you classify follow-ups. Two reviewers will disagree on the edge. Write the rule: unhit path, unreproducible claim, or missing owner. Not "I did not like the style."

A 14-day window can miss a rare surface. A 90-day window can hide a bad week under a good month. If the blast radius is high, shorten the decision loop, not the sample until it flatters you.

Switch cost is local. My 16-hour example is illustrative. Your procurement, your SSO, your secret broker. Fill the number. Do not borrow mine.

Free lanes change. Availability is not a contract I can sign for you. Re-score when the lane you measured is not the lane you have.

And tests can be as outdated as the generation they fail to bound. If AI output routinely steps around the assertions you still worship, FDR will look fine while production will not. Boundedness is about hitting the new path, not collecting more of the old green.

The question that actually closes the meeting

I do not care whether you "believe in" a free lane. Belief does not reverse.

Which field on your card, if it flipped this week, would kill Keep: FDR, evidence lag, boundedness, or isolation need? If you cannot name it, you are not choosing. You are hoping the next green build means done. It might not.