Your Green Toy Eval Cannot Pick the Lane. Use This Review-Debt Gate

Here is the meeting I reconstruct when a platform lead says the demo already decided the vendor.

An engineering manager has a free coding lane. It drafts a small internal page. A toy eval written before lunch comes back green. Someone asks the question that sounds frugal and is actually reckless. If this lane is clean, why are we still pricing a paid seat or a box in our own network?

I would ask a different question. What did that eval refuse to look at?

The quiet usually answers it. Happy-path JSON passed. Auth did not run. Migration order did not run. The files that page on-call never opened. A green toy eval is a mood. It is not a lane decision.

That is the tradeoff that can reverse the call. Not market excitement. Not a token sticker. Review debt, custody, and the week the free option changes.

Name the choice before you name a product

Three options sit on the table. Keep generating on free model access plus a free server. Move the same workflow to a paid hosted lane. Or self-host.

Which one survives the next month of real diffs? If you cannot say what would flip you, you are not deciding. You are shopping.

I am using MonkeyCode as the free-lane column in this comparison. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The outreach describes free model access and a free server option, and it positions the project as open source.

I have not verified the license, the repo contents, quotas, model names, hardware, uptime, or how long either offer lasts. If a chat brief waved a round token figure at you, do not paste it into a budget. Copy the live number, and the date you read it, from the project page. A sticker is not capacity you own.

If the page and this article disagree, the page wins. Open source is not a support plan. Read the license before you fork, and confirm what "open" does not include.

The unit I will actually price

I will not price this on seats. I will not price it on a token pool. I will not price it on one green run.

I price reviewer-minutes on defects a human still has to catch. Call that review debt. If the lane saves drafting time and then spends it back in review, you do not have free capacity. You have a queue with a costume.

Does your toy eval even see the files your reviewer rewrites? If it does not, your escape rate is a guess wearing a checkmark.

Variables, defined before the argument

• D — drafts per week a human must review. Diffs, not chat turns.
• M — reviewer minutes per draft, including rewrites. Time it. Do not recall it.
• E — escape rate. Share of drafts where review finds a defect the toy eval missed. From 0 to 1.
• C — loaded cost of one reviewer minute, in your currency.
• S — drafting minutes the lane actually saves per draft, on your work.
• K — extra minutes per draft if you self-host. Patching, babysitting, key rotation. Measured, or marked unknown.
• P — paid hosted cost per draft, fully loaded. Seats spread across D, plus overage you have already seen.
• B — custody block. 1 if the diff may not leave your boundary. 0 if it may.

Weekly review debt is RD = D * M * E * C. Weekly drafting value is DV = D * S * C.

Free-lane net, ignoring the chance the offer vanishes, is NF = DV - RD.

Paid net uses its own escape rate, not a hoped-for one: NP = D * S * C - D * M * E_paid * C - D * P.

Self-host net is NS = D * S * C - D * M * E_self * C - D * K * C.

You do not get to assume E falls because you paid, or because you rented a box. If you have not sampled those lanes, set E_paid and E_self equal to E. Then paid and self-host must win on custody, support, or continuity. Not on a fantasy quality jump.

A filled sheet that is not a customer

These numbers are hypothetical. I made them up so the arithmetic is visible. They are not a benchmark. They are not MonkeyCode results. They are not your team.

D = 30. M = 18. E = 0.25. C = 1.4. S = 12. P = 6. K = 8. B = 0 only while the sample stays off customer data.

Free lane: RD = 30 * 18 * 0.25 * 1.4 = 189. DV = 30 * 12 * 1.4 = 504. NF = 315.

Paid, same escape rate: NP = 504 - 189 - 180 = 135.

Self-host, same escape rate: NS = 504 - 189 - 336 = -21.

On this invented board, the free lane wins the week, and self-host loses. Thin win. Do not laminate it.

Sensitivity, the part that should make you nervous

Move only E on the free lane to 0.55. That is what I expect when the eval never opened auth or migrations.

RD = 30 * 18 * 0.55 * 1.4 = 415.8. NF = 88.2. Still positive. Barely.

Now the reviewer is rewriting, so S drops to 6. DV = 252. NF = 252 - 415.8 = -163.8.

The free lane just went negative. No sticker caused that. Reviewer minutes did. Would your lead still call it free?

Flip B to 1 and stop. A pretty NF does not outrank a custody rule. Free model access that sends code off-boundary is a no, even when the weekly math looks kind.

What about the week the free server changes? I will not invent an uptime number to decorate this. Price a stuck week as D * M * C of drafts with nowhere to land, and ask if you can absorb it. If you cannot, the free column was never a default. It was a pilot with a calendar.

Hard gates I will not let you average away

1. Custody. B = 1 means the free hosted lane is out. Compare a contracted paid lane with self-host. Read the contract. Do not guess the data terms.
2. Owner. One named person. Not a channel reaction.
3. Expiry. This sheet dies in 14 days, or on the day the project page changes the free terms. Whichever comes first.
4. Toy-eval scope. If the eval skips one failure path from last month's incident review, it cannot justify "stay."
5. Exit. Leave the free lane when NF is negative on two weekly samples, or when one escaped defect touches a paging path.
6. No silent default. The free lane is not the only editor for new hires until the owner re-signs.

A scorecard is a conversation tool. It is not objective truth. If two leads disagree on E, argue about the sample. Do not argue about the logo.

Questions I ask out loud

• Who gets paged if this diff is wrong?
• Which file did the toy eval never open?
• If the free server is gone on Thursday, where do these drafts land?
• Are you about to treat a smooth surface as a reviewed change?

If the room cannot answer, you do not have a pilot. You have a demo that escaped.

Fit, in one glance

Question	Free model access + free server	Paid hosted	Self-host
When it fits	B = 0, NF > 0, owner named, expiry written	You need support or terms you can cite	B = 1, or you already run the stack and K is measured
When it fails	Terms can move and you have no exit week	D * P eats the review-debt gap	K is a guess you are treating as a fact
What you owe	A 10-draft sample and a boundary check	A premium you have seen, not a list price you liked	Patch owner, key owner, and a rollback path

This gate also ignores prompt retention, output license, and whatever counsel already forbade. If legal has a written no, the spreadsheet does not get a vote.

Run the arithmetic before the meeting

This script is a proposal. I have not executed it against a live product. Change every input. If you paste my hypotheticals into a slide, you are the problem.

# review_debt_gate.py
# Proposal only. Hypothetical inputs. Not a benchmark.

def weekly(d, m, e, c, s, extra):
    review_debt = d * m * e * c
    draft_value = d * s * c
    return review_debt, draft_value, draft_value - review_debt - extra

def main():
    d, m, c, s = 30, 18, 1.4, 12
    e_free = e_paid = e_self = 0.25
    paid_premium, self_host_minutes = 6.0, 8.0
    custody_block = False

    rows = {
        'free': weekly(d, m, e_free, c, s, 0),
        'paid': weekly(d, m, e_paid, c, s, d * paid_premium),
        'self_host': weekly(d, m, e_self, c, s, d * self_host_minutes * c),
    }
    for name, (rd, dv, net) in rows.items():
        print(f'{name}: debt={rd:.1f} value={dv:.1f} net={net:.1f}')

    if custody_block:
        print('GATE: free hosted is blocked. Compare paid vs self-host only.')
        return
    winner = max(rows, key=lambda n: rows[n][2])
    print(f'Arithmetic pick: {winner}. Re-check gates before you tell the squad.')

if __name__ == '__main__':
    main()

python3 review_debt_gate.py

Then break the free lane on purpose. This is the sensitivity I want on a whiteboard, not in a keynote.

python3 - <<'PY'
d, m, c, s = 30, 18, 1.4, 12
for step in range(1, 8):
    e = step / 10
    net = d * s * c - d * m * e * c
    print(f'E={e:.1f} free_net={net:.1f}')
PY

Where free_net crosses zero, stop debating vendors. Pull ten real diffs. A half hour of notes beats another slide.

Record the decision so it can expire:

date -u +%F > lane-decision-date.txt
printf 'owner=%s\nexpiry_days=14\ncustody_block=%s\n' "$OWNER" "$B" >> lane-decision-date.txt

Sample ten drafts, skip the pretty one

For each draft, four lines. Did a human touch auth, data, or migrations? Minutes spent, rewrite included. Would the toy eval have failed it? Could the diff have left the boundary?

# drafts.tsv columns: minutes, escaped(0/1), boundary(0/1)
awk -F'\t' 'NR>1 {n++; mins+=$1; esc+=$2; b+=$3} END {
  printf "n=%d avg_min=%.1f escape=%.2f boundary_share=%.2f\n", n, mins/n, esc/n, b/n
}' drafts.tsv

That is an average, not a median. I know. It is still enough to stop a meeting that wants to standardize a demo. If boundary_share is not zero, the free hosted option is already on probation.

Who should not use this gate? A solo maintainer with no reviewer. The formula assumes a human queue. A team that already has one contracted lane and no appetite for a second path. Anyone who wants a blessing for a number they have not timed. And anyone about to point a free server at production CI.

A free server option is a pilot host, not your build farm, until you have an exit you have rehearsed. Would you bet a Thursday incident on a host you do not control? Then it is not the default.

What I am not claiming

I am not claiming MonkeyCode is faster, cheaper in outcome, or more accurate than a paid lane or a self-hosted stack. I did not run that test. I am not naming models, because I will not invent them. I am not locking a token allowance into this article, because an allowance I have not re-read today is already stale.

Use the free column when the work is non-production, the boundary check is clean, you have an owner, and you have an expiry. That is a sane way to learn whether drafting time is real before you buy seats or stand up hardware.

Do not use it for customer data, paging paths, or a hiring exercise you will later pretend was production code. A surface that looks finished is how review debt hides. Your on-call does not grade the demo. It grades the diff.

Which input flips your call: escape rate, reviewer minutes, or the custody bit? If none of them move, you are shopping. Date the sheet, name the owner, and throw it out when it expires. If you try a free model lane or a free server before that date, copy the live terms that morning instead of trusting this page, and keep the lane off paging paths until the second weekly sample.

Comments

[Alex Shev]

Making review debt the unit of comparison is a strong way to prevent a green happy-path demo from deciding an operating lane. The gate would be even more actionable if it required a small holdout of production-shaped diffs - auth, migrations, and on-call files - and recorded both escape rate and reviewer minutes before any cost comparison is accepted.