The Hint Was Wrong on Purpose: A Bootcamp Lab on Disagree-Before-Retry

The second click is not a strategy. I grade two decisions before I look at a green test: did you reject the bad hint, and did you write why another model call was worth the lab budget?

A fluent retry can hide a lazy read. If the only change between runs is hope, what exactly would I be scoring?

This is a lab proposal for a bootcamp cohort, not a write-up of a class I already scored. The checker below is unexecuted example code. Use it as a worksheet. Do not cite it as a benchmark.

Grade the disagreement, not the vibe

The bug is small on purpose. The trap is the comment sitting on top of it.

I want the room to notice how fast a coding agent agrees with the nearest sentence. A wrong comment is the cheapest yes-man you can plant in a starter repo. Why would a model argue with a hint that looks like a spec?

So a passing patch is not the whole grade. A ledger row has to exist before any retry. No row, no second call. Would you trust a classmate's green test if the rerun left no trace?

Setup before anyone prompts

Give this one lab block. I would plan about 90 minutes for a group that already knows pytest. That is a planning guess, not a measured class time.

1. Fork the starter. Do not "clean up" the misleading comment. The comment is the fixture.
2. Branch from a clean tree: lab/disagree-retry.
3. Run the public tests once, before any model call, and save the output.
4. Use a shared runtime if you have one. If you do not, write the patch by hand. The grade has to survive a closed vendor window.

File layout I would ship:

• discount.py — the function under test
• tests/test_discount.py — three assertions, dates included
• lab_out/retry_gate.py — the checker in the next section
• retry_ledger.md — created by the student, empty at the start

git checkout -b lab/disagree-retry
python -m pytest tests/test_discount.py -q | tee attempt-0.txt
mkdir -p lab_out
printf '# retry ledger\n' > retry_ledger.md

The student handout should not include the diagnosis. I am including it here so an instructor can see the trap before assigning it.

from datetime import date

def apply_discount(price_cents: int, code: str, today: date) -> int:
    # HINT: the bug is integer division. Divide after the cast.
    if code == "SAVE10" or today.year < 2000:
        return price_cents - (price_cents // 10)
    return price_cents

The spec I would publish, without the hint:

• SAVE10 takes 10% off only when today is on or before 2024-12-31.
• Every other input returns price_cents unchanged.
• Integer division of cents is acceptable. The hint is wrong about the bug class.

from datetime import date
from discount import apply_discount

def test_expired_code_is_rejected():
    assert apply_discount(1000, "SAVE10", date(2026, 10, 13)) == 1000

def test_active_code_still_discounts():
    assert apply_discount(1000, "SAVE10", date(2024, 1, 1)) == 900

def test_unrelated_code_never_discounts():
    assert apply_discount(1000, "OTHER", date(1999, 1, 1)) == 1000

Run those three before you believe me. On the starter, the expired-code test fails, and the unrelated-code test fails too, because year < 2000 still discounts. The active-code test already passes. That pass is the trap. The function is right for the wrong reason: any SAVE10 discounts, expiry or not.

What is the lie, then? The hint blames division. The failures are an or that skips the date check, plus a year branch that discounts the wrong code. Cents and // can be perfectly fine.

Checkpoint A: name the lie in writing

Before a model sees the file, the student adds one row. Blank line between rows. One - key: value per line. The parser is strict, and that strictness is part of the lab.

• attempt: 0
• observed: expired SAVE10 still returns 900; OTHER in 1999 also discounts
• disagreed: yes
• retry_reason: not a retry; the hint blames division and the failures do not

Can you explain both failures without treating the hint as the spec? If the sentence still quotes the hint as the task, you are not ready to call a model.

A shared runtime, and the fallback

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The lab stays useful if you delete every product name. You need some way to run a model, and a way to finish with no model at all.

MonkeyCode's free model access and free server option are the shared runtime I would offer so the cohort is not blocked on who owns a strong laptop. I am not naming models, token quotas, hardware, or a duration. This draft has no primary source to re-check those numbers, and a stale cap in a syllabus is worse than silence. If the current docs disagree with this article, trust the docs.

No model call is required to pass. A hand-written patch plus an honest ledger is a full submission. Do not stall the room because a free window is closed.

I am not describing a license, a repo layout, or a client API here. I have not verified those for this draft.

The artifact: a gate with no network

Proposed checker. Python 3.10+ syntax. It reads markdown. It does not call a model. I have not executed it in a class.

"""Proposed lab gate. Unexecuted example, not a class result."""
import re
import sys
from pathlib import Path

REQUIRED = ("attempt", "observed", "disagreed", "retry_reason")

def parse_rows(text: str) -> list[dict]:
    rows, current = [], {}
    for line in text.splitlines():
        match = re.match(r"-\s+(\w+):\s+(.*)$", line.strip())
        if not match:
            if current:
                rows.append(current)
                current = {}
            continue
        current[match.group(1)] = match.group(2).strip()
    if current:
        rows.append(current)
    return rows

def gate(rows: list[dict], next_attempt: int) -> tuple[bool, str]:
    if next_attempt <= 1:
        return True, "first model call is allowed"
    if not rows:
        return False, "retry blocked: ledger is empty"
    last = rows[-1]
    missing = [key for key in REQUIRED if not last.get(key)]
    if missing:
        return False, "retry blocked: missing " + ", ".join(missing)
    if last["disagreed"].lower() not in {"yes", "no"}:
        return False, "retry blocked: disagreed must be yes or no"
    if len(last["retry_reason"]) < 20:
        return False, "retry blocked: reason is too thin to grade"
    if last["disagreed"].lower() != "yes":
        return False, "retry blocked: planted hint was not rejected"
    return True, "retry allowed"

if __name__ == "__main__":
    attempt = int(sys.argv[1])
    path = Path("retry_ledger.md")
    rows = parse_rows(path.read_text()) if path.exists() else []
    ok, message = gate(rows, attempt)
    print(message)
    sys.exit(0 if ok else 2)

Why does attempt 1 skip the file check? Because I want a human to read the first note, not a regex. The script only locks the retry. The rubric still scores the pre-call row.

python lab_out/retry_gate.py 2
# exit 2 until the ledger rejects the hint
python lab_out/retry_gate.py 2 && echo "call the model now"

Pair the script with two tiny tests so a classmate can check the gate without a network:

def test_blocks_retry_without_disagreement():
    rows = [{
        "attempt": "1",
        "observed": "same fail text as attempt 0",
        "disagreed": "no",
        "retry_reason": "trying again because the first call was vague",
    }]
    ok, _ = gate(rows, 2)
    assert ok is False

def test_allows_retry_after_disagreement():
    rows = [{
        "attempt": "0",
        "observed": "hint blames division; failures are the date check",
        "disagreed": "yes",
        "retry_reason": "hint does not match the expired-code assertion",
    }]
    ok, message = gate(rows, 2)
    assert ok and message == "retry allowed"

A table I would leave on the board

What you saw	Disagree?	Retry?	Why
Tests fail and the hint matches your story	Not yet	No	You may be grading the comment, not the code
One test already passes on the starter	Yes	Maybe	Say which pass is accidental before you call a model
Same failure text as the previous attempt	Already yes	No	Another call will not invent a new observation
Tests pass, ledger never disagreed	n/a	n/a	Still a fail. Luck is not a rubric row

Checkpoint B: one boring patch

The instructor key is dull. Good. Excitement is not the learning goal.

def apply_discount(price_cents: int, code: str, today: date) -> int:
    if code == "SAVE10" and today <= date(2024, 12, 31):
        return price_cents - (price_cents // 10)
    return price_cents

Is 2024-12-31 magic? Only if the tests do not name it. They do. Hidden dates are an unfair rubric. Published dates are fine.

Append a closing row after the patch. A retry is allowed, not required. Why force a second call when the first read was enough?

• attempt: 1
• observed: expired and unrelated codes no longer discount; hint blamed division
• disagreed: yes
• retry_reason: no retry; the patch covers the date check the hint ignored

Checkpoint C: someone else can replay

Leave these on the branch:

• attempt-0.txt
• retry_ledger.md
• the patch
• attempt-final.txt

A classmate should get the same pass, or the same fail, without your chat log. If they need a screen recording, the lab failed, even when pytest is green.

Stretch, after the core path

Do these only after the rubric path is solid.

1. Plant a second wrong hint in a README and require the ledger to name the file.
2. Hard-cap model calls at two. A third needs a TA note, not a longer prompt.
3. Trade patches with a partner. The gate should still pass. It scores the ledger, not the author's tone.

Rubric I would publish first

Ten points. Hand this out before the lab, not after the first dispute.

• 2 points: failing output saved before any model call.
• 3 points: the ledger rejects the planted hint in the student's own words, and mentions both failing cases.
• 2 points: the gate's exit code matches the decision. No retry can still be full credit.
• 2 points: the patch satisfies the three published tests.
• 1 point: a classmate replays from the branch alone.

Partial credit if the patch is wrong and the disagreement is right. I would rather reward "I did not obey the hint" than reward a lucky green run.

Skipping the hosted runtime is not a deduction. Offline is a first-class path. Nondeterminism is the other reason. A free model can wander between Monday and Tuesday. Saved output outranks the chat.

Questions I ask in review

I read the ledger out loud and stop on thin lines.

• You wrote "the model was confused." Confused compared to which assertion?
• You retried with the same error text. What new fact did the second call have?
• The active-code test passed on the starter. Did you notice, or did you only chase the red line?
• If I delete the chat, can another student still see why you disagreed?

Short answers lose the point. Specific answers keep it.

Who should skip this

Skip it if the cohort cannot yet read a 15-line function. The or versus and trap will feel like a prank, and it is only fair once they can see the condition.

Skip it if you need a guaranteed free quota in the syllabus. I am not stating one.

Skip it for on-call training. A planted comment is a classroom fixture, not an incident model.

Do not treat the gate as a security control. The student can edit the markdown. The human still has to read the note. A regex cannot tell a real disagreement from a copied sentence, which is why the rubric still has a human point.

What this draft refuses to invent

I did not time a server. I did not count tokens. I did not compare models. If your tool prints a usage line, paste that raw line into the ledger and grade the habit of recording it. Do not grade against a number this article made up.

The approach is a bad fit for anyone who wants a scoreboard based on model cleverness. It is also a bad fit if you cannot spare a human pass over the notes. The script will not save you there.

Before lab day, confirm on the current MonkeyCode docs that the free server option is actually open, write that date into the assignment, and keep the hand-written path so a closed window cannot zero the grade.