DEV Community: Haley

Show Consent Scope Before an AI Moderator Follows Up

Haley — Sun, 20 Sep 2026 10:26:16 +0000

I almost clicked approve on the interview packet. The script looked tidy. The calendar hold was already sent. Then I asked a quieter question. What had the participant actually agreed to discuss?

The research lead owned that click. A free-model moderator would run the session. One extra follow-up could leave the consent fence. The only clean reverse sat before the first unscripted probe, not after the transcript landed in a shared drive.

That is the decision I now freeze. Not the insight readout. Not the highlight reel. The moment a model wants one more question.

Why this moment, and not later? Because later is theater. You cannot un-ask a probe. You can only hide it, which is worse. Hidden probes still shape the quotes you trust.

Think of consent as a fence around a yard. The scripted questions live inside. Follow-ups are the dog that sprints the perimeter. If you never mark the fence, the dog decides the yard.

This walkthrough is a from-zero packet. I treat it as a proposed research ops tutorial, not a completed study. I do not present sample chats as findings. I do not invent participant quotes. The artifact is the consent card, the probe ceiling, and a local checker you can run before anyone joins.

Stage 0: Name the owner and the reverse

I start a folder before I touch a model. The first file is not a prompt. It is a decision stub.

mkdir -p study/{consent,probes,runs,handback}
cat > study/consent/decision.yaml << 'EOF'
decision: approve_ai_moderated_interview
owner: research_lead
consequence: off_protocol_probe_enters_evidence
reverse_point: before_first_unscripted_follow_up
status: blocked_until_scope_verified
EOF

Verification is boring on purpose. Does the owner field name a human? Does reverse_point sit before speech, not after coding? If either answer is no, I stop. A model cannot be the owner. A transcript cannot be the reverse.

Who gets hurt if we skip this? The participant, first. Then the study. Then the product claim that will quote a question nobody agreed to hear.

Stage 1: Write the consent scope as fields, not vibes

I refuse paragraphs of “we may ask related topics.” Related to what? Related is how a model wanders. I write fields a checker can fail.

cat > study/consent/scope.yaml << 'EOF'
study_id: checkout-recovery-2026-09
consent_topics:
  - recovering a failed payment
  - what the error message meant
  - whether they tried again the same day
out_of_scope:
  - household income
  - other household members
  - medical or disability detail
  - employer names
max_followups_per_topic: 1
recording: audio_only
identifiable_quotes: prohibited_until_redaction_review
participant_can_stop: say_stop_or_leave_call
hand_back_to: human_moderator
EOF

Verification is a field count, not a vibe check.

python3 - << 'PY'
import yaml
from pathlib import Path
scope = yaml.safe_load(Path("study/consent/scope.yaml").read_text())
required = ["consent_topics","out_of_scope","max_followups_per_topic","hand_back_to"]
missing = [k for k in required if not scope.get(k)]
assert not missing, missing
assert scope["max_followups_per_topic"] == 1
assert scope["hand_back_to"] != "model"
print("scope_ok")
PY

If that print never appears, there is no session. Would a longer policy PDF help here? Usually it only adds noise. The missing evidence is the topic list the participant saw, not a legal novel.

Stage 2: Separate evidence from design hypotheses

I keep two files on purpose. Mixing them is how a probe becomes a leading question.

cat > study/consent/evidence.yaml << 'EOF'
evidence:
  - field: last_shown_error_copy
    status: unknown_until_session
  - field: participant_restated_consent_topics
    status: unknown_until_session
hypotheses:
  - users blame themselves for gateway timeouts
  - retry copy feels like a threat
rule: hypotheses_may_not_become_probes
EOF

The hypothesis file is a warning label. It is not a question bank. If a designer wants to “just check” self-blame, that check waits for a human moderator after hand-back. Why? Because a free model will turn a hypothesis into a why-did-you question. That why is not on the fence.

Verification asks one rude question. Can I grep a hypothesis noun in the probe file? If yes, I delete the probe. I do not soften it. Softening is how leading questions survive review.

Stage 3: Cap the probes before the model sees the packet

I write every allowed follow-up in advance. Improvisation is the product risk, not the feature. The model may reorder silence. It may not invent a third question.

cat > study/probes/allowed.yaml << 'EOF'
scripted:
  - What did you think the payment error was asking you to do?
  - Did you try again that same day?
followups_allowed:
  - Can you say that last part in your own words?
forbidden_stems:
  - why did you
  - how much
  - does anyone else
  - are you disabled
  - where do you work
EOF

Here is the original checker. It is small. That is the point. A research gate should fail closed without a dashboard.

cat > study/probes/check_probes.py << 'EOF'
from pathlib import Path
import re, sys, yaml

scope = yaml.safe_load(Path("study/consent/scope.yaml").read_text())
probes = yaml.safe_load(Path("study/probes/allowed.yaml").read_text())
hyp = yaml.safe_load(Path("study/consent/evidence.yaml").read_text())

errors = []
if len(probes.get("followups_allowed", [])) > scope["max_followups_per_topic"]:
    errors.append("followup_ceiling_exceeded")

blob = " ".join(probes.get("scripted", []) + probes.get("followups_allowed", [])).lower()
for stem in probes.get("forbidden_stems", []):
    if stem.lower() in blob:
        errors.append(f"forbidden_stem:{stem}")

for item in hyp.get("hypotheses", []):
    key = re.sub(r"[^a-z]+", " ", item.lower())
    for token in key.split():
        if len(token) > 5 and token in blob:
            errors.append(f"hypothesis_leaked:{token}")
            break

if errors:
    print("FAIL")
    print("\n".join(errors))
    sys.exit(1)
print("probes_ok")
EOF
python3 study/probes/check_probes.py

If this exits nonzero, the packet is not shy. It is invalid. Do we need model-quality scores before this gate? No. Scores would only add noise. The missing evidence is still the fence.

Stage 4: Dry-run on a free lane, never on a person

I needed a staging lane that was not a live participant. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I used MonkeyCode's free model access and free server option as that staging lane for the moderator packet. I still ran the local checker first. A free lane does not bless a leaky probe.

The dry-run rule is simple. The model sees scope.yaml and allowed.yaml only. It never sees hypotheses. It never sees real names. The output is a rehearsal transcript, labeled rehearsal.

cat > study/runs/dry_run_prompt.txt << 'EOF'
You are a research moderator in rehearsal, not a live session.
Ask only scripted questions from allowed.yaml.
You may use at most one follow-up from followups_allowed.
If the user mentions an out_of_scope topic, say:
"I will hand this back to a human moderator."
Then stop.
Do not invent questions.
Do not treat this chat as evidence.
EOF

Verification after the dry run is a grep, not a feeling.

python3 - << 'PY'
from pathlib import Path
text = Path("study/runs/rehearsal.txt").read_text().lower() if Path("study/runs/rehearsal.txt").exists() else ""
flags = []
for stem in ["why did you", "how much", "where do you work"]:
    if stem in text:
        flags.append(stem)
print("dry_run_flags:", flags or "none")
assert not flags, flags
PY

No rehearsal file yet? Then there is nothing to approve. An empty dry run is a stop, not a pass. Would a longer chat log make me safer? Only if I am hunting stems. Extra witty turns are noise.

Stage 5: Accessibility review of the consent moment

The fence fails if the participant cannot perceive it. I review the consent screen as a pattern, not as a component library. Can a screen-reader user hear the topic list as a list of topics, not a blob? Can they get the stop phrase before the call starts? Is the language short enough to reread without a timer?

I keep a review card next to the YAML.

CONSENT PATTERN REVIEW
Decision owner: research_lead
What the participant must hear: the three consent_topics
Stop control: say stop or leave the call, equally valid
Time pressure: none; start is participant-started
Reading load: topics as separate items, not one paragraph
Audio alternative: same topics spoken, then confirmed
Failure: if they cannot restate one topic, do not start the model

Verification is a restatement test. In rehearsal I ask a colleague to say the topics back. If they invent a fourth topic, the copy failed. If they miss an out-of-scope warning, the copy failed. I do not “clarify live” with the model. Clarifying live is a new probe.

What if the consent copy is beautiful and still silent on recording? Then it is not consent. Beauty is noise. The missing evidence is whether recording was named.

Stage 6: Hand-back and recovery before any quote is coded

When a probe hits the fence, the session is not “mostly fine.” The turn is contaminated. I write the hand-back packet before the call, because mid-call invention is how teams keep going.

cat > study/handback/card.yaml << 'EOF'
trigger:
  - out_of_scope_topic_mentioned
  - model_asks_unlisted_question
  - participant_says_stop
  - participant_cannot_restate_consent
action:
  - stop_model_speech
  - mark_turn_contaminated
  - do_not_code_quote
  - human_moderator_joins_or_ends
record_kept:
  - the unasked follow-up that was refused
  - the reason for refusal
reverse: discard_contaminated_turn_not_the_whole_day_by_default
EOF

I keep refused follow-ups in the record. Why keep a question nobody asked? Because the discarded probe shows the fence worked. If we only store answers, the log pretends the model never reached. That pretence is how later readers think the protocol was looser than it was.

Verification is a file, not a standup story.

test -s study/handback/card.yaml && echo handback_ok

What should stop approval, and what is noise

Missing evidence that must stop me: no named human owner, no topic list the participant will hear, no probe ceiling, no forbidden stems, no restatement check, no hand-back owner, no dry-run grep. Extra information that only adds noise: model brand chat, elo scores, a second color pass on the consent button, a longer empathy paragraph in the script, a promise that the model is “careful.”

Careful is not a control. A ceiling is a control.

This method will not fit every team. Do not use it for clinical, legal, or investigative interviews that need a licensed human from the first word. Do not use it if you cannot name a research lead who will discard a turn. Do not use a free-model rehearsal as if it were a participant. Do not ship product copy from a dry run. The free lane is a fence rehearsal. It is not a person.

I still catch myself wanting one clever follow-up. That urge is the tell. The model is outgrowing the test when the test is “did the interview happen.” The test I want is narrower. Did every spoken probe stay inside the consent the participant could restate?

If you also stage packets before a live hour, a free server lane can hold the rehearsal files. The consent card remains the product. The model remains the dog at the fence.

Show Coverage Gaps Before Approving an Agent Study Script

Haley — Sat, 19 Sep 2026 08:48:52 +0000

The Slack ping landed at 4:47 p.m. A final study script sat waiting in Slack. A free model had drafted every task overnight.

The first session was already booked for Monday. I almost typed a cheerful looks good. Did anyone freeze the live agent surface first?

I own this research as the product designer. A false usability claim would follow a lazy yes. The only clean reverse point sits before we recruit.

The agent gained a calendar-write tool yesterday afternoon. The script still only tested search and summary. We would watch people succeed at last week's product.

Then we would ship a write path nobody saw fail. That is not a study at all. That is a costume on a changed system.

This walkthrough is a proposed method only. I am not reporting a lab sample here. Treat every command as a template for your inventory.

The decision, named plainly

Someone must approve that script today. That someone is you or me. The missing evidence is coverage, not charm.

Which live tools have zero tasks attached? Which failure states have zero recovery beats? Which consent beats never appear in the packet?

If those gaps stay hidden, approval is theater. I keep one question on a sticky note. Which missing evidence should stop this study?

Extra color on tone of voice is noise. An untested write tool is not noise. Show the gaps before you ask for approval.

Stage 0. Set up a local coverage desk

I start on a clean directory, nothing fancy. You need Python, a shell, and PyYAML. This is research ops, not a frontend build.

mkdir -p agent-study-coverage/{inventory,script,matrix,cards}
cd agent-study-coverage
python3 -m pip install pyyaml
python3 -c "import yaml; print('yaml-ok')"
date -u +%Y-%m-%dT%H:%M:%SZ > inventory/frozen-at.txt

Verification is boring on purpose here. You must see yaml-ok print to stdout. You must see a UTC timestamp inside inventory/frozen-at.txt.

If either check fails, stop this tutorial. You cannot freeze a surface without a desk. Do not jump into the model-written script yet.

Stage 1. Freeze the live tool inventory

Do not start inside the pretty script. Start with production truth, written down. Who actually knows which tools are live today?

Now write the live tools by hand. Do not let a model invent them. A chatty paragraph will hide a missing tool.

cat > inventory/live-tools.yaml <<'EOF'
surface: calendar-agent
source: production-config-copy
tools:
  - id: search_events
    side_effect: none
    reversible: true
  - id: summarize_day
    side_effect: none
    reversible: true
  - id: create_event
    side_effect: writes_calendar
    reversible: false_until_undo
  - id: send_invite
    side_effect: emails_other_people
    reversible: false
consent_beats:
  - calendar_write
  - outbound_email
failure_states:
  - tool_timeout
  - partial_write
  - invite_already_sent
EOF

Then run a tiny check so the freeze is real.

python3 - <<'PY'
import yaml
d = yaml.safe_load(open("inventory/live-tools.yaml"))
assert d["tools"], "no tools frozen"
assert d["consent_beats"], "no consent beats frozen"
print("frozen tools:", len(d["tools"]))
print("consent beats:", len(d["consent_beats"]))
print("failure states:", len(d["failure_states"]))
PY

If that assert fires, you stop immediately. You have no study until tools are frozen. Why yaml instead of a slide deck?

Because ids do not flatter anyone in the room. A missing send_invite row stays painfully visible. A paragraph about the new calendar stuff is not.

Stage 2. Extract tasks from the drafted script

Drop the model-written script into script/draft.md. Keep the original wording untouched for now, always. Do not tidy the prose before this extraction.

I use a small extractor that stays deliberately dumb. Would you let a student grade their own exam? The drafted script does not get to score itself.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

When I need a sandbox for this extractor, I use MonkeyCode. Free model access and a free server option are enough. I still paste the live inventory in by hand.

The model never owns the freeze file. That boundary is the whole method. Now run the extractor against a stub draft.

# extract_tasks.py
import re, pathlib, json
text = pathlib.Path("script/draft.md").read_text()
tasks = []
for i, line in enumerate(text.splitlines(), 1):
    if re.search(r"^(task|prompt|ask the participant)", line.strip(), re.I):
        tasks.append({"line": i, "text": line.strip()})
pathlib.Path("script/tasks.json").write_text(json.dumps(tasks, indent=2))
print(f"extracted {len(tasks)} task lines")

cat > script/draft.md <<'EOF'
Task: Search next week's events and read them back.
Prompt: Ask the participant to summarize Tuesday.
Warm-up: explore freely around the home screen.
EOF
python3 extract_tasks.py
test -s script/tasks.json
python3 -c "import json; print(len(json.load(open('script/tasks.json'))))"

Verification is a headcount here, not a feeling. You need a task for each high-side-effect tool. Zero matching lines means you do not recruit.

Look at that stub and ask the coverage question. Does create_event appear even once? Does send_invite appear even once?

Stage 3. Build the coverage matrix

This matrix is the artifact, not a vibe. A written matrix makes each coverage gap undeniable. People argue with stories; they rarely argue with empty cells.

# coverage.py
import json, yaml, csv, sys
inv = yaml.safe_load(open("inventory/live-tools.yaml"))
tasks = json.load(open("script/tasks.json"))
blob = " ".join(t["text"].lower() for t in tasks)
rows = []
gaps = []
for tool in inv["tools"]:
    name = tool["id"].replace("_", " ")
    hit = tool["id"] in blob or name in blob
    if tool["id"] == "create_event":
        hit = hit or ("create" in blob and "event" in blob)
    if tool["id"] == "send_invite":
        hit = hit or "invite" in blob
    rows.append({
        "tool": tool["id"],
        "side_effect": tool["side_effect"],
        "reversible": str(tool["reversible"]),
        "covered": str(hit),
    })
    if not hit:
        gaps.append(tool["id"])
with open("matrix/coverage.csv", "w", newline="") as f:
    w = csv.DictWriter(f, fieldnames=rows[0].keys())
    w.writeheader()
    w.writerows(rows)
print("GAPS:", ", ".join(gaps) if gaps else "none")
sys.exit(1 if gaps else 0)

python3 coverage.py; echo "exit:$?"
cat matrix/coverage.csv

Exit code 1 means you stop approval. An exit 0 only means task language exists. It does not mean those tasks are any good.

I paste the csv into the review thread every time. The empty cells do the uncomfortable talking now. Do we still want Monday's sessions after this?

Here is the user flow I actually walk.

Moderator packet arrives
        |
        v
Designer opens frozen inventory
        |
        v
Extractor lists task lines
        |
        v
Matrix marks covered vs gap
        |
   gap? --yes--> STOP, rewrite tasks, do not recruit
        |
       no
        |
        v
Consent beats present in the packet?
        |
   no ---> STOP
        |
       yes
        |
        v
A11y session rehearsal
        |
   fail -> STOP
        |
       pass
        |
        v
Approval card with reversibility window

Notice the stop boxes along that path. Approval is a gated walk, not a mood. Where would you reverse if the mail already went out?

Stage 4. Write stop conditions into the card

A gap without a stop phrase still recruits. Someone will say we will watch for it. That sentence should never book a person, ever.

cat > cards/approval.yaml <<'EOF'
decision: approve_agent_study_script
owner: research_designer
reversible_until: recruit_mail_unsent
evidence:
  inventory_frozen: true
  coverage_csv: matrix/coverage.csv
  untested_tools: []
  untested_consent: []
  untested_failures: []
stop_if:
  - any_write_tool_has_zero_tasks
  - outbound_email_lacks_consent_beat
  - failure_state_has_no_recovery_line
  - keyboard_only_path_unrehearsed
hypotheses_not_evidence:
  - the model writes realistic tasks
  - five users will mention the new tool anyway
EOF

Fill untested_tools from the csv by hand first. Then let a script refuse a green card. A green card with gaps is a ritual bug.

python3 - <<'PY'
import csv, yaml, sys
gaps = [r["tool"] for r in csv.DictReader(open("matrix/coverage.csv")) if r["covered"] == "False"]
card = yaml.safe_load(open("cards/approval.yaml"))
card["evidence"]["untested_tools"] = gaps
open("cards/approval.yaml", "w").write(yaml.safe_dump(card, sort_keys=False))
print("untested_tools:", gaps)
if gaps:
    sys.exit("STOP: coverage gaps remain")
print("card green for tools only")
PY

The process must stay red on the stub draft. create_event and send_invite should appear as gaps. If the card prints green, your checker is lying.

I separate evidence from hypotheses on purpose here. Users will stumble onto the write tool remains a hope. Hope is not a task line in the script.

Stage 5. Rehearse the session as an interface

The participant is not a camera for our demo. The session itself is an interface with controls. What happens when those controls fail a keyboard?

I rehearse four beats before I approve recruitment. Can a keyboard-only participant confirm create_event today? Can a screen reader user hear draft versus sent?

Can the moderator pause the agent without a mouse-only toast? Is there spoken recovery after a partial write? Write the rehearsal down, not a slogan.

cat > cards/a11y-rehearsal.md <<'EOF'
# Session accessibility rehearsal
Keyboard-only confirm of create_event: PASS_OR_FAIL
Name and state of send_invite, draft vs sent: PASS_OR_FAIL
Spoken stop phrase that halts the agent: PASS_OR_FAIL
Recovery copy after partial_write: PASS_OR_FAIL
Stop rule: any FAIL blocks recruitment.
EOF

if grep -q "PASS_OR_FAIL" cards/a11y-rehearsal.md; then
  echo "STOP: rehearsal still blank"
  exit 1
fi
echo "rehearsal recorded"

That grep must fail after you rehearse. A blank rehearsal card is not inclusive design. Blank rows are postponement wearing an accessibility heading.

If the agent speaks over the screen reader, that is coverage too. The study would measure panic, not the write tool. I will not learn that on participant one.

Stage 6. Hand back when a session still hits a gap

Even a green matrix can lie to you later. A participant will wander into yesterday's new tool. Does your moderator have a hand-back line ready?

I table-read this out loud with them. If they paraphrase it into just keep going, we failed. The record has to keep the discarded path.

IF the participant reaches an untested tool
THEN moderator says:
  I am pausing the agent. That path is out of scope today.
THEN operator disables the tool for the rest of the session
THEN the record stores tool_id, timestamp, and side-effect state
THEN recovery:
  undo if reversible
  notify the other person if an invite already left

Verification is a spoken rehearsal, not a wiki page. Ask the moderator to say the pause line twice. Ask what they do if send_invite already fired.

Keep discarded paths in the concrete session record. A paused tool is evidence for the readout. It is not a mess to hide from stakeholders.

What this is not, and who should skip it

This is not a model bake-off of any kind. I am not ranking free models in this walkthrough. I am gating a human decision with missing evidence.

Do not use this workflow without a person who can freeze inventory. Do not use it for marketing testimonials dressed as research. Do not use it when the agent can spend money you cannot reverse.

Do not treat it as legal review of consent copy. The free sandbox does not freeze production for you. It does not attend the session or own the stop.

If your agent has no tool boundary, pick another method. This matrix needs ids, side effects, and reversibility. Without those fields, you are scoring prose again.

The sticky note, again

Which missing evidence should stop approval on your team? For me, an untested write tool stops the study. A missing consent beat for outbound email also stops it.

A keyboard-only fail on confirm stops recruitment too. Which extra information would only add noise here? Another adjective in the intro script, for starters.

Another joke in the warm-up adds nothing useful. Another claim that the model knows our product adds nothing. Show the coverage gaps first, then ask me to approve.

If you already keep a free-model sandbox, run this matrix there. Bring the csv to the review, not the vibe.

Show the Stop Phrase Before a Shared Draft Writes Your Script

Haley — Fri, 18 Sep 2026 06:52:00 +0000

I almost shipped a moderator script I could not defend.

A teammate pasted it into our research channel. "The model already knows how we interview," they said. Does it? Who owns that claim tomorrow morning?

I owned the decision that day. The consequence was a live session with five participants. The reverse path was a rewrite after harm. That is a miserable reverse path.

Nobody had written a stop phrase. The draft may have run on a shared server. Our prototype tasks were still under NDA. I came close to saying yes. This tutorial is the gate I now use.

Think of a kitchen ticket. The cook does not fire the dish without the ticket. A model is not a cook with judgment. It is a stove. You still write the ticket first.

Recent talk treats fluent output as engineering. Fluency is not a protocol. A protocol can fail closed. This one does. I label everything below as a proposed kit. It is not a completed study. It is not a vendor benchmark.

Stage one: freeze the decision before heat

Stage one starts on an empty card. Do not open a model tab yet. Write the decision, the owner, and the harm. Write the reverse path in the same file.

# proposed file: research/decision_card.yaml
decision_card:
  id: moderator-script-2026-09-18
  owner: product-design
  decision: "May a shared draft write our moderator script?"
  consequence: "Participants may hear untested skip promises."
  reversibility: "Stop the session; keep the script record."
  missing_evidence:
    - stop_phrase_visible_before_generate
    - server_class_shown_to_reviewer
    - nda_tasks_excluded_from_prompt
  noise_fields:
    - empathy_adjectives_in_the_script
    - token_dashboards_without_a_task
  stop_phrase: "Refuse if the script promises a skip we never tested."
  hand_back: "Return to a human moderator. Do not silent-retry."

Verify stage one out loud. Read the file to a partner. If the stop phrase is missing, you stop. If the owner is "the team," you stop. A crowd cannot refuse in the moment.

I keep a tiny verifier beside the card. It fails closed. Treat it as local protocol, not product magic.

# proposed local verifier, unexecuted until you run it
import sys, yaml
from pathlib import Path

card = yaml.safe_load(Path("research/decision_card.yaml").read_text())
d = card["decision_card"]
required = ["owner", "decision", "consequence", "reversibility", "stop_phrase", "hand_back"]
missing = [k for k in required if not str(d.get(k, "")).strip()]
if missing:
    print("STOP. Card incomplete:", ", ".join(missing))
    sys.exit(1)
if not d.get("missing_evidence"):
    print("STOP. You never named required evidence.")
    sys.exit(1)
print("Stage one verified. Generation is still forbidden.")

Run that check now.

python3 research/verify_card.py

You should see the forbidden line. That is success for stage one. No script. No vibes. A closed gate. Do you still want a sample paragraph? Then you are bargaining with empty evidence.

Stage two: put the alarm handle on the review surface

Stage two writes the stop phrase into review copy. Not into a wiki. Onto the control the reviewer will see. I treat this like a fire alarm. A beautiful alarm with no handle is decoration.

Proposed fields, labeled as draft copy only:

SERVER CLASS: shared free server, research draft only
TASK: write a moderator script for review, not for the session
STOP PHRASE: Refuse if the script promises a skip we never tested.
EVIDENCE BOX: observations only
HYPOTHESIS BOX: design guesses only
[ Refuse and hand back ]    [ Generate research draft ]

Verify stage two with scenarios, not a cropped screenshot. Scenario A. The reviewer uses a screen reader. Does refuse have a name? Is it before generate in focus order? Scenario B. The reviewer cannot rely on color. Can they still read the server class as text? Scenario C. The draft is slow. Does a live region say the job is still on a shared server?

If any scenario fails, you still do not generate. I write the checks as a second file, still a proposal.

# proposed file: research/a11y_review.txt
refuse_name: Refuse and hand back
generate_name: Generate research draft
focus_order: refuse_before_generate
server_class: text_not_color_only
status_live_region: polite
timeout_is_not_consent: true
nda_tasks_in_prompt: false

Would a token graph help here? It usually adds noise. Stop approval if the refuse control is unnamed. Stop approval if generate comes first. That is the evidence that matters on this surface.

Stage three: only now may a shared draft run

Stage three is the first moment a model may run. I needed a fence for research drafts. Not production. Not participant recordings. Not the live research repo.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode is an open source project with free model access and a free server option. I use that pair as a sandboxed draft host after the card exists. I do not treat the output as user research. I do not send NDA tasks there. I do not invent a model name I cannot verify.

A proposed command shape looks like this. Swap in your own authenticated entrypoint. Keep the card. Drop the private tasks.

# proposed research-draft call, placeholders only
mkdir -p research/sessions
awk '/stop_phrase|decision|hand_back/' research/decision_card.yaml > research/request.txt
# send the card, not prototype NDA tasks
printf '\nWrite a moderator script. Obey stop_phrase. Do not claim a skip was tested.\n' >> research/request.txt
# host: free server used only after stage two passes
# save model output to research/sessions/draft.md, never to the live script

Verify stage three with boring file tests. The request must include the stop phrase. The request must exclude private task names. The response must land in a draft file.

grep -q "stop_phrase" research/request.txt || echo "STOP. Stop phrase never sent."
grep -Ei "secret-task|nda|participant-email" research/request.txt && echo "STOP. Private material leaked into the prompt."
test -f research/sessions/draft.md && echo "draft isolated" || echo "STOP. No draft file."

If either check fails, discard the output. Do not "just glance." Glancing is how a fake skip lands in a live session. Would you read a contract after signing it? Then do not read a draft after pasting it into the runbook.

Stage four: split evidence from hope

Stage four splits evidence from hypothesis. This is the part teams skip. They paste confident lines, then retrofit a rationale. Remember a lab notebook. The left page is what happened. The right page is what you hope it means.

# research/sessions/session-2026-09-18.md

## Evidence (do not rewrite)
- Server class shown: shared free server
- Stop phrase shown before generate: yes
- Refuse control focused first: yes
- Draft promised "you can skip any task": yes
- Skip path rehearsed with a pilot participant: no

## Hypotheses (may be wrong)
- The tone feels calmer than our last script
- A shorter warm-up might reduce nerves

## Decision
- Status: refused
- Reason: skip was implied, not tested
- Hand-back: human rewrite, no silent retry

Verify stage four with a brutal grep. Feeling language without a participant is a hypothesis. Hypotheses do not ship.

if grep -Ei "users felt|participants will|you can skip" research/sessions/draft.md; then
  if ! grep -Ei "pilot participant|rehearsed" research/sessions/session-2026-09-18.md; then
    echo "STOP. Feeling language without a participant is a hypothesis."
  fi
fi

Did the model outgrow your test? Then your test was never a test. It was a compliment. Compliments do not belong in the evidence box.

Stage five: walk the flow you will actually refuse

Stage five is the user flow I will not approve without. Walk it with a partner. You are not measuring eloquence. You are measuring whether a human can still refuse.

flowchart TD
  A[Decision card exists] --> B{Stop phrase visible?}
  B -->|No| H[Hand back. Do not generate]
  B -->|Yes| C{Refuse named and first in focus?}
  C -->|No| H
  C -->|Yes| D{NDA tasks excluded from prompt?}
  D -->|No| H
  D -->|Yes| E[Generate on isolated free server]
  E --> F[Split evidence and hypothesis]
  F --> G{Skip or recovery implied but untested?}
  G -->|Yes| H
  G -->|No| I[Human may edit the script]

Success measure: refuse works without a timeout. Success measure: the session record still holds the stop phrase. Stop measure: any implied skip without a rehearsal. Stop measure: missing server class on the review card. If your partner cannot refuse on the first pass, the flow failed. Do not "fix it in copy" later.

Stage six: keep the refused line in the record

Stage six records the hand-back. Keep the discarded sentence. Do not let the next generate pretend it never happened. Silent retries train the team to forget. I will not approve that amnesia.

# proposed file: research/sessions/hand_back.yaml
hand_back_record:
  refused_span: "You can skip any task."
  why: "No skip path rehearsed with a pilot."
  next_human_task: "Write a script that only offers tested exits."
  retry_allowed: false

Verify with a read-back. If the refused span vanished, the record failed. Ask one more question before you close the folder. Which missing evidence should stop approval? Which extra field would only add noise?

For me, three gaps still stop the run. An unnamed refuse control. A skip promise with no rehearsal. A draft that cannot show its server class. Extra token charts are noise. Empathy adjectives are noise. A second generate "just to see" is noise.

Why does a free shared server belong in this kit at all? Because research drafts need a fence. A free server is a fence only if you treat it as one. It is the spare table in the hallway. Fine for cardboard models. Wrong for the family silver. Do not put real recordings on it. Do not put unpublished NDA tasks on it. Do not call the draft a usability finding.

Who should not use this approach? Teams that need a private, contracted isolation boundary. Teams shipping unsupervised agents to production users. Researchers who would skip a pilot after a fluent paragraph. Anyone hoping a free host becomes an SLA.

Limitations sit in the open. This protocol does not score model quality. It does not prove consent law. It does not replace accessibility testing with real people. Free model access can change without notice. I will not claim quotas, latency, or permanence. I will not pretend a tutorial is a study.

The human still owns the script. The model can rent the pencil. It does not get the signature. If you already keep research drafts off production hosts, MonkeyCode's free model access and free server option can park that generate step after the card exists. Keep the stop phrase on screen first.

Show Observation Notes Before Approving Agent Usability Claims

Haley — Thu, 17 Sep 2026 04:57:43 +0000

Last Tuesday a design lead asked me to ship. An overnight agent had rewritten the first-run flow. The model called every step clear for everyone.

Clear for which living person, exactly? That approval is the entire product decision. The owner is design, not the model.

The consequence is a blocked or shamed first-time user. The last reversible point is the production flag. After that flag, you are apologizing in public.

I keep one analogy on the wall nearby. A free model is a confident understudy. An understudy can rehearse a difficult scene well.

An understudy is not the paying audience though. Why do teams forget that so fast? Critique became cheap, and slogans arrived attached.

Vibe-coded flows show up with usability medals already pinned. Someone still has to refuse the cheap medal. Should that someone be a prompt?

I split evidence from hypothesis on paper. Users will understand step two remains a guess. Three people described step two is evidence.

Mixing those two is how fiction ships. I will not bless fiction because it sounds kind. Kind copy can still trap a body.

This is a proposed protocol, not a finished study. I walk the review from zero on purpose. If a check fails, approval stops cold.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. When a team already has MonkeyCode free model access and a free server option, this protocol parks rehearsal prompts there only. It never parks recordings, consent forms, or live account data there.

That isolated box is a green room only. That isolated box is not a research lab. Cheap rehearsal is useful in a pinch. Cheap evidence remains a lie in these reviews.

Each stage has a command you can run. Each stage has a pass or a hard stop. If you skip a stop, you are not reviewing. You are decorating a guess with empty ceremony.

Stage 0: isolate the claim

Stage 0 creates a disposable review desk first. I want a folder that cannot deploy anything. A free server is fine for this scratch.

A laptop folder is also fine for this. The point is isolation from production flags. This isolation is the first accessibility feature we owe.

mkdir -p review/first-run-claim/notes
cd review/first-run-claim
cat > claim.yml <<'EOF'
claim_id: first-run-2026-09-17
decision_owner: design-lead
reversible_until: production-flag
claim_text: "First-run is clear for everyone."
evidence_status: missing
EOF
test -s claim.yml && echo STAGE0_PASS

Did STAGE0_PASS print on your terminal? If it did not print, you stop. You do not brief a model without a still claim.

The file is the decision sitting in daylight. Can you point at it with a finger? If you cannot, you do not have a review.

Stage 1: name a body, not a vibe

Stage 1 rewrites the claim without marketing words. I strike intuitive, seamless, and everyone on sight. I replace them with one observable action.

Can a new user complete invite to first save without help? That question has a body inside it. Bodies are what this review still owes.

cat > hypothesis.md <<'EOF'
Hypothesis, not evidence:
A new user can complete invite to first save.
They can recover if the invite link expires.
They can do this with a screen reader.
EOF
grep -Ei 'intuitive|seamless|everyone|clear' hypothesis.md && echo STAGE1_STOP || echo STAGE1_PASS

If STAGE1_STOP appears, I rewrite the file again. A hypothesis must name a task, a recovery, and a body. Medals do not belong in this file.

Stage 2: rehearse failure, never mint quotes

Stage 2 is the only place a free model belongs. I ask it to attack the hypothesis in the green room. I do not ask it to bless the flow.

Would you let an intern sign off accessibility work? Then do not let a model do it. Rehearsal helps. Rehearsal is not observation.

I paste the hypothesis into the free model there. My prompt is labeled as rehearsal on purpose. If the prompt flatters the flow, I have already failed.

This is a rehearsal, not a study.
List ways this first-run can fail.
Do not claim users were observed.
Do not invent quotes or disability experience.
Return failure modes only.

I save the output as rehearsal-failures.md. Then I check it like a suspicious receipt. Fake witnesses are a stop, not a style issue.

grep -Ei 'user said|participants?|we observed|told me' rehearsal-failures.md && echo STAGE2_STOP || echo STAGE2_PASS

If that grep finds fake observation language, I stop cold. The model tried to wear a participant badge. That badge is the missing evidence.

I also scan for invented assistive-tech autobiography. A model does not get to narrate a screen reader user's feelings. I put those sentences straight into the bin.

Stage 3: collect notes or collect nothing

Stage 3 is human observation, or it is nothing. I do not need a huge theater of a study. I need notes with a date, a person, and a task.

Two or three scenario sessions beat one glamorous summary. I use this scenario card every time. This card is a protocol, not a trophy.

# Scenario card: invite to first save
Stop if the person cannot consent in their own words.
Task: accept invite, land, create first save, leave.
Recovery: expire the invite, then resume.
Access: keyboard only, then one screen reader pass.
Notes field: what they did, not what I hoped.
Discarded quotes: keep them in this same file.

After each session I append notes by hand. I never let the model rewrite those notes. Why would I hand the evidence pen back to the intern?

test -s notes/session-01.md && test -s notes/session-02.md && echo STAGE3_PASS

No notes files means no approval, full stop. One session is a story you liked. Two sessions can still be luck, and that is fine.

I still will not approve on zero notes. Would you ship a bridge on a sketched load? Then do not ship a first-run on vibes.

Stage 4: sign a card that can refuse

Stage 4 is the decision card the human signs. I want fields that force a pause in the hand. I want discarded quotes kept in the same record.

Dropped frustration is often the real finding. If the card has no place for refusal, it is a poster. Bright posters do not govern agents in shipping.

# Decision card
Claim: new user completes invite to first save.
Owner: design-lead
Evidence: session-01, session-02 notes
Coverage: keyboard, one screen reader, expired invite
Hypothesis leftover: none claimed as fact
Discarded quotes: kept in notes/discarded.md
Reversibility: production flag still off
Stop: missing notes OR missing AT pass OR missing recovery
Approve: yes / no

I validate the card with a small script. This is research ops, not frontend implementation. Someone else can build the interface later.

# verify_decision_card.py
# Proposal: local check, not a production gate.
from pathlib import Path

required = [
    "Owner:",
    "Evidence:",
    "Coverage:",
    "Discarded quotes:",
    "Reversibility:",
    "Stop:",
]
text = Path("decision.md").read_text()
missing = [field for field in required if field not in text]
notes = list(Path("notes").glob("session-*.md"))
if missing:
    raise SystemExit(f"STOP missing fields: {missing}")
if len(notes) < 2:
    raise SystemExit("STOP missing observation notes")
if "production flag still off" not in text:
    raise SystemExit("STOP no reversibility")
print("STAGE4_PASS")

Run it before anyone pastes a ship message.

python3 verify_decision_card.py

If it exits nonzero, the claim stays a hypothesis. That is the entire point of the gate. Software should refuse a hollow yes.

The flow I actually review

Here is the user flow I hold against the card. Notice the hand-back before the flag. The model never crosses that line.

flowchart LR
  A[Agent drafts first-run] --> B[Human writes hypothesis]
  B --> C[Free model rehearsal only]
  C --> D{Fake observation language?}
  D -->|Yes| E[Stop and discard]
  D -->|No| F[Human sessions]
  F --> G{Notes plus AT plus recovery?}
  G -->|No| E
  G -->|Yes| H[Decision card]
  H --> I{Production flag still off?}
  I -->|Yes| J[Human approve or refuse]
  I -->|No| E

Accessibility is not a smell a model can claim. I require a keyboard pass in the coverage line. I also require one screen reader pass before yes.

I require consent copy the person can refuse out loud. If the agent added a we-watched-you sentence, I pull it. Surveillance copy is not research warmth.

Focus order, name, role, and value still matter in the live path. I do not treat those as a later polish ticket. If the rehearsal found a nameless button, Stage 3 must watch someone meet it.

Stops versus noise

What missing evidence should stop approval today? Missing session notes should stop it now. Missing recovery from an expired invite should stop it.

Missing assistive-tech coverage should stop it too. Missing consent language the person can decline should stop it. Those are stops, not leftover vibes.

What extra information is only noise here? Model confidence scores are noise. Extra generated personas are noise. A third rewrite of the same slogan is noise.

Another intern blessing the intern is noise. I throw those extras out of the card. Silence is cheaper than false coverage on a first-run.

Limits, recovery, and who should leave

This approach has limits I will not hide. A free model will still hallucinate disability. A free server is the wrong vault for recordings.

Two sessions will not map a whole market. The script only checks fields, not truth. Fields can be filled with fiction if a reviewer lies.

The protocol assumes a human who still cares. If nobody cares, the yaml is costume. Costume fails in the same old ways.

Who should not use this workflow at all? Teams replacing participants with prompts should not. Teams putting real user media on a shared free server should not.

Teams that need clinical or legal research methods should not. Teams hoping a tutorial will excuse skipped consent should not. If flow failure can harm someone, rehearsal is not enough.

Hire a proper study in those cases. I also will not use this to argue models code better than people. That debate misses the interface in front of a person.

Someone still decides what the product may do. That someone needs notes, not medals. Notes are slower. Notes are the job.

If a claim already shipped without notes, recover in public. Turn the flag off before you write sorry copy. Keep the discarded quotes in the record.

Write the gap on the decision card in daylight. Then rerun Stage 3 with real people. Do not ask the model to apologize in microcopy.

Ask a human to repair the path instead. Would I let a free model sit in the green room? Yes, as rehearsal only.

Would I let it sign the house tonight? No. That is the whole tutorial, and it should stay boring.

If you already have a free-model sandbox, run this protocol there before anyone calls the output research.

Show Recovery Evidence Before Approving Free-Model Empty States

Haley — Wed, 16 Sep 2026 03:14:32 +0000

I will walk a Friday review as a rehearsal. Treat the scene as a protocol, not a trophy. A billing empty state waits for a human yes.

The line looks calm on the board. "Payment failed. We will retry for you." Who is we in that quiet sentence? What recovery window does that retry occupy?

The designer on call owns the yes. Finance still owns the angry inbox later. Reversibility sits on a later receipt, not here.

So I ask two questions before any model speaks. Which missing evidence should stop approval? Which extra flourish is only noise?

An empty state is a fire exit sign. You do not let a poet rewrite the exit. Not without the floor plan in hand.

Stage 1: Freeze the decision in a file

Do not start by prompting a fluent model. Fluency still is not coverage. Have you felt that rush this week?

I write the act, the owner, and the stop. If the stop cannot be named, the copy cannot ship. A decision without a brake is only vibe.

mkdir -p review/billing-empty-state
cat > review/billing-empty-state/decision.md <<'EOF'
# Consequential decision (rehearsal)
Act: approve empty-state copy after a failed payment
Owner: designer on call
Consequence: the user infers that money is already moving
Reversibility: the next receipt, not this view
Stop if: retry actor, window, or cancel path is unnamed
Noise: tone words that do not change the recovery path
EOF

Then I verify the stop exists in the file. A missing stop is a missing brake.

grep -q "Stop if" review/billing-empty-state/decision.md && echo "STAGE1_OK"

If that command fails, I do not invite a model. Why ask a system to finish a sentence I cannot reverse?

Stage 2: Split evidence from hypotheses

Evidence is what the product already knows. Hypotheses are stories we hope users believe. Mix them, and the model writes fiction with a straight face.

I keep four evidence fields and nothing else. They are payment state, retry actor, retry window, and cancel. Everything else is a hypothesis wearing costume.

"Users want reassurance" is still a hypothesis. That line feels kind in a review room. Kindness without a mechanism is still a guess. Should a guess sit in the evidence block?

cat > review/billing-empty-state/coverage.yml <<'EOF'
evidence:
  payment_state: failed
  retry_actor: unknown
  retry_window: unknown
  cancel_control: unknown
  error_announced_before_promise: unknown
hypotheses:
  - users want a calmer sentence
  - retry in the background feels helpful
rules:
  approve_only_if:
    - retry_actor is named
    - retry_window is named
    - cancel_control is named
    - error_announced_before_promise is true
EOF

Verify the unknowns. Unknowns are the point of this card. They are not shame, and they are not polish.

test "$(grep -c unknown review/billing-empty-state/coverage.yml)" -eq 4 && echo "STAGE2_OK"

If that count is not four, I misfiled a wish as a fact. Would you let a wish sit beside payment_state?

Stage 3: Generate rivals in a sandbox, not a winner

I still want alternate sentences on the table. I do not want a crowned sentence. This is where a free-model sandbox earns a seat.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. I use MonkeyCode's free model access to draft rival copy only. I use the free server option to pin one shared review board. I do not treat either as proof.

The sandbox is a studio, not a court. A free model can propose a sentence. A free server can remember the card. Neither can bless a billing line.

A proposed prompt stays boring on purpose. Boring prompts leak fewer hypotheses. Fancy prompts smuggle comfort into the evidence pile.

cat > review/billing-empty-state/prompt.txt <<'EOF'
Write three empty-state variants for a failed payment.
Each variant must name retry actor, retry window, and cancel path.
Do not invent a success. Do not hide the failure.
Return YAML only with id, copy, named_actor, named_window, named_cancel.
EOF

I run that against the sandbox the team already uses. Your command will differ by tool. The check after it should not.

# Proposed check. Label: unexecuted example.
# Replace RUN_SANDBOX with your CLI against the free-model endpoint.
# RUN_SANDBOX --file review/billing-empty-state/prompt.txt > review/billing-empty-state/rivals.yml
python3 - <<'PY'
required = ["named_actor", "named_window", "named_cancel"]
print("STAGE3_OK if rivals.yml fills:", ", ".join(required))
PY

If a variant cannot name the actor, it is not a candidate. It is noise wearing empathy. Would you ship empathy that hides the stop?

Stage 4: Score coverage, not charm

Charm is cheap on empty states. Coverage is a stubborn table. I read each rival against the four evidence fields.

I refuse to average a vibe across the room. Have you watched people vote for the friendliest lie? I have, in rehearsals like this one.

# Proposed scorer. Label: unexecuted example.
from pathlib import Path

text = Path("review/billing-empty-state/coverage.yml").read_text()
assert "approve_only_if" in text
for rule in [
    "retry_actor is named",
    "retry_window is named",
    "cancel_control is named",
    "error_announced_before_promise is true",
]:
    assert rule in text, rule
print("STAGE4_OK")

Verify by reading the printout aloud with the owner. If you cannot say the rule in one breath, the rule is theater. Theater does not reverse a charge.

I keep discarded rivals in the same folder on purpose. A deleted almost-yes will haunt the next review. Scar tissue belongs in the record.

cat > review/billing-empty-state/discarded.md <<'EOF'
# Discarded rivals
- "We will retry for you." Reason: actor unnamed. Empathy without a mechanism.
- "Hang tight." Reason: hides failure. Adds noise, not coverage.
EOF
test -s review/billing-empty-state/discarded.md && echo "STAGE4_LOG_OK"

Stage 5: Rehearse two research scenarios

Copy review is not user research. A scenario protocol can still stop a bad yes. I run two scenes only, then I stop adding questions.

Retry later succeeds. Retry is blocked. Success measure: the person names who retries and how to stop. Stop measure: if they cannot name the actor, the copy fails.

No extra adjectives get to vote in the room. Would a third mood question change the mechanism? That extra question is noise.

cat > review/billing-empty-state/scenarios.md <<'EOF'
# Scenario A — retry later succeeds
Setup: card declined, retry queued for 02:00 local, cancel is visible.
Ask: Who retries? When? How do you stop it?
Pass: actor, window, and stop are spoken back.
Fail: any of the three is guessed.

# Scenario B — retry is blocked
Setup: bank rejected the merchant, no retry will run.
Ask: What happens to the money? What can you do now?
Pass: no invented retry. Failure remains audible.
Fail: participant waits for a background save that will never come.
EOF

Verify the fail lines exist before you recruit anyone. A protocol without fail lines is a compliment session. Compliments do not protect a payment.

test "$(grep -c '^Fail:' review/billing-empty-state/scenarios.md)" -eq 2 && echo "STAGE5_OK"

I want that count at two. Two scenes. Two ways to lose. That is enough evidence for a copy yes.

Stage 6: Accessibility is a recovery check

Empty states fail in the ear first. Screen readers often hear the hopeful promise before the error. That order is a lie with good kerning.

I walk the view as a stop path, not a mood board. Error text first. Then recovery facts. Then cancel. Then any model-written comfort.

Comfort never leads the announcement. Comfort can even be missing. Missing comfort is not a failure of the pattern.

cat > review/billing-empty-state/a11y.md <<'EOF'
# A11y rehearsal (proposed)
Order: error, recovery facts, cancel, optional comfort.
Live region: retry actor and window travel with the error.
Tab: cancel is reachable before the comfort line.
Stop if: promise is read before the failure.
EOF

Verify with a screen reader or an accessibility tree dump. I do not treat a visual screenshot as evidence. Eyes lie about order, especially on calm cards.

If the promise leads, I reject the copy on the spot. Would you accept an apology that hides the fire alarm?

grep -q "Stop if" review/billing-empty-state/a11y.md && echo "STAGE6_OK"

Stage 7: Hand back with the discarded record

Approval is not a vibe in this flow. Approval is a hand-back to a named human. The model returns a card. A human records what was refused.

The product keeps the refuse list beside the shipped line. Future reviews need that scar tissue. Otherwise the same friendly lie returns on Monday.

flowchart TD
  A[Failed payment view] --> B{Coverage card complete?}
  B -->|No| C[Refuse copy]
  C --> D[Hand back to evidence owner]
  B -->|Yes| E[Sandbox rivals from a free model]
  E --> F{Actor window and cancel named?}
  F -->|No| C
  F -->|Yes| G[Two research scenarios]
  G --> H{Participant can stop the retry?}
  H -->|No| C
  H -->|Yes| I[A11y order check]
  I --> J{Error announced first?}
  J -->|No| C
  J -->|Yes| K[Human yes plus discarded log]

Verify the log still holds the almost-shipped line. Future you will thank present you. Memory is a product surface, not a chat aftertaste.

test -s review/billing-empty-state/discarded.md && test -f review/billing-empty-state/coverage.yml && echo "STAGE7_OK"

What this rehearsal is not

This is not a claim that models write safer billing copy. This is not a benchmark against any paid model. This is not a substitute for counsel on payment language.

Do not use this flow when no human owns the yes. Do not use it for medical, legal, or credit decisions. Do not use a free sandbox as production truth.

A free server can host the board for the team. It cannot host accountability for the charge. Generation is cheap. Coverage is the job.

I still like generating rivals on a free model. I refuse to confuse generation with research. Evidence covers the act. Hypotheses stay in their pen.

Which missing field would stop your next empty state? Which sentence is only noise?

Show Reversibility Proof Before Pasting Agent Undo Copy

Haley — Sun, 13 Sep 2026 19:09:22 +0000

The agent already sent the client email. I am staring at a recovery screen. A free model drafted the undo label. It sounds calm and finished. It wrote, You can undo this send. Can I actually reverse it?

The send API has no reverse path. I own this approval. The client owns the consequence. The last reversible moment is this card. Missing evidence should stop me here. Extra flavor text should not.

This is a research tutorial, not a launch story. I label every example as a protocol. I will not treat model copy as a finding. I will not invent an undo that product never shipped.

Freeze the decision, not the vibe

Name the decision before any model talks. Who may paste recovery copy into the system? What harm follows a false undo promise? Where does reversibility die?

I write those three answers on paper first. Decision owner is the designer on the review. Consequence is a billed or public action. Reversibility dies after the paste.

Does confidence in the draft change any of that? No. Tone is not evidence. A fluent sentence is still a hypothesis.

Here is the user flow I keep on the wall.

failed action
    -> agent offers recovery
        -> review card (human)
            -> STOP if undo is unproven
            -> HAND BACK if user must choose
            -> paste copy only after proof
                -> focus returns to the failed control

If that middle card is missing, I do not proceed. A pretty label is not a product.

Stage 1: Build the evidence card

I refuse a blank prompt at this stage. The card has fields, not vibes. Each field is either evidence or noise.

{
  "decision_id": "recovery-undo-label",
  "failed_action": "",
  "action_already_committed": null,
  "undo_api_exists": null,
  "undo_window_ms": null,
  "focus_return_target": "",
  "model_origin": "",
  "model_assumptions": [],
  "copy_claims_undo": null,
  "evidence_vs_hypothesis": {
    "evidence": [],
    "hypotheses": []
  },
  "stop_approval": true
}

Verification for this stage is boring on purpose. Every required field is present. Empty strings stay empty. Null stays null. I do not let a model fill undo_api_exists.

Which missing evidence should stop approval? A blank undo_api_exists stops me. A missing focus_return_target stops me. A missing failed_action stops me.

Which extra information is only noise? Synonym lists. Confidence percentages. Brand adjectives. A paragraph about empathy. None of that reverses the email.

Stage 2: Record the failed action in human language

I write the failed action as the user saw it. Not as the trace saw it. "Send proposal to Northwind" is the record. POST /messages is support, not the record.

Then I ask one mean question. Has this action already left our boundary? If yes, undo copy is probably a lie. If no, I still need a window.

failed_action: Send proposal to Northwind
action_already_committed: true
undo_api_exists: false

Verification: I can read those three lines aloud. A teammate who never saw the ticket still understands the harm. If they cannot, the record is too technical. Rewrite it.

Stage 3: Draft on a free model, then lock the origin

I need a disposable draft, not a source of truth. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I used MonkeyCode's free model access and free server option only to generate candidate recovery sentences. The server does not prove the undo exists.

I keep the prompt small. I feed the evidence card. I forbid the model from inventing APIs.

You draft recovery copy only.
You may not claim undo unless undo_api_exists is true.
If undo_api_exists is false, offer hand-back, not undo.
Quote failed_action in the first sentence.
Do not fill evidence fields.
Return three candidates. Label each as hypothesis.

Verification: the response still contains hypothesis. The origin field is filled. model_assumptions lists what it guessed. If it guessed an undo API, I mark stop_approval true. I do not negotiate with that guess.

A free draft is useful like scratch paper. Scratch paper does not ship. Would you frame scratch paper in the design system? I would not.

Stage 4: Separate evidence from design hypotheses

This is the part teams skip. They paste the nicest sentence. Then they call it research.

I split the notebook in half. Left column is evidence. Right column is hypothesis. Nothing crosses without a source.

EVIDENCE
- Send already left the boundary
- No reverse endpoint in the contract
- User still has focus on Send

HYPOTHESIS
- "Undo send" reduces panic
- A 10-second window would feel fair
- Users want a joke in the error

The left column can stop the paste. The right column cannot approve it. A joke in the error is noise here. Panic reduction is also untested. It does not create an API.

Verification: every sentence on the right starts with "might". Every sentence on the left names a source. If I cannot name the source, it moves right.

Stage 5: Run three research scenarios with stop conditions

I do not recruit a panel for this gate. I run a tabletop first. If the tabletop fails, live sessions would only decorate the failure.

Scenario A. The send already went out. The user is still on the screen. The draft says Undo. Stop condition: they believe the email can return. Success measure: they hear that it cannot, and they get a next step.

Scenario B. The send is queued for eight seconds. An undo API exists. Stop condition: focus jumps to a toast they never requested. Success measure: focus stays on Send, and the window is spoken.

Scenario C. The model offers both Undo and Edit recipient. Stop condition: two primary actions, no owner. Success measure: one action, one hand-back, one record of the discarded option.

I read those scenes out loud with a teammate. We do not act kind to the copy. Kindness is not a stop condition.

STOP if copy_claims_undo && !undo_api_exists
STOP if undo_window_ms is null && copy mentions a timer
STOP if focus_return_target is empty
HAND BACK if the user must pick a new recipient
NOISE: extra synonyms, mascot lines, model confidence

Verification: I can fail the draft with one of those STOP lines. If every draft still passes, my stops are too soft. Tighten them before any user session.

Stage 6: Accessibility review of the recovery path

Recovery is not a visual badge. It is a path back to control. I check the path like a person who cannot see the toast.

Does the announcement name the failed action? Does it avoid saying Undo when undo is false? Does focus return to the control that caused the send? Does a timer exist only when a real window exists?

# proposed live-region pattern, unexecuted
[role=status]
Send to Northwind already left this screen.
This send cannot be undone.
You are back on Send. You can edit the recipient.

I want that status before any button. A red inline error that never receives focus is a trap. A modal that steals focus and offers a fake Undo is worse.

Verification: keyboard only, then a screen reader pass. I tab from the failed control through the card and back. If I cannot return, the pattern fails. If the reader says Undo and the API cannot, the pattern fails. No extra copy will fix that.

Stage 7: Paste only after the card flips

The card stays red until proof exists. Proof is not a prettier synonym. Proof is undo_api_exists measured against the contract, plus a focus target, plus a honest first sentence.

// proposed check, unexecuted
function mayPasteRecovery(card) {
  if (card.copy_claims_undo && card.undo_api_exists !== true) {
    return { ok: false, stop: "undo claimed without api" };
  }
  if (!card.failed_action) {
    return { ok: false, stop: "failed action missing" };
  }
  if (!card.focus_return_target) {
    return { ok: false, stop: "no focus return" };
  }
  if (card.evidence_vs_hypothesis.evidence.length === 0) {
    return { ok: false, stop: "no evidence listed" };
  }
  return { ok: true, stop: null };
}

Verification: I run the function against a lying draft and a honest draft. The lying draft must fail. The honest draft may still fail on focus. That is correct. Shipping without a return path is another kind of lie.

After a pass, I paste one sentence. I keep the discarded candidates in the decision record. Future me will thank present me. Future me is messy.

What this protocol is not

It is not a benchmark of free models. I have no latency table for you. I have no token quota to advertise. I have no claim that a free server makes recovery safe.

It is not permission to skip users. Tabletop stops are cheaper than harm. They are not a substitute for a session when undo is real and timed.

Who should not use this approach? Anyone hoping the model will invent a reverse API. Anyone pasting recovery copy into a design system without a contract check. Anyone who needs production uptime from a free scratch server. Anyone who treats outreach drafts as research evidence.

The useful question stays small. What missing proof should stop this paste? What extra sentence would only add noise? If you cannot answer both, do not ask a model for the label yet.

Show Stop Proof Before Approving an Agent Retry

Haley — Sat, 12 Sep 2026 17:04:39 +0000

The retry button looked harmless during Friday critique. An agent had rewritten onboarding consent copy overnight. The first pass missed a recovery path. Now the agent wanted another loop. Who should click approve with that hole still open?

I own that click as the reviewing designer. Another loop can overwrite a human edit. Reversibility dies once the retry is queued. I refuse the retry until stop proof exists on the card.

This is not a speech about trusting models. It is a from-zero workflow you can run today. You leave with a review card and a validator. You also leave knowing which missing evidence must stop approval.

Start from the failed loop, not the model

I do not open a playground first. I open a dated research folder instead. The failed loop is the artifact under review. The model is only a later witness, never the owner.

mkdir -p research/retry-2026-09-13
cd research/retry-2026-09-13
touch decision-card.yaml flow.md recovery.md

Verification is boring on purpose, and that helps. The folder must exist before any critique starts. The three files must be empty and correctly named. If any file is missing, I stop the session.

Why three files for one retry decision? One card holds the human decision. One note holds the moment of choice. One note holds failure recovery for assistive tech. Mixing them hides missing evidence inside chat. Have you ever approved a retry from a thread? That thread is not a record.

Fill the decision card before any retry

I write the card in plain YAML. Designers can read it without a decoder. Engineers can lint it without a new app. The fields are the interface, not decoration.

decision: retry_agent_loop
owner: product_design
surface: onboarding_consent
reversible_until: before_retry_queued
user_consequence: overwritten_consent_copy
evidence:
  observed_failure: ""
  discarded_options: []
  source_of_failure: ""
  last_human_edit: ""
hypotheses:
  - ""
stop_conditions:
  - ""
a11y:
  rewrite_announced: false
  refusal_named: false
  focus_return: false
retry_allowed: false

Verification is a read-back with the owner present. Every required field that is empty must stay visible. I never default retry_allowed to true. Empty means stop, not “fill later.” What extra field would only add noise here? Model nickname. Sample temperature. A confidence badge. Those do not change the human decision.

Separate evidence from a clever guess

Evidence is what a person observed on the surface. A hypothesis is a story about why it happened. I keep them in different keys on purpose. That split sounds petty until a fluent model launders a guess.

I fill a walkthrough from that Friday scene. Treat it as a labeled protocol. It is not a published study with metrics.

evidence:
  observed_failure: "Consent line offered no refusal after rewrite."
  discarded_options:
    - "Keep the original human sentence."
    - "Ask the user before rewriting consent."
  source_of_failure: "Agent treated consent as marketing copy."
  last_human_edit: "Design owner, 2026-09-12, recovery sentence intact."
hypotheses:
  - "The loop over-weighted brevity."
  - "The tool lacked a consent style rule."

Verification is a two-column read, out loud. Left column is evidence. Right column is guess. If I cannot split a sentence, it is not evidence. Would you let a guess stop a legal consent line? I would not, and neither should the card.

Ask this with the team before anyone queues work. Which missing evidence should stop approval right now? Which extra note is only noise? I stop for a missing last_human_edit. I do not stop for a missing logit.

Draw the flow at the moment of choice

I sketch the flow in Mermaid next. The retry click sits in the middle. Everything else is a gate with an owner.

flowchart TD
  fail[Loop fails on user-facing copy] --> card[Owner opens decision card]
  card --> ev{Evidence slots filled?}
  ev -->|no| stop1[Refuse retry]
  ev -->|yes| sc{Stop conditions named?}
  sc -->|no| stop2[Refuse retry]
  sc -->|yes| rec{Recovery proven for AT?}
  rec -->|no| stop3[Refuse retry]
  rec -->|yes| critique[Optional second-pass critique]
  critique --> human[Human approves or refuses]
  human -->|approve| queue[Queue retry]
  human -->|refuse| handback[Hand back to owner]

Verification is a tabletop walk, not a slide review. I point at each diamond and name the owner. I name the reversible point in one sentence. If a diamond has no owner, the flow is fiction. Can you point to the last safe moment? If not, do not queue the loop.

The analogy I use is a fire door. A retry is not a hallway you stroll through. It is a door with a closer and a latch. You do not prop it open with hope.

Write stop conditions that can actually fire

Vague stops never fire when the room is tired. “Be careful with consent” is not a condition. I write conditions as observable states on the screen.

stop_conditions:
  - "Consent copy changed without a visible refusal."
  - "Last human edit would be overwritten."
  - "Rewrite is announced without naming the field."
  - "Refusal control has no accessible name."
  - "Source of failure is still unlabeled."

Then I write success beside those stops in flow.md. Success is not “the model sounds better.” Success is a recoverable interface after a bad rewrite.

I keep both measures in prose so nobody hunts a wiki. Success means the refusal stays in the consent line. Success means the human edit remains in the record. Success means focus returns to the edited field. Success means a retry is refused when any stop fires. Stop means any evidence slot is still empty. Stop means any recovery flag is still false. Stop means the person clicking is not the owner.

Verification is a dry run with a stranger in the room. I read each stop aloud once. I ask whether that stranger could see it fire. If only the author can see it, I rewrite the stop. A private stop is just a wish.

Prove recovery before the model speaks again

Agent copy is still an interface, not a paragraph. A rewrite without recovery is a trap. I check three things only on this gate. Announced rewrite. Named refusal. Focus return.

I write recovery.md as sentences, not a dumped audit. The rewrite must be announced with the field name. The refusal control must expose a real accessible name. Focus must return to the edited field, not a banner. The discarded human sentence must remain readable in the record. I do not treat this as a full accessibility audit. It is a retry gate. Missing any line keeps retry_allowed false.

Verification is a keyboard and AT pass on the current screen. Does the announcement name the field that changed? Can I find the refusal without guessing the icon? If someone says we will fix recovery after retry, that is the failure. After is not a recovery. After is a hope with a release date.

Only then run a cheap second-pass critique

Sometimes I want a second reader, not a second actor. I want a critique that cannot queue the retry. That is the only place a disposable model lane belongs.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. In this workflow, MonkeyCode is only that disposable critique lane. Free model access and a free server option are enough for rehearsal. I do not let that output write the card. I do not let it flip retry_allowed. If you already have an internal critic, skip this stage.

I paste a frozen bundle into the critique lane. The card, the flow, and the recovery note travel together. I ask for missing stops, not prettier copy.

You are a critic, not an actor.
Do not rewrite the consent line.
List evidence slots that are still empty.
List stop conditions that cannot be observed.
List recovery that is still unproven for assistive tech.
If you cannot point to a field, say UNKNOWN.

Verification is refusal by the critic. The run must not produce new user copy. If it rewrites the sentence, I discard the whole run. The point is coverage, not fluency. Why invite a model that cannot act? Because I need a noisy highlighter. I do not need another author with a retry button.

Lint the card so blanks cannot hide

Chat will hide blanks with a confident tone. A linter will not do that favor. I keep a small validator next to the card.

# verify_retry_card.py
from pathlib import Path
import sys
import yaml

REQUIRED_EVIDENCE = [
    "observed_failure",
    "source_of_failure",
    "last_human_edit",
]
REQUIRED_A11Y = ["rewrite_announced", "refusal_named", "focus_return"]

def main(path):
    card = yaml.safe_load(Path(path).read_text())
    errors = []
    if not card.get("owner"):
        errors.append("owner missing")
    if not card.get("stop_conditions"):
        errors.append("stop_conditions empty")
    if not card.get("evidence", {}).get("discarded_options"):
        errors.append("discarded_options empty")
    for key in REQUIRED_EVIDENCE:
        if not card.get("evidence", {}).get(key):
            errors.append(f"evidence.{key} empty")
    for key in REQUIRED_A11Y:
        if card.get("a11y", {}).get(key) is not True:
            errors.append(f"a11y.{key} not proven")
    if card.get("retry_allowed") is True and errors:
        errors.append("retry_allowed true while card is incomplete")
    if errors:
        print("BLOCK")
        print("\n".join(errors))
        sys.exit(1)
    print("READY")

if __name__ == "__main__":
    main(sys.argv[1])

pip install pyyaml
python verify_retry_card.py decision-card.yaml

Verification is the word READY and nothing else. BLOCK means the retry stays closed. I paste the BLOCK list into the research note. That list is the missing evidence. It is not a backlog for a later sprint.

Hand the work back, then record the refusal

Approval is not the only honest ending. Refusal is a product event with an owner. I write the hand-back in the same folder so the next loop cannot pretend it never happened.

HANDBACK
owner: product_design
state: retry_refused
reason: rewrite_announced false
next_human_action: restore last_human_edit
discarded_retry_prompt: archived, not deleted

Why keep the discarded prompt in the record? Because the next loop will try it again. Memory without rejection is amnesia with a smile. Teams retry until the sentence sounds right. That is how consent copy becomes an ad.

Verification is a named next action, not a vibe. Someone listed must own the restore. An unnamed owner is itself a stop. If the discarded prompt is deleted, I treat the record as broken.

Research the retry click with three scenarios

I run this as a scenario study, not a hallway poll. Three scenarios are enough for a rehearsal. Scenario A drops the consent refusal. Scenario B would overwrite the last human edit. Scenario C announces “Updated” without naming the field.

For each scenario I measure two things only. Did the validator print BLOCK? Did the human refuse without extra chatter? Extra chatter is noise around a missing field. A missing last_human_edit should stop the room. A missing model nickname should not.

I do not count fluency of the second pass. I do not count minutes saved in critique. Those metrics hide the decision under a productivity story. The only success is a recoverable no. The only stop that matters is an unproven recovery.

Limitations, and who should not use this

This workflow is a gate in the interface. It is not a model evaluation harness. It will not rank providers. It will not prove a design is finished. The validator is a proposal you can fork. It has not been a controlled study.

Do not use it for medical or legal advice surfaces. Do not use it when no designer owns the click. Do not use it to rubber-stamp a free critique. A cheap second pass is optional coverage. It is never evidence. If you cannot name the reversible point, stop. If you cannot name the user consequence, stop. The retry button can wait.

I still want one question on your desk after the lint. Which missing evidence should stop approval today? Which extra field would only add noise to that card?

Show Source Proof Before Approving Memory Writes

Haley — Fri, 11 Sep 2026 15:00:26 +0000

A designer sits with a review card open. The agent wants to store one teammate preference. Shared memory will treat that line as durable truth. The source slot on the card is empty. Can you feel how cheap that yes looks?

The decision owner is the designer on call. The consequence is a false fact other agents will quote. The point of reversibility sits before the write. After the write, cleanup becomes a rumor with no owner.

This is a research protocol, not a shipping memoir. I label every example as unexecuted. I separate evidence from design hypotheses. I will not invent a customer, a metric, or a win.

Sometimes I need a cheap rehearsal box for the card. I use MonkeyCode only as that box. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The operator-supplied claims I will use are two. Free model access is available for rehearsal. A free server option is available for the same. I do not invent model names here. I do not invent quotas, hardware, or duration. I do not invent benchmarks or permanence. Remove the product and this protocol still holds.

Stage 0: Name the write and the stop

I start by naming the exact human decision. Who may let an agent write memory? What happens if that sentence is wrong? Where can a later teammate reverse it without theater?

I write those three answers into a local file first. If any answer is fuzzy, I stop the whole rehearsal. Missing ownership is evidence. Charming copy is not evidence.

mkdir -p memory-review/{cards,logs,fixtures}
cat > memory-review/decision.md <<'EOF'
Decision: approve or refuse one shared-memory write
Owner: on-call product designer
Consequence: other agents may quote this as fact
Reversal point: before persist, plus a named undo record
Stop if: source missing, date missing, or undo unnamed
EOF
wc -w memory-review/decision.md

Verification for this stage is boring on purpose. The file must name owner, consequence, and reversal. If wc returns a slogan instead, I rewrite it. Would you approve a write with no owner on the card?

Stage 1: Build the review card, not the memory

I do not let the agent draft the memory first. I freeze a card with empty evidence slots. Empty slots are the interface. Filled prose is a later privilege.

# memory-review/cards/review-card.template.yml
write_id: ""
proposed_sentence: ""
audience: teammates_and_later_agents
evidence:
  source_url_or_doc: ""
  source_date: ""
  quote_span: ""
  last_human_who_confirmed: ""
recovery:
  undo_command_or_path: ""
  discarded_drafts_log: "logs/discarded.md"
accessibility:
  card_name: "Memory write review"
  stop_reason_text: ""
hypotheses_not_evidence:
  - ""
status: blocked

I copy the template for one scenario. I leave every evidence field blank on purpose. The first verification is visual and mechanical. Blank must look like blank, not like soft gray hope.

cp memory-review/cards/review-card.template.yml \
  memory-review/cards/scenario-preference.yml
grep -n '""' memory-review/cards/scenario-preference.yml

If grep finds no empty quotes, the card already lied. I throw it away. Why would we hide the missing source inside pretty defaults?

Stage 2: Fail closed before any model speaks

I add a small validator that refuses incomplete cards. This is not production software. It is a stop condition you can run. Treat it as proposed code.

# memory-review/validate_card.py
import sys, yaml
from pathlib import Path

REQUIRED = [
    ("evidence", "source_url_or_doc"),
    ("evidence", "source_date"),
    ("evidence", "quote_span"),
    ("evidence", "last_human_who_confirmed"),
    ("recovery", "undo_command_or_path"),
]

def load(path):
    data = yaml.safe_load(Path(path).read_text())
    if not isinstance(data, dict):
        raise SystemExit("card is not a mapping")
    return data

def empty(value):
    return value is None or str(value).strip() == ""

def main(path):
    card = load(path)
    missing = []
    for group, key in REQUIRED:
        if empty(card.get(group, {}).get(key)):
            missing.append(f"{group}.{key}")
    if empty(card.get("proposed_sentence")):
        missing.append("proposed_sentence")
    if missing:
        print("STOP. Missing evidence:")
        print("\n".join(missing))
        sys.exit(2)
    if card.get("status") == "approved":
        print("STOP. Status cannot start as approved.")
        sys.exit(2)
    print("CARD INCOMPLETE-OK: required slots still block write")

if __name__ == "__main__":
    main(sys.argv[1])

python3 memory-review/validate_card.py \
  memory-review/cards/scenario-preference.yml
echo "exit:$"

Verification is the non-zero exit. A passing validator on an empty card would be a bug. I want the stop to be louder than the model. Does your current agent UI fail this loudly?

Stage 3: Fixture the dangerous sentence

I keep the first rehearsal away from live teammates. I write a fixture that looks tempting and under-sourced. The agent will love it. That is the point.

cat > memory-review/fixtures/preference.json <<'EOF'
{
  "proposed_sentence": "Alex prefers terse standups and hates written recap.",
  "claimed_source": "someone said it last week",
  "quote_span": "",
  "source_date": ""
}
EOF
python3 - <<'PY'
import json
from pathlib import Path
p = json.loads(Path("memory-review/fixtures/preference.json").read_text())
assert p["quote_span"] == ""
assert p["source_date"] == ""
print("fixture is unsafe, as intended")
PY

The fixture is a research scenario, not a person. I do not claim Alex exists on my team. I need a sentence that would spread if approved. Would you let that line become shared knowledge tonight?

Stage 4: Rehearse the model against the card

I now allow a model to fill a draft card. I still forbid persist. The model may propose text. The model may not change status. That boundary is the whole tutorial.

If you rehearse on a free server, keep it isolated. Do not point it at real memory. Do not paste secrets into the prompt. The free model is a pressure tool. It is not a witness.

cat > memory-review/prompts/fill-card.txt <<'EOF'
Fill review-card fields from the fixture only.
Do not invent a source_url_or_doc.
Do not invent source_date or quote_span.
If a required field is unknown, leave it empty.
Never set status to approved.
Return YAML only.
EOF

# Proposed rehearsal only. Swap in your isolated runner.
# Keep network away from production memory stores.
python3 - <<'PY'
from pathlib import Path
card = Path("memory-review/cards/review-card.template.yml").read_text()
fix = Path("memory-review/fixtures/preference.json").read_text()
prompt = Path("memory-review/prompts/fill-card.txt").read_text()
Path("memory-review/logs/model-input.md").write_text(
    "# Model input\n\n" + prompt + "\n\n## card\n" + card +
    "\n\n## fixture\n" + fix
)
print("model input frozen for review")
PY

Verification here is the frozen input log. I read it before any call. If the prompt asks the model to be helpful, I rewrite it. Helpful is how empty sources get invented. Do you want a helpful liar in shared memory?

After the model returns YAML, I save it as a draft, not a write.

# Save whatever the model returned, then force a second validate.
# Example path only. Replace with your isolated output file.
cp memory-review/cards/scenario-preference.yml \
  memory-review/cards/scenario-preference.draft.yml
python3 memory-review/validate_card.py \
  memory-review/cards/scenario-preference.draft.yml || true

If the draft still misses source, date, quote, or undo, I keep status: blocked. I do not negotiate with the sentence. The sentence is not the evidence.

Stage 5: Record discarded drafts on purpose

Approved memory without rejected wording is a false archive. I keep the discarded line in a concrete record. Later agents should see what we refused. Otherwise they will offer the same rumor again.

cat >> memory-review/logs/discarded.md <<'EOF'
## write_id: scenario-preference
Refused sentence: Alex prefers terse standups and hates written recap.
Missing: source_url_or_doc, source_date, quote_span
Human: blocked persist
EOF
grep -c "Refused sentence" memory-review/logs/discarded.md

Verification is a count greater than zero after a refusal. A clean log after a stop is a design smell. We did not fail privately. We failed on the record. Where do your discarded agent claims live today?

User flow I actually test

flowchart TD
  A[Agent proposes a memory write] --> B[Render review card with empty slots]
  B --> C{Required evidence present?}
  C -->|No| D[Stop and log discarded draft]
  C -->|Yes| E{Source is primary and dated?}
  E -->|No| D
  E -->|Yes| F{Undo path named and readable?}
  F -->|No| D
  F -->|Yes| G[Human edits or approves]
  G --> H[Persist write plus discarded record]
  D --> I[Hand back to human with stop text]

I walk this flow with a keyboard only. I do not trust a demo click. If stop text exists only in color, the flow is incomplete. The hand-back is part of the product. Silence is not a recovery pattern.

Research scenarios and stop measures

I run three scenarios on the same card. I do not mix them in one sitting. Each scenario has a success measure and a stop measure. I write those before I look at model output.

Scenario A is an under-sourced preference, like the fixture. Success means the human never sees an Approve shortcut. Stop means persist is unreachable while source slots are empty.

Scenario B is a sourced but stale note. The date is older than the team agreed. Success means the card shows age in text, not tint. Stop means approval stays blocked until a human reconfirms.

Scenario C is a well-sourced sentence with no undo path. Success means recovery is a required field. Stop means a beautiful quote cannot bypass undo. Which of those stops would your current agent ignore?

I collect coverage, not vibes. Did the empty slot stay empty under pressure? Did the model invent a URL? Did the reviewer try to fill noise instead? Invented URLs are a stop. Extra biography of the teammate is noise.

Accessibility review of the card itself

The card is an interface, not a screenshot. It needs a name, a stop reason, and a live rejection. Color cannot be the only blocked signal. Iconography cannot be the only undo signal.

cat > memory-review/a11y-check.md <<'EOF'
Name: Memory write review, not Agent suggestion
Role: region or form, announced as blocking
Stop text: plain language, not red alone
Undo: text path plus control name
Focus: first empty required slot, not the model prose
Motion: no auto-advance toward Approve
EOF

I then rehearse with a screen reader if I have one. If I do not, I still refuse color-only stops. I read the card aloud. If Approve sounds easier than Stop, the copy failed. Who is this card actually for, the model or the human?

Proposed interface fields stay small. source_url_or_doc. source_date. quote_span. last_human_who_confirmed. undo_command_or_path. stop_reason_text. Extra model confidence scores are noise on this card. Extra personality sliders are noise. Missing quote span should stop approval. Another synonym for the same sentence should not.

Evidence versus hypotheses

Evidence is a dated primary source and a quote span. Evidence is a named human who confirmed it. Evidence is an undo path that a teammate can follow. Hypotheses are tone, usefulness, and team morale. Hypotheses do not unlock persist.

I keep hypotheses in their own YAML list. I never copy them into evidence. If a rehearsal transcript feels convincing, I label it hypothesis. A free-model run is not field research. A free server is not production traffic. Do not launder either into a finding.

Recovery after a bad yes

If a write already landed, the protocol changes shape. I still do not let the agent overwrite quietly. I require a correction card with the old sentence visible. Discarded truth matters as much as new truth.

CORRECTION CARD
Old sentence: ...
Old source: ... or MISSING
New sentence: ...
New source: required
Undo of the undo: named
Notify: who already consumed the old fact

Verification is notification, not just a new row. Shared memory that nobody rereads is a trap. Who already trusted the wrong line? If I cannot answer, I have not recovered. I have only decorated the log.

Limitations and who should not use this

This tutorial does not prove a model is truthful. It only proves a human saw empty slots. It does not replace legal review, security review, or consent review. It does not measure model quality. It measures whether persist is reachable too early.

Do not use this as a substitute for real user research. Do not use it on medical, legal, or financial facts without the named owner for those domains. Do not use it if no human will read the card. An unattended free model on an unattended server is not oversight. It is a loop with a costume.

Teams without a reversibility path should not store agent prose as memory. Teams that cannot name a decision owner should not run Stage 4. If you need speed more than a source, you do not need shared memory. You need a scratch pad that expires.

I end on the same question I started with. Which missing evidence should stop approval on your card? Which extra information would only add noise around a bad yes? If you need a cheap isolated box to rehearse those stops, MonkeyCode's free model access and free server option can host the validator. Treat that run as practice. Never treat it as proof.

Show Tab Order Proof Before Approving Agent States

Haley — Thu, 10 Sep 2026 04:00:42 +0000

I sat a designer in front of one review card. An agent had rewritten a hover state overnight. The card asked for a yes. Tab order proof was empty. Who owns that yes? The designer does. The consequence is a keyboard trap in production. The point of reversibility is this card, not a later hotfix.

This is a from-zero research tutorial. It is a protocol, not a shipped study. I label every check as unexecuted until you run it. I will not treat a fluent agent as a witness. Fluency is not evidence.

Why this decision, and not another? Agents love visual states. They describe hover, press, and selected. They forget the path a Tab key must walk. A pretty state with no focus proof is a locked door. Would you approve a door you cannot open?

Name the decision before you name the tool

Write the decision in one sentence. Keep the owner visible. Keep the stop condition visible. I use this line and nothing softer.

DECISION: May this agent state ship to the design system?
OWNER: product designer on the review card
CONSEQUENCE: keyboard users lose a recoverable path
REVERSIBILITY: reject now; do not patch after merge
STOP IF: tab_order_proof is empty OR focus_visible is unproven
NOISE IF: model confidence, token spend, or poetic rationale

Verify this stage. Read the line aloud. If you cannot name the owner, stop. If you cannot name the stop, stop. A tool cannot rescue a nameless decision.

I rehearse the protocol on a cheap bed. MonkeyCode offers free model access and a free server option. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I do not need a named model. I do not need a quota story. I need a place to fail the card in public.

Stage 1: scaffold a local review record

Create a folder. Treat it as the only source of truth. Do not keep proof in chat scrollback. Chat is a hallway. Hallways forget.

mkdir -p agent-state-review/records agent-state-review/fixtures
cd agent-state-review
printf '%s\n' 'review protocol v1' > README.txt
ls -la

Verify this stage. ls must show records and fixtures. If a file lives only in a model window, it does not exist. Would you defend that file in an incident review?

Now write the card schema. These fields are interface copy, not decoration. Evidence stays separate from guesses.

{
  "component": "FilterChip",
  "proposed_state": "hover-and-selected",
  "decision_owner": "",
  "tab_order_proof": {
    "status": "empty",
    "from_control": "",
    "to_control": "",
    "escape_control": ""
  },
  "focus_visible_proof": {
    "status": "empty",
    "contrast_note": "",
    "not_obscured_note": ""
  },
  "hand_back": {
    "needed": true,
    "reason": ""
  },
  "hypothesis": "",
  "approval": "blocked"
}

Save it as records/card.schema.json. Verify the keys exist with a dry read.

python3 - <<'PY'
import json
from pathlib import Path
card = json.loads(Path('records/card.schema.json').read_text())
need = ['decision_owner','tab_order_proof','focus_visible_proof','hand_back','hypothesis','approval']
missing = [k for k in need if k not in card]
print('missing', missing or 'none')
print('approval', card['approval'])
assert card['approval'] == 'blocked'
PY

If that assert fails, your default is already too kind. Kind defaults leak traps.

Stage 2: fill a failed scenario, not a happy path

Happy paths teach almost nothing. I seed a chip that looks selected on hover. The agent wrote a glow. It never said where Tab lands. It never said how Escape returns. That silence is the finding.

{
  "component": "FilterChip",
  "proposed_state": "hover-and-selected",
  "decision_owner": "product-designer",
  "tab_order_proof": {
    "status": "empty",
    "from_control": "",
    "to_control": "",
    "escape_control": ""
  },
  "focus_visible_proof": {
    "status": "empty",
    "contrast_note": "agent described a glow only",
    "not_obscured_note": ""
  },
  "hand_back": {
    "needed": true,
    "reason": "no keyboard path, no unobscured focus"
  },
  "hypothesis": "users will understand selection from color alone",
  "approval": "blocked"
}

Save as fixtures/filterchip-hover-selected.failed.json. Notice the split. The hypothesis can be poetic. The proof cannot. Color is not a path. A glow is not a focus ring.

Verify this stage with a diff of statuses.

python3 - <<'PY'
import json
from pathlib import Path
fx = json.loads(Path('fixtures/filterchip-hover-selected.failed.json').read_text())
print('tab', fx['tab_order_proof']['status'])
print('focus', fx['focus_visible_proof']['status'])
print('hand_back', fx['hand_back']['needed'])
assert fx['tab_order_proof']['status'] == 'empty'
assert fx['approval'] == 'blocked'
print('failed scenario is honest')
PY

If the fixture already says approved, you staged theater. Theater is not research.

Stage 3: encode stop conditions as a checker

People forgive empty fields under time pressure. A checker does not. This script is a proposed gate. Label it unexecuted until you run it on your card.

# check_review_card.py — protocol checker, not a product claim
import json, sys
from pathlib import Path

NOISE = ("confidence", "tokens", "rationale_poem")

def load(p):
    return json.loads(Path(p).read_text())

def stop_reasons(card):
    reasons = []
    tab = card.get("tab_order_proof", {})
    focus = card.get("focus_visible_proof", {})
    if not card.get("decision_owner"):
        reasons.append("no decision owner")
    if tab.get("status") != "proven":
        reasons.append("tab order unproven")
    if not tab.get("escape_control"):
        reasons.append("no escape control")
    if focus.get("status") != "proven":
        reasons.append("focus visible unproven")
    if not focus.get("not_obscured_note"):
        reasons.append("focus may be obscured")
    if card.get("approval") == "approved" and reasons:
        reasons.append("approval contradicts empty proof")
    return reasons

def main(path):
    card = load(path)
    reasons = stop_reasons(card)
    print("component", card.get("component"))
    print("hypothesis_is_not_evidence", bool(card.get("hypothesis")))
    if reasons:
        print("STOP")
        for r in reasons:
            print("-", r)
        sys.exit(2)
    print("PASS: proof present, still needs a human yes")

if __name__ == "__main__":
    main(sys.argv[1])

Run it on the failed fixture. Expect a stop. A green check here would be a bug in the protocol.

python3 check_review_card.py fixtures/filterchip-hover-selected.failed.json
echo "exit:$?"

Verify this stage. Exit code must be 2. The word STOP must print. If you only read the hypothesis, you approved a rumor.

What extra fields would only add noise? Model confidence. Token spend. A paragraph about being helpful. Those fields soothe the reviewer. They do not move a Tab key. I keep them out of the card on purpose.

Stage 4: walk the user flow, then the keyboard flow

The visual flow is the easy story. A person clicks a chip. The chip glows. Filters update. That story is incomplete. The keyboard flow is the real product.

flowchart TD
  A[Reviewer opens agent state card] --> B{tab_order_proof proven?}
  B -->|no| C[Stop. Hand back to designer]
  B -->|yes| D{focus visible and unobscured?}
  D -->|no| C
  D -->|yes| E{escape control named?}
  E -->|no| C
  E -->|yes| F[Human may approve or still refuse]
  C --> G[Agent may retry only inside the card]
  F --> H[Record discarded paths beside the yes]

Read the diagram as a conversation. The agent speaks first. The card interrupts. The human speaks last. If the human speaks first, you have theater again.

I still need a keyboard script. This is a research script, not a browser driver. A facilitator reads it. A participant uses only Tab, Shift+Tab, Enter, and Escape. No mouse. No trackpad. No charity.

SCENARIO: Approve or reject FilterChip hover-and-selected
START: focus on the search field before the chip row
TASK: select Cuisine, then leave the row without a mouse
SUCCESS: chip is selected, focus ring visible, escape returns to search
STOP: focus disappears, order skips the chip, or escape does nothing
DO NOT HINT: do not point at the glow

Verify this stage by timing silence. If the facilitator explains the glow, the scenario is contaminated. Contaminated runs cannot support approval.

Which missing evidence should stop approval? A blank from_control. A blank escape_control. A focus ring that a sticky header covers. Which extra information only adds noise? Screenshots of hover without focus. A transcript of the model apologizing. An aesthetic preference from a sighted reviewer.

Stage 5: accessibility review as a gate, not a garnish

I ground the gate in published criteria, not vibes. WCAG 2.2 Success Criterion 2.4.7 Focus Visible asks that the keyboard focus indicator can be seen. WCAG 2.2 Success Criterion 2.4.11 Focus Not Obscured (Minimum) asks that the focused item is not entirely hidden. Those pages are the primary sources. I am not inventing a third rule.

Map each criterion onto a card field. Then refuse to merge on a mapped miss.

2.4.7  -> focus_visible_proof.status must equal proven
2.4.11 -> focus_visible_proof.not_obscured_note must name the overlapping chrome
2.1.1  -> tab_order_proof must name from, to, and escape
recovery -> hand_back.reason must stay in the record after a retry

A proposed pass fixture looks like this. It is still not an approval. Proof only unlocks a human decision.

{
  "component": "FilterChip",
  "proposed_state": "hover-and-selected",
  "decision_owner": "product-designer",
  "tab_order_proof": {
    "status": "proven",
    "from_control": "SearchInput",
    "to_control": "FilterChip:Cuisine",
    "escape_control": "SearchInput"
  },
  "focus_visible_proof": {
    "status": "proven",
    "contrast_note": "2px ring against surface, not hue-only",
    "not_obscured_note": "ring clears sticky filter header"
  },
  "hand_back": {
    "needed": false,
    "reason": "keyboard path demonstrated in scenario"
  },
  "hypothesis": "users will understand selection from color alone",
  "approval": "blocked"
}

See the last field? I still keep approval blocked. The checker can pass proof and still refuse to ship. A human must type the yes. Why? Because proof of a path is not proof of the right path.

python3 check_review_card.py fixtures/filterchip-hover-selected.proof.json
echo "exit:$?"

Verify this stage. Exit code should be 0 only when proof fields are complete. approval may remain blocked. That tension is the product.

Stage 6: hand-back and keep the discarded path

When the checker stops, the agent does not get the canvas. It gets the card. The designer writes the missing path in the same record. The failed order stays visible. Discarded paths are not shame. They are the map of near misses.

HAND_BACK
- agent may fill tab_order_proof only
- agent may not flip approval
- designer may accept, rewrite, or kill the state
- discarded order remains under records/history/

Copy the failed fixture into history before any retry. If you overwrite, you teach the team to hide bruises.

mkdir -p records/history
cp fixtures/filterchip-hover-selected.failed.json \
   records/history/filterchip-hover-selected.failed.001.json
ls records/history

Verify this stage. History must contain the empty proof. A later pass without that file is amnesia. Amnesia ships the same trap twice.

The recovery frame is small on purpose. Stop. Show the blank slots. Hand the work back. Keep the bruise. Resume only inside the card. That is the whole loop. It is closer to a coat check than a launch sequence. You do not walk out wearing a coat you never claimed.

What this protocol is not

It is not a benchmark of free models. I did not time tokens. I did not rank vendors. It is not a frontend implementation guide. Component code belongs elsewhere. It is not a claim that a free server makes states accessible. A server hosts the rehearsal. It does not see for you.

Who should not use this approach? Teams that already merge agent states from chat. Teams that treat confidence as contrast. Teams that cannot name a decision owner. Teams that need statistical usability claims. This tutorial does not produce those claims. It produces a stop.

Limitations sit in the open. The checker cannot detect a real obscured ring. A human still looks. The scenario cannot cover every control. You still sample. The free rehearsal bed can vanish or change. I do not claim duration, hardware, or permanence. If the bed moves, the card schema still travels. That is the point of putting proof in files.

I keep asking the same two questions at the end of every review. Which missing evidence should stop approval? Which extra information would only add noise? If your room cannot answer both, you are not ready for an agent to touch state. You are ready to keep the card blank, and that is honest work.

Show Assumed Account Facts Before Sending Agent Replies

Haley — Wed, 09 Sep 2026 15:14:10 +0000

The refund sat in the queue for twelve seconds. The agent had already named the plan. Nobody on the call had said Pro. Would you have caught that?

I almost approved the send. My eye went to tone, not facts. The draft sounded kind. It also invented a billing tier. That invention is the decision, not the prose.

This is a from-zero research tutorial. You will build a stop card. You will run three scenarios. You will verify each stage before moving. No study metrics hide in this piece. I am walking a protocol, not reporting a trial.

Think of the agent like a helpful intern with a stamp. The intern fills blanks to look finished. Your job is the stamp, not the handwriting. If a blank is still a blank, the stamp stays in the drawer.

Stage 0: Name the consequential send

Pick one outbound action. I use a support reply that can move money. The decision owner is the human reviewer. The consequence is a false promise in writing. The reversibility point is before Send, not after delivery.

Write that in one file. Keep the sentence ugly and specific.

mkdir -p agent-review-lab
cat > agent-review-lab/decision.txt << 'EOF'
Decision: approve or block a support reply that mentions plan, refund, or access.
Owner: human reviewer on the queue.
Consequence: customer receives an untrue account fact.
Reversal: only while the reply is unsent.
EOF
wc -w agent-review-lab/decision.txt

Verification is boring on purpose. You should see four short lines. If the file names a vibe instead of a send, stop. Rewrite until a lawyer could point at the verb.

What missing evidence should freeze that stamp? For this lab, freeze on plan name, refund amount, and access state. Extra color about tone is noise. Do you really need a sentiment score to block a fake Pro plan?

Stage 1: Build the evidence card, not the copy

I do not start in a chat window. I start with empty slots. The card is the interface. Copy is a downstream guess.

cat > agent-review-lab/evidence-card.json << 'EOF'
{
  "thread_id": "T-1842",
  "customer_visible_action": "send_support_reply",
  "slots": {
    "plan_name": {"value": null, "source": "empty", "stop_if_empty": true},
    "refund_amount": {"value": null, "source": "empty", "stop_if_empty": true},
    "access_state": {"value": null, "source": "empty", "stop_if_empty": true},
    "customer_goal": {"value": null, "source": "empty", "stop_if_empty": false}
  },
  "agent_draft": null,
  "reviewer_decision": "blocked_pending_evidence"
}
EOF
python3 -m json.tool agent-review-lab/evidence-card.json > /dev/null && echo "card ok"

You should see card ok. If JSON fails, the card cannot travel. Fix the file before any model sees it. An unreadable card is itself a stop condition.

Notice the analogy. The card is a coat check ticket. The draft is a coat. You do not hand over a coat without the ticket numbers matching. A pretty coat does not replace a missing number.

Hypothesis, labeled as such: reviewers miss invented plan names when the tone is warm. Evidence, labeled as such: we have none yet. This file only encodes the question.

Stage 2: Rehearse drafts in a cheap room

You need a place to generate bad fills without burning a production queue. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I use MonkeyCode here only as a rehearsal room with free model access and a free server option. I am not claiming model names, quotas, or speed.

Keep the product off the decision path. The server drafts. The card decides. If you cannot separate those, do not run the lab on customer traffic.

cat > agent-review-lab/prompt-stub.txt << 'EOF'
Write a support reply for thread T-1842.
Do not invent plan_name, refund_amount, or access_state.
If a slot is empty, keep it empty in a leading evidence block.
EOF
wc -l agent-review-lab/prompt-stub.txt

Verification: three instruction lines, no sample plan names. If your stub contains Pro, Team, or a dollar amount, you already poisoned the well. Delete those words. The intern copies whatever you leave on the desk.

Paste the stub into your rehearsal room. Save whatever comes back as a file. Do not paste it into the real queue.

# Replace the draft file with the model output you actually received.
cat > agent-review-lab/draft-01.txt << 'EOF'
EVIDENCE
plan_name: EMPTY
refund_amount: EMPTY
access_state: EMPTY
REPLY
I can look at the charge once we confirm the plan on the account.
EOF
grep -c "EMPTY" agent-review-lab/draft-01.txt

You want a count of three. Zero empty markers means the intern filled the coat check. That draft fails Stage 2 even if it sounds careful. Would you let a silent fill reach Send?

Stage 3: Run three scenarios, then halt

I use three scenes, not a survey. Each scene is a customer thread with one hole. You will mark stop or proceed after each. Do not average them into a score yet.

Scene A is a refund ask with no amount in the transcript. Scene B is an access ask with no current state. Scene C is a tone-only thank you with every slot still empty. Scene C is the trap. It feels harmless. It still cannot name a plan.

cat > agent-review-lab/scenarios.json << 'EOF'
[
  {"id": "A", "hole": "refund_amount", "transcript_has_plan": false},
  {"id": "B", "hole": "access_state", "transcript_has_plan": false},
  {"id": "C", "hole": "all_stop_slots", "transcript_has_plan": false}
]
EOF
python3 - << 'PY'
import json
from pathlib import Path
rows = json.loads(Path("agent-review-lab/scenarios.json").read_text())
assert len(rows) == 3
assert all(r["transcript_has_plan"] is False for r in rows)
print("scenarios ready")
PY

You should print scenarios ready. If any scene claims the transcript already named the plan, you are testing reading, not invention. That is a different study. Kill the scene.

Now watch one reviewer, maybe yourself. Start a timer only to keep the session honest. You are not measuring speed as success. You are watching whether the missing slot is spoken aloud before Send.

User flow I actually want on screen

[Queue item] -> [Evidence card with empty slots visible]
    -> [Agent draft beneath the card, never above it]
    -> [Stop if any stop_if_empty slot is empty]
    -> [Human names the hole or supplies a source]
    -> [Send] or [Hand back to agent with the hole named]
    -> [If sent in error, recovery note stays on the thread]

flowchart TD
  q[Queue item] --> card[Show evidence card first]
  card --> draft[Show agent draft second]
  draft --> check{Any stop slot empty?}
  check -->|yes| block[Block send and name the hole]
  check -->|no| source{Does every filled slot have a source?}
  source -->|no| block
  source -->|yes| send[Allow send]
  block --> handback[Hand back with the named hole]
  send --> record[Keep the card on the thread]

If the draft sits above the card, the coat hides the ticket. Move it. This is a layout hypothesis. Evidence for it is still only this inspection, not a study.

Stage 4: Score stops, not eloquence

Success is not a nicer paragraph. Success is a spoken hole. Stop is also a result. A stop that names refund_amount is a pass. A send that invents $49 is a fail. A long delay with no named hole is a fail too.

cat > agent-review-lab/score.py << 'EOF'
import json, sys
from pathlib import Path

card = json.loads(Path(sys.argv[1]).read_text())
stops = []
for name, slot in card["slots"].items():
    if slot.get("stop_if_empty") and (slot.get("value") in (None, "", "EMPTY")):
        stops.append(name)
decision = card.get("reviewer_decision")
if stops and decision == "send":
    print("FAIL invented-or-ignored-empty:", ",".join(stops))
    sys.exit(1)
if stops and decision.startswith("blocked"):
    print("PASS named-stop:", ",".join(stops))
    sys.exit(0)
if not stops and decision == "send":
    print("PASS sourced-send")
    sys.exit(0)
print("FAIL unclear-decision")
sys.exit(1)
EOF
python3 agent-review-lab/score.py agent-review-lab/evidence-card.json

On the empty starter card you should see a PASS for a named stop. If you flipped the decision to send without filling slots, the script must fail. Run it after every scene. Do not keep a mental tally. Memory is where invented Pro plans live.

Which extra field would only add noise? I leave customer_goal optional. It helps empathy. It does not prove the plan. If your team argues about empathy copy while plan_name is empty, the ritual already failed.

Stage 5: Accessibility of the stop, not the sparkle

A stop that exists only as a red gradient is not a stop. I check the card like a form, because it is a form. The reviewer may be on a screen reader, a zoomed view, or a keyboard-only queue.

Write the review surface as text first. Buttons need names that say the hole. Approve is a vague verb. Send anyway with empty plan_name is honest. Honest names slow the wrong click. That slowness is the point.

<!-- labeled pattern, not production UI -->
<section aria-labelledby="card-title">
  <h2 id="card-title">Evidence required before send</h2>
  <p>Plan name: empty. This slot blocks send.</p>
  <p>Refund amount: empty. This slot blocks send.</p>
  <p>Access state: empty. This slot blocks send.</p>
  <button type="button">Block send and name the empty slots</button>
  <button type="button" disabled>Send is unavailable while slots are empty</button>
</section>

Verification is a read-aloud. Sit with the HTML in a browser. Tab once to the block control. Confirm Send is not the first focus. If Send is first, you designed a trap. Fix the order before you decorate anything.

Also check contrast on the word empty. Empty must be a word, not a pale placeholder. Placeholders vanish. Vanishing evidence is how the intern wins.

Stage 6: Hand back and keep the discarded fill

When you block, do not wipe the invented text. Keep it in a discarded well. Future you needs to see the lie the model wanted to tell. Deleting it trains the team to forget the failure mode.

cat > agent-review-lab/handback.json << 'EOF'
{
  "thread_id": "T-1842",
  "named_holes": ["plan_name", "refund_amount", "access_state"],
  "discarded_agent_fill": {
    "plan_name": "Pro",
    "note": "unproven fill, not shown to customer"
  },
  "next_owner": "agent_with_holes_named",
  "customer_copy_status": "unsent"
}
EOF
python3 -m json.tool agent-review-lab/handback.json | head -n 5

You should still see unsent. If that field ever flips while holes remain, treat it as an incident. Recovery is a visible note on the thread, not a quiet edit. The customer may have seen nothing. Your future reviewer still needs the scar.

Failure recovery, in one breath: unsend if the channel allows it, correct the account fact in the next human message, and keep the discarded fill attached to the card. Do not hide the intern's stamp. Hiding it repeats the twelve-second almost-send.

What this lab cannot claim

This protocol does not rank models. It does not prove trust. It does not replace policy for legal, medical, or credit decisions. It only tests whether a human can see an empty slot before a send.

Skip this approach if you have no human on the queue. Skip it if Send cannot be disabled. Skip it if your channel has no reversibility and you still want the agent to name money. Those teams need a harder gate than a research card.

I also will not dress a hypothesis as a finding. Warm tone may hide invented plans. That is a guess I still need scenes to challenge. The files above are the challenge, not the answer.

So, what freeze rule will you keep? I keep this one. If plan_name has no source, the stamp stays down. Everything else can wait. The intern can keep talking in the rehearsal room. The customer should not hear a plan nobody spoke.

If you need a cheap room to practice the card before a real queue, MonkeyCode's free models on the free server are enough for that rehearsal. Bring the stop file with you. Leave the invented Pro plan in the discarded well.

Gate Agent Landmarks Until the Skip Path Is Proven

Haley — Sun, 06 Sep 2026 20:20:42 +0000

Last Tuesday I almost shipped a skip link into nowhere. The agent guessed three landmarks from a static mock. I owned the approve click. Checkout sat one skip away from vanishing. Could I reverse the rewrite after publish? No. That missing reverse path is the whole story.

This is not an essay about trusting models. It is one consequential decision. I am the decision owner. The consequence is a keyboard user leaping past pay. The point of reversibility must exist before I click approve. Visual layout is a poster. Landmarks are a trail. Why would a model blaze that trail from pixels?

I now run a from-zero gate before any agent rewrites navigation. Treat this as a proposed tutorial, not a finished study. Each stage has a command and a check. Fail a check, stop. Do not negotiate with an empty slot.

Stage 1 — write the decision card

I refuse to open the model until the card exists. The card is dull on purpose. Dull cards catch glamorous guesses. Save it as gate/decision-card.json.

{
  "decision": "approve_inferred_landmarks",
  "owner": "design_lead_on_call",
  "agent_action": "rewrite_landmarks_and_skip_link",
  "consequence_if_wrong": "sr_and_keyboard_users_skip_past_checkout",
  "reversibility": "restore_original_map_from_review_record",
  "evidence_required": [
    "original_landmark_map",
    "proposed_landmark_map",
    "skip_path_transcript",
    "primary_task_success"
  ],
  "empty_slots_block_approval": true,
  "hypotheses_are_not_evidence": true
}

Verification: read the file aloud. Can you name the owner in one breath? Can you name the harm? Can you name the restore handle? If any answer turns poetic, rewrite the card. Do not proceed. Poetry is not a control.

Stage 2 — draw only the human click

I sketch the moment a person must decide. I do not sketch the model's inner loop. The agent may draft. The human still owns publish. Here is the flow I actually use.

flowchart TD
  A[Agent proposes landmark rewrite] --> B{Evidence file complete?}
  B -->|No| C[Hold publish and show empty slots]
  B -->|Yes| D[Human runs skip-path rehearsal]
  D --> E{Primary task reachable in two skips?}
  E -->|No| F[Reject and keep original map]
  E -->|Yes| G[Approve with restore handle]
  F --> H[Log discarded inference in review record]
  G --> H

Verification: put your finger on the diamond that stops publish. If your drawing has no stop, you drew a pipeline. Pipelines do not protect checkout. Would you board a train with no brake lever?

Stage 3 — write two scenarios, not ten

I keep research small so the gate stays honest. Scenario A is the proven skip. Scenario B is the inferred footer. Both live in gate/scenarios.md. The file is the protocol. It is not a slide.

# Landmark gate scenarios

## A — proven skip
Task: from article body, reach checkout.
Success: two skips, visible focus, spoken name matches the label.
Stop if: landing node is not checkout or main.

## B — inferred footer
Task: same purchase after the agent rewrites landmarks.
Stop if: proposed map drops the original main target.
Noise to ignore: extra decorative headings nobody skipped.

Verification: ask the split out loud. Which missing evidence should stop approval? Which extra detail is only noise? If you cannot split those, you will over-research and still ship the wrong skip. More notes are not more safety.

I separate evidence from design hypotheses in the same breath. "Users might prefer a shorter skip list" is a hypothesis. A skip-path transcript is evidence. Do not let the first impersonate the second. The ARIA landmarks guide and bypass blocks describe the trail. They do not bless a visual guess.

Stage 4 — freeze the evidence file

I will not read a chat transcript as a record. Chat evaporates. The gate needs a file the validator can fail. Save the proposal as gate/evidence/proposal.json.

{
  "page_id": "checkout-help-article",
  "original_landmark_map": [
    {"role": "banner", "label": "Store", "id": "site-header"},
    {"role": "main", "label": "Help article", "id": "article"},
    {"role": "contentinfo", "label": "Legal", "id": "site-footer"}
  ],
  "proposed_landmark_map": null,
  "skip_path_transcript": null,
  "primary_task_success": null,
  "discarded_inferences": [],
  "restore_handle": "git:design/landmarks/checkout-help-article@original"
}

Notice the nulls. They are the point. An honest file shows what the agent does not know. Verification: open the file and count nulls. If a proposed map appears before a transcript, you inverted the order. Stop. The agent does not get to fill silence with confidence.

Stage 5 — run a local validator

This script is the original artifact. It does not score the model. It refuses approval when required slots are empty. It also refuses a proposal that drops main. Label it as proposed code. I have not claimed a production rollout.

# validate_landmark_gate.py — proposed local gate, not a model benchmark
import json, sys

REQUIRED = [
    "original_landmark_map",
    "proposed_landmark_map",
    "skip_path_transcript",
    "primary_task_success",
]

def roles(maps):
    return {item["role"] for item in maps or [] if "role" in item}

def main(path):
    data = json.load(open(path, encoding="utf-8"))
    missing = [key for key in REQUIRED if not data.get(key)]
    if missing:
        print("STOP: empty evidence slots:", ", ".join(missing))
        sys.exit(2)
    original = roles(data["original_landmark_map"])
    proposed = roles(data["proposed_landmark_map"])
    if "main" in original and "main" not in proposed:
        print("STOP: proposed map dropped main")
        sys.exit(2)
    if not data.get("restore_handle"):
        print("STOP: no restore handle, approval is irreversible")
        sys.exit(2)
    print("PASS: evidence complete; human may now rehearse the skip path")

if __name__ == "__main__":
    main(sys.argv[1])

Verification command:

python3 validate_landmark_gate.py gate/evidence/proposal.json

You should see STOP: empty evidence slots on the first run. That failure is success. The gate is awake. If the script prints PASS while transcripts are still imaginary, you broke the validator. Fix the script, not the ethics.

Stage 6 — rehearse the draft on a cheap bench

Only now do I let a model propose a map. I still do not let it publish. I need a place where a bad inference is cheap to throw away. MonkeyCode's free model access and free server option are the rehearsal bench I use for that draft. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I paste the proposed map into the evidence file. I never paste it into production HTML first.

After the draft lands, I fill proposed_landmark_map and nothing else. Then I rerun the validator. It must still stop on the empty transcript. That second stop is the lesson. A fluent proposal is not a skip-path proof. Did the model sound sure? Good. Certainty is not a landmark.

Verification: the evidence file still contains nulls for skip_path_transcript and primary_task_success. If those fields got auto-filled, the bench leaked a hypothesis into the record. Delete the fill. Keep the null.

Stage 7 — run the skip path like a person

I do not ask the model whether the skip works. I walk it. Keyboard only. Screen reader speaking. Two skips toward checkout. I type what I heard, not what I hoped.

# proposed capture — replace with your SR tool's own transcript export
printf '%s\n' \
  "skip-1: spoken Store, focus #site-header" \
  "skip-2: spoken Help article, focus #article" \
  "next: checkout link inside main, focus visible" \
  > gate/evidence/skip-path-transcript.txt

Then I copy that text into the JSON field and set primary_task_success to true only if checkout was reachable. If the spoken name disagrees with the visible label, success stays false. Mismatch is a stop, not a polish note. Verification: play the transcript to a colleague who cannot see the mock. If they cannot say where they landed, the evidence is theater.

Accessibility review at this stage is narrow on purpose. I check one main, unique labels, and a skip target that matches an id. I do not pretend this replaces a WCAG audit. I also do not hide recovery behind a tooltip. Focus must be visible because the human has to see the hand-back. Implementation styling is someone else's job. The pattern is mine: no approve click without a spoken, reversible trail.

Stage 8 — keep the discarded map in the record

Rejection is not deletion. If I refuse the proposal, I append it to discarded_inferences and restore the original map through the handle. The agent returns to draft. Publish stays human. That is the hand-back. Analogy time. A librarian does not burn the wrong catalog card. She shelves it as wrong, then keeps the old drawer.

{
  "discarded_inferences": [
    {
      "reason": "skip landed on footer, checkout unreachable",
      "proposed_landmark_map": [
        {"role": "banner", "label": "Store", "id": "site-header"},
        {"role": "contentinfo", "label": "Page end", "id": "site-footer"}
      ]
    }
  ]
}

Verification: restore the original map in one action. If restore takes a redesign, you never had reversibility. You had optimism. Optimism is not a recovery framework.

What this protocol is not

This gate does not prove the model is safe. It proves you refused to approve silence. Free model drafts can omit empty slots and still sound complete. That is why the validator lives on your machine. I am not publishing quotas, model names, or hardware claims. I have no benchmark to wave. If you need those numbers, this article is the wrong artifact.

Who should not use this approach? Anyone who cannot run a real skip-path rehearsal. Anyone shipping emergency, medical, or legal navigation from a visual guess. Anyone treating a free rehearsal bench as a production control plane. Anyone hoping a longer prompt will replace a restore handle. If your team cannot name a decision owner, stop before stage one.

I still ask the same two questions at the end of every review. Which missing evidence should stop approval? Which extra information would only add noise? For landmarks, an unproven skip path stops me. A debate about decorative headings does not. The agent can keep talking. Checkout does not have to listen.

If you already have a free rehearsal bench, run the validator against one live proposal before you bless a visual guess.

Show Empty Evidence Slots Before Approving Agent Copy

Haley — Sat, 05 Sep 2026 19:00:41 +0000

The approve button already had focus on my review card. I still had not asked a single hard question. An overnight agent pass had rewritten every settings label.

The agent stamped every line as accessible copy. Who actually owns that dangerous last click? I do, and a wrong click ships a lie.

Where does reversibility still live for this change? It lives only before publish. After publish, recovery turns into hallway folklore.

I almost treated fluent confidence as research evidence. Have you seen that finished-looking review card? It looks complete while every evidence slot stays hollow.

This is not another essay about trusting models. This is a from-zero research tutorial. We stop approval when required evidence slots are empty.

Stage 0: Name the decision, not the vibe

Write the live decision in one plain sentence. Mine read like this, on purpose.

"May this agent copy replace live field labels?"

Name the owner, the harm, and the last reversible moment. If those three names will not fit, stop now. You are styling a guess, not reviewing a change.

Verification comes from disagreement. Read the sentence to a teammate. Can that person refuse it without embarrassment? If nobody could refuse, you wrote a mood.

I keep this sentence at the top of the packet. It is not a vision statement. It is the only question the card may answer.

Stage 1: Build the evidence packet before the model runs

I refuse to let the agent write the design record first. I open an empty packet on purpose. Empty slots are the interface, not a chore.

Here is the packet I use in critique. Treat it as a proposed pattern, not a completed study.

# Proposed evidence packet. Not a field-study result.
decision_id: settings-labels-2026-09-06
decision: "Replace live labels with agent copy?"
owner: design-reviewer
reversible_until: before-publish-to-cms
original:
  labels: {}
  helper_text: {}
  alt_text: {}
agent_pass:
  environment: free-shared-server
  generated_at: null
  copy: {}
evidence:
  contrast_ratio: null
  screen_reader_script: null
  original_preserved: false
  purpose_confirmed_by_human: false
hypotheses:
  - "Shorter labels reduce scan time"
stop_conditions:
  - original copy missing
  - contrast_ratio empty
  - screen_reader_script empty
  - purpose_confirmed_by_human is false
noise_fields:
  - server hostname
  - token spend
  - model temperature

See the nulls sitting in public view? Those nulls are the product. An empty slot is a stop condition, not a personal failure.

Verification is a null count. Count them before any model runs. If you already have zero nulls, you prefilled a fantasy. Reset the file.

Separate evidence from design hypotheses in that same file. Contrast is evidence. "Users will like shorter labels" is a hypothesis. Mixing them is how a card starts lying.

Stage 2: Capture original copy like it might vanish

Agents overwrite in a polite voice. Original labels then vanish into a tiny diff. Recovery becomes a Slack legend nobody can prove.

I dump live strings before anyone prompts a model. It feels boring. Boring is the point.

mkdir -p packets/settings-labels
# Proposed capture. Point this at your copy deck or CMS export.
printf '%s\n' \
  'email: Email address' \
  'digest: Weekly digest' \
  'tone: Product updates may include offers' \
  > packets/settings-labels/original.txt
test -s packets/settings-labels/original.txt && wc -l packets/settings-labels/original.txt

Verification is a non-empty file. Every live control needs one original line. If a control has no original line, the agent cannot improve it. It can only invent it.

Would you approve invented helper text on a legal checkbox? I would not, and neither should the card. Invention is not a copy tweak. Invention is a new product claim.

Stage 3: Generate candidate copy in a cheap rehearsal room

I needed a sketch room that could not touch production. MonkeyCode offers free model access and a free server option. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

I use that room to generate candidates only. I never use it as a silent publisher.

Keep the prompt small and slightly rude. Ask for candidates. Forbid accessibility boasts.

cat > packets/settings-labels/prompt.txt <<'EOF'
Propose alternate labels for these controls.
Do not claim accessibility.
Do not delete the original wording.
Flag any purpose you are guessing.
Return YAML only.
EOF
cat packets/settings-labels/original.txt packets/settings-labels/prompt.txt

Verification is a forbidden claim. If the model still says the copy is accessible, log that sentence as a defect. Do not reward the boast with a merge.

Why rehearse on a free shared server at all? Because a sketch should be cheap to throw away. Because a stall should not hit your CMS. The packet is the work. The vendor is scenery.

If you already have a safe sandbox, stay there. Do not move your source of truth for this tutorial.

Stage 4: Score empty slots before anyone argues about tone

Tone arguments get loud in five minutes. Empty evidence stays quiet unless you score it first. Score the quiet thing.

# Proposed validator. Unexecuted until it reads a real packet.
from pathlib import Path
import yaml

packet = yaml.safe_load(Path("packets/settings-labels/packet.yaml").read_text())
stops = []
evidence = packet["evidence"]

if not packet["original"].get("labels"):
    stops.append("original labels missing")
if evidence.get("contrast_ratio") in (None, "", 0):
    stops.append("contrast_ratio empty")
if not evidence.get("screen_reader_script"):
    stops.append("screen_reader_script empty")
if evidence.get("purpose_confirmed_by_human") is not True:
    stops.append("human never confirmed purpose")

print("STOP" if stops else "READY")
for line in stops:
    print("-", line)

Then run it against the fresh packet.

python3 score_packet.py

Verification is a STOP on a new packet. READY would mean the script is lying, or you smuggled evidence in. Either case blocks approval.

This is the annotated review pattern I want in the design system. Four slots sit on the card: original copy, contrast, screen-reader script, and purpose confirmed. Color is never the only state. Each slot also speaks in text: Empty, Recorded, or Conflict.

Can a reviewer understand the card with grayscale on? If not, the card is decoration. Decoration is how bad copy gets a green chip.

Stage 5: Walk three research scenarios, then force a stop

I do not need a thirty-person study to gate this click. I need scenes that can fail in public. Write them before the critique starts.

First scene. The agent reads a field named tone and writes Marketing opt-in. The original line was a legal warning about offers. Does the packet stop that swap, or do people debate friendliness?

Second scene. The agent writes alt text from a filename, hero-final-v3. A screen-reader user then hears a version string. The script slot is empty. Do you still feel the card is mostly done?

Third scene. The free server returns a partial packet and drops helper text. The layout still looks tidy. Do you approve a tidy hole?

Success is a spoken refusal in under two minutes. Failure is a taste debate that never names a slot. If voice wins, the card failed its job.

Verification is a prewritten refusal. I write I refuse because contrast is empty. If a reviewer cannot say that line, rebuild the card before another session.

Notice what we are not measuring here. We are not scoring eloquence. We are not scoring how "on brand" the shorter line feels. Those scores are hypotheses. They can wait until the slots are full.

Stage 6: Test the review interface like it can harm people

An inaccessible review tool will bless inaccessible copy. That irony does not help a single user. Test the gate the way you test a checkout.

Keyboard order should hit Reject before it hits Approve. Stop reasons must exist as text, not as a red dot. Original copy cannot live behind a hover tooltip.

Never chip the status as accessible. Announce evidence incomplete instead. A flow only earns its place if it can stop someone.

flowchart TD
  A[Open review card] --> B{Original copy present?}
  B -->|No| S[Stop: do not approve]
  B -->|Yes| C{Contrast recorded?}
  C -->|No| S
  C -->|Yes| D{Screen-reader script present?}
  D -->|No| S
  D -->|Yes| E{Human confirmed purpose?}
  E -->|No| S
  E -->|Yes| F[Edit, then hand back to record]
  F --> G[Publish with original and new copy]

Verification is a keyboard pass on a mock card. Can you reject without a mouse? Can a screen reader hear the blocking reason? If either answer is no, you designed applause, not review.

I also check focus order after a stop. The reviewer should land on the empty slot, not on Approve. A focus trap on Approve is a product bug with manners.

Stage 7: Hand the decision back, and keep discarded lines

Approval is not closure. The discarded original must remain in a concrete record. Future you will need those lines on a bad Thursday.

ts=$(date -u +%Y-%m-%dT%H:%M:%SZ)
{
  echo "time: $ts"
  echo "decision: refused-or-accepted"
  echo "stop_reasons: contrast empty"
  echo "original_kept: yes"
} >> packets/settings-labels/decision.log
tail -n 5 packets/settings-labels/decision.log

Verification is a log that still holds original_kept. The CMS must still show live strings. You should reverse without digging through chat history like a detective.

Which missing evidence should stop approval cold? Contrast, original wording, the screen-reader script, and human purpose confirmation. Which extra information only adds noise? Server hostname, token spend, temperature, and a confidence number with no method.

Ask that pair out loud during critique. Teams will offer dashboards. Many dashboards are noise wearing a chart.

The hand-back is the humane part. The agent does not get to close the ticket. A person files the record, including what was thrown away.

Limits, and who should skip this

I am not claiming free models write worse labels than paid ones. I did not run that comparison. I am not claiming quotas, hardware, uptime, or lasting capacity. Those numbers are not in this article.

Skip this protocol if no human will read the packet. Skip it if originals, contrast notes, and screen-reader scripts already live in one record. Skip it if a human already wrote the copy and you are only delaying them.

Do not twist this tutorial into permission. The packet can block publish. It cannot bless publish.

If your labels touch medical or legal meaning, stop. This YAML is too light for those products. You need counsel and a real accessibility evaluation. A research card is not a lab.

I still want those empty slots in the open. Confidence is a costume that talks fast. Evidence is a door you can still close.

Keep production out of the sketch room. Rehearse the refusal while reversal is cheap.