Research Silent Assumptions With Stop Conditions

I almost clicked approve. The agent had drafted a refund notice. It looked careful. It even sounded kind. Then I noticed the shipping address. Who confirmed it? Nobody had. The agent had filled a silent gap, and I was one click from mailing it as fact.

That click is the decision. I own it, not the model. The consequence is a wrong address in a real inbox. The point of reversibility is the review card, not the send log. If the card never shows the assumed fact, I cannot refuse in time.

This is not a trust essay. It is a from-zero protocol. You will name one decision, write probes, rehearse them, and verify a stop. Each stage has a command you can run. Each stage has a check that fails closed.

Stage 0: freeze the consequential click

Pick one flow you already ship. Do not start with a platform rewrite. I use a refund notice because money and identity sit in the same sentence. Your analog might be a calendar invite, a price change, or an access grant. Same shape. Same risk.

Write the click in one line. Who decides. What happens if they are wrong. When they can still reverse it. If you cannot name those three, you are not ready to probe.

# labeled example: decision freeze, not a production record
decision: send refund notice
owner: on-call designer of record
consequence: mail a wrong address as confirmed fact
reversible_until: review card is submitted
irreversible_after: provider accepts the send

Verification: read the file aloud. Can a teammate point at the owner without guessing? If they guess, rewrite the line. Missing ownership is itself missing evidence.

Stage 1: list the facts the draft needs

Open the agent draft. Highlight every noun that must be true. Address. Order id. Consent timestamp. Locale. Channel. You will feel silly. Do it anyway. Agents love filling fog with weather.

I treat those nouns like props on a stage. If the prop never entered from the wings, the actor invented it. Invented props do not belong under an Approve button. They belong in a yellow field that says unproven.

# labeled example: evidence map for one draft
required_facts:
  - shipping_address
  - address_confirmed_at
  - refund_amount
  - user_locale
  - send_channel
source_of_truth:
  shipping_address: order record
  address_confirmed_at: last user confirmation event
  refund_amount: ledger line
  user_locale: profile, not the draft tone
  send_channel: last inbound channel

Verification: for each required fact, name a system of record. If the source is “the model inferred it,” you found an assumption. Circle it. That circle is the rest of this tutorial.

Stage 2: write probes that try to catch the fill

A probe is a small scenario, not a vibe. I write three. Happy path with complete evidence. Fog path with one missing field. Hostile path where the missing field looks complete. The hostile one matters most. Models polish empty rooms until they look furnished.

Ask yourself. Which missing evidence should stop approval? Which extra sentence would only add noise? If a probe cannot change the click, delete the probe. Noise is not research.

# labeled example: three research scenarios
probes:
  - id: P1-complete
    given: all five facts present in the record
    expect: review card shows evidence ids, Approve enabled
    stop_if: any evidence id is blank
  - id: P2-missing-confirm
    given: address present, address_confirmed_at absent
    expect: Approve disabled, assumed_facts contains address_confirmed_at
    stop_if: draft still reads as if the user confirmed
  - id: P3-polished-void
    given: locale missing, draft written in fluent local tone
    expect: locale marked unproven, tone not treated as proof
    stop_if: card cites “sounds native” as evidence

Verification: run a dry read with a colleague. They should predict the stop without seeing your expect line. If they cannot, the probe is still a hypothesis. Label it that way. Do not ship a hypothesis as a finding.

Stage 3: rehearse on a cheap lane before the real queue

You need a place to fail that is not production. I want the same review card, not a prettier demo. A free model lane and a free server option are enough for this rehearsal. They are not enough for a regulated send path. Keep those two facts separate.

Disclosure: This article was prepared as part of MonkeyCode's product outreach. I use MonkeyCode here only as that cheap rehearsal lane: free model access plus a free server option, so the probe file can run without borrowing the production agent. I am not claiming model names, quotas, or hardware. If your team already has another isolated lane, use that. The protocol does not care about the logo.

Keep the runner local and boring. The server holds the probe file. The model sees only the draft plus the evidence map. It must not see your expect lines. Contamination is how silent fills sneak back in.

# labeled example: local rehearsal, not a vendor CLI
mkdir -p rehearsal/probes rehearsal/out
cp evidence_map.yml rehearsal/
cp probes.yml rehearsal/probes/

# strip expect lines so the model cannot cheat
python3 - <<'PY'
from pathlib import Path
import re
raw = Path("rehearsal/probes/probes.yml").read_text()
clean = re.sub(r"(?m)^\s*expect:.*\n", "", raw)
Path("rehearsal/probes/probes.public.yml").write_text(clean)
print("public probes ready")
PY

python3 probe_assumptions.py \
  --draft refund_draft.md \
  --evidence rehearsal/evidence_map.yml \
  --probes rehearsal/probes/probes.public.yml \
  --out rehearsal/out/review_card.json

Verification: grep expect rehearsal/probes/probes.public.yml must print nothing. Then open review_card.json. If assumed_facts is empty on P2, the rehearsal failed. Do not “fix” it by prompting the model to be careful. Fix the card contract.

Stage 4: put assumed facts on the review card

The card is the interface. Prose in the draft is not the interface. I want four fields, always in the same order. Decision. Evidence. Assumed facts. Stop condition. Designers scan order. Screen readers scan order. Do not hide the dangerous field in a tooltip.

Think of the card as a coat check. The draft may wear a borrowed coat. The ticket must still name the owner. No ticket, no door.

{
  "decision": "send refund notice",
  "evidence": [
    {"field": "shipping_address", "record_id": "ord_1842", "status": "proven"},
    {"field": "address_confirmed_at", "record_id": null, "status": "missing"}
  ],
  "assumed_facts": ["address_confirmed_at"],
  "draft_claims_unproven": true,
  "approve_enabled": false,
  "stop_reason": "unproven confirmation treated as fact in draft",
  "hand_back": "request user confirm address, then re-run P2"
}

# labeled example: fail closed if the draft still asserts a missing fact
import json, pathlib, sys
card = json.loads(pathlib.Path("rehearsal/out/review_card.json").read_text())
missing = {row["field"] for row in card["evidence"] if row["status"] == "missing"}
assert missing <= set(card["assumed_facts"]), "missing facts not surfaced"
if missing:
    assert card["approve_enabled"] is False, "Approve must not light up"
    assert card["draft_claims_unproven"] is True, "draft still smuggles a fact"
print("card contract held")

Verification: run that script. A red assertion is a research win. It means the human still has a brake. A quiet pass on P2 is the failure. That is the agent smiling while it invents a timestamp.

Stage 5: accessibility is part of the stop, not a polish pass

If the stop lives in color, some people will not see it. If Approve is a mouse-only overlay, some people cannot refuse in time. If assumed facts are an icon with no name, a screen reader will hear a decorative star. That is not a review. That is a shrug.

I read the card in one linear pass. Heading. Decision. Evidence list. Assumed facts. Stop reason. Hand-back. Then I tab. Approve must not be reachable when approve_enabled is false. Refuse and hand-back must be reachable. Focus order is a recovery path.

# labeled example: keyboard pass on the review card
Tab -> Decision heading
Tab -> Evidence list (each fact + status as text, not color alone)
Tab -> Assumed facts (named, not icon-only)
Tab -> Stop reason
Tab -> Hand-back action
Tab -> Refuse
Tab -> Approve (skipped or disabled when unproven facts exist)

Verification: disable the pointer. Complete a P2 refuse using keyboard only. Then replay with a screen reader. You should hear the missing confirmation before you hear the draft body. If the draft is read first, the assumed fact already won the room.

Stage 6: name stop conditions and the hand-back

A stop is not a vibe of discomfort. It is a rule the card can evaluate. I use three. Any required fact is missing. The draft asserts a missing fact in plain language. The model cites tone, fluency, or “user probably meant” as evidence. That third one is the polite lie.

Hand-back is not a generic “ask a human.” Name the next owner and the next evidence. In the refund case, the owner is the user. The evidence is a confirmation event. The agent does not get to skip that and keep my name on the send.

stop when:
  - a required fact has status missing
  - draft_claims_unproven is true
  - evidence source is model inference or tone
do not stop when:
  - the draft is inelegant
  - a proven fact could be phrased more warmly
hand_back:
  owner: user for confirmation, designer for resend
  artifact: confirmation event id
  re_enter_at: Stage 2, probe P2

Verification: take one noisy request, like “make it sound more apologetic.” Run it through the card. Approve should stay disabled if confirmation is still missing. Style is not evidence. If style can re-enable Approve, your stop is theater.

What this does not prove

This protocol finds silent fills in one flow. It does not prove the model is honest. It does not prove the ledger is right. It does not replace consent records, legal review, or on-call judgment. I have not run a published benchmark here. Treat every sample as a labeled example until your team records its own P2 failures.

Who should not use this as-is? Anyone sending medical, legal, or credit decisions from a rehearsal lane. Anyone who cannot name a system of record. Anyone hoping a free model pass will bless a production agent. A cheap lane is for catching invented props. It is not a court.

If you already rehearse somewhere isolated, steal the card contract this week. Run P2 on one real draft. Ask the same two questions I started with. Which missing evidence should stop approval? Which extra sentence would only add noise? Keep the noise off the button. Keep the missing fact on the card.