DEV Community: Haley

Tutorial: Test the Refusal Path Before You Trust a Free Model Draft

Haley — Fri, 06 Nov 2026 16:19:51 +0000

Tutorial: Test the Refusal Path Before You Trust a Free Model Draft

Last Tuesday a support lead approved a refund reply she had not read. The draft looked calm, polite, and correct. The gate that let it through asked exactly one question: did a human click?

She was the decision owner. The consequence was a refund we cannot un-send. The point of reversibility closed the second she hit send.

Free models make that gate cheaper to build. They do not make it optional. So here is the honest question. Would your gate refuse, or would it just look busy?

I built this tutorial to answer that with a test suite, not an opinion.

What the free tier actually buys you

MonkeyCode is an open source project, as the operator describes it. It offers free model access and a free server option. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The promotion I was told about includes ten million free tokens. Check the live terms yourself before you plan around them, because quotas move.

That is the whole commercial story. Everything below is the part I actually care about.

We will build a gate that refuses first and drafts second. Each stage ends with a check. Skip a check and you are testing your hope.

Stage 1: Stand up a workspace and prove the endpoint answers

mkdir gate-lab && cd gate-lab
python -m venv .venv && source .venv/bin/activate
pip install httpx pytest

Put your endpoint and key in the environment, never in the file. Copy the request path straight from your dashboard instead of guessing it.

export MC_BASE="<your server URL from the dashboard>"
export MC_PATH="<the request path shown in your dashboard>"
export MC_KEY="<your key>"

Now the check. A refusal here is cheap. A refusal in production is not.

curl -sS -o /dev/null -w "%{http_code}\n" \
  -H "Authorization: Bearer $MC_KEY" \
  "$MC_BASE$MC_PATH"

A 401 means your key is wrong, not your code. Fix that before you write another line.

Stage 2: Define the envelope before you call anything

The gate needs a shape it can reject. Mine has six fields. Who decides, what the action is, whether it reverses, how it recovers, what evidence exists, and where the stop conditions live.

# envelope.py
from dataclasses import dataclass, field
from datetime import datetime, timedelta, timezone

MAX_EVIDENCE_AGE = timedelta(hours=24)

@dataclass
class Evidence:
    source: str
    fetched_at: datetime
    covers: str

@dataclass
class Envelope:
    decision_owner: str
    action: str
    reversible: bool
    recovery: str
    evidence: list[Evidence] = field(default_factory=list)
    stop_conditions: list[str] = field(default_factory=list)

    def refusals(self, now=None) -> list[str]:
        now = now or datetime.now(timezone.utc)
        reasons = []
        if not self.decision_owner:
            reasons.append("no named decision owner")
        if not self.reversible and not self.recovery:
            reasons.append("irreversible with no recovery path")
        if not self.evidence:
            reasons.append("no evidence attached")
        for item in self.evidence:
            if now - item.fetched_at > MAX_EVIDENCE_AGE:
                reasons.append(f"stale evidence: {item.source}")
        if not self.stop_conditions:
            reasons.append("no stop conditions declared")
        return reasons

Check: open a Python shell and build one broken envelope by hand. If refusals() returns an empty list for an action with no owner, your validation is decoration.

Stage 3: Make the gate fail closed

The gate calls the model only after the envelope survives review. Order matters more than elegance here.

# gate.py
import httpx
from envelope import Envelope

class FreeModelGate:
    def __init__(self, client, path):
        self.client = client
        self.path = path

    def review(self, env: Envelope) -> dict:
        reasons = env.refusals()
        if reasons:
            return {"status": "refused", "reasons": reasons, "text": ""}
        try:
            resp = self.client.post(self.path, json={
                "action": env.action,
                "evidence": [e.source for e in env.evidence],
                "stop_conditions": env.stop_conditions,
            }, timeout=20.0)
        except httpx.TimeoutException:
            return {"status": "blocked", "reasons": ["model timeout"], "text": ""}
        return {"status": "drafted", "reasons": [], "text": resp.text}

Look at the timeout branch. It returns blocked, not drafted. A slow free server must never become an accidental approval.

Check: call review() with an empty envelope and a fake client that raises on use. If your fake client gets called, the gate is wired backwards.

Stage 4: Write the tests that prove refusal

This is the stage most tutorials skip. The interesting behaviour is not the draft. It is the no.

# test_refusal.py
from datetime import datetime, timedelta, timezone
from envelope import Envelope, Evidence
from gate import FreeModelGate

class SpyTransport:
    def __init__(self):
        self.calls = 0
    def post(self, *a, **k):
        self.calls += 1
        raise AssertionError("model called on a refused envelope")

def fresh():
    return Evidence("billing-api", datetime.now(timezone.utc), "refund window")

def test_missing_owner_never_calls_model():
    spy = SpyTransport()
    env = Envelope("", "send refund reply", True, "", [fresh()], ["amount > policy cap"])
    out = FreeModelGate(spy, "/draft").review(env)
    assert out["status"] == "refused"
    assert spy.calls == 0

def test_irreversible_without_recovery_never_calls_model():
    spy = SpyTransport()
    env = Envelope("ana", "delete account", False, "", [fresh()], ["legal hold flag"])
    out = FreeModelGate(spy, "/draft").review(env)
    assert out["status"] == "refused"
    assert spy.calls == 0

def test_stale_evidence_is_refused():
    old = Evidence("billing-api", datetime.now(timezone.utc) - timedelta(days=3), "refund window")
    env = Envelope("ana", "send refund reply", True, "resend correction", [old], ["amount > cap"])
    out = FreeModelGate(SpyTransport(), "/draft").review(env)
    assert any("stale" in r for r in out["reasons"])

$ pytest -q
3 passed

Three green tests are your first real evidence. They are also free to run, which matters on a free tier.

Stage 5: Order the review card around the stop conditions

The card is what a human reads. So put the refusal reasons first, not the model's prose. My order is: action, reversibility, stop conditions, evidence sources with ages, then the draft.

Check: read the card aloud. If you say "looks fine" before you say the stop conditions, reorder it.

Stage 6: Review the card for access, not aesthetics

Status must be a word, never a colour. Stop conditions must appear before the draft in reading order. Ages must be written as durations, not timestamps only.

Check: remove colour entirely and read again. If the decision changes, the design was relying on something a screen reader never gets.

Stage 7: Decide when this gate is the wrong tool

Situation	Use the gate?	Why
Draft replies with a named reviewer	Yes	Refusal is cheap and reversible
Irreversible actions with audit duties	Not alone	You need a durable record, not a script
High-volume, sub-second paths	No	Human review is the bottleneck, not the model
No named decision owner yet	Not yet	The gate would only formalise avoidance

That last row is the one I would defend hardest. A gate without an owner is a queue with better manners.

What I would do first if I were you

Run the three refusal tests before you write a single prompt. They cost nothing and they tell you the most. Then let the free server answer one boring draft and read your own card out loud.

If you want to try this on the free server, start with the tests, not the model. The tests are the part that survives the promotion.

Which missing evidence would stop your approval today? If the answer is none, you do not have a gate yet.

Tutorial: Show the Serving Tier Beside Every AI Draft

Haley — Thu, 05 Nov 2026 14:57:47 +0000

Tutorial: Show the Serving Tier Beside Every AI Draft

Last Tuesday a reviewer approved an AI draft in four seconds. The draft had quietly dropped a refund clause. Nobody in that room knew which model had answered, or whose server ran it. Was the reviewer careless? Not really. She had no evidence to weigh.

That is a design failure, not a people failure. The interface asked her to trust a draft while hiding where the draft came from. So let me ask you something. If your product cannot say which tier served a request, what exactly is your reviewer approving?

This tutorial builds that missing evidence, end to end. You will run a probe, render a tier strip, write stop conditions, and check accessibility. Each stage ends with a verification step you can actually execute.

A word on the tool I use throughout. MonkeyCode is an open-source AI coding project you can try in the browser or self-host. Its outreach brief advertises free model access and a free server option, plus a ten-million-token allowance. I treat all three as claims to verify, not facts to design around. That is the whole point of the harness below.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Stage 0: Write the claims file first

Before touching code, write down what you were told. Mine looks like this.

mkdir -p claims && cat > claims/free-tier.md <<'EOF'
# Free tier claims (unverified)
- free model access        | source: outreach brief | checked: never
- free server option       | source: outreach brief | checked: never
- 10,000,000 token grant   | source: outreach brief | checked: never
- open source repository   | source: outreach brief | checked: never
EOF

One line per claim, with a source and a check status. The token number is the one I trust least, because quotas move quietly. Write it down anyway. An unverified number in a file is honest; an unverified number in a slide is a bug.

Verify: cat claims/free-tier.md shows four lines, each with checked: never. If any claim has no source, delete it.

Stage 1: Probe the endpoint once

Now get one real response and record what the wire gives you. Node 20 has fetch, so no dependencies.

// probe.mjs — run: TIER=free node probe.mjs
const base = process.env.MC_BASE_URL;      // from the project docs, not guessed
const key  = process.env.MC_API_KEY;
const model = process.env.MC_MODEL;        // never hardcode a model name
const tier = process.env.TIER ?? 'unknown';

if (!base || !key || !model) {
  console.error('missing MC_BASE_URL / MC_API_KEY / MC_MODEL');
  process.exit(2);
}

const t0 = Date.now();
const res = await fetch(`${base}/chat/completions`, {
  method: 'POST',
  headers: { 'content-type': 'application/json', authorization: `Bearer ${key}` },
  body: JSON.stringify({
    model,
    messages: [{ role: 'user', content: 'Summarize this clause in one line: refunds within 14 days.' }],
    max_tokens: 120
  })
});

const body = await res.json().catch(() => ({}));
console.log(JSON.stringify({
  tier,
  http: res.status,
  model_returned: body.model ?? 'unknown',
  finish_reason: body.choices?.[0]?.finish_reason ?? 'unknown',
  usage: body.usage ?? 'unknown',
  ms: Date.now() - t0
}, null, 2));

The endpoint shape above assumes an OpenAI-compatible route. Check the project docs before you trust it. If the shape differs, fix the script, not the log format.

Verify: run it twice. You want a finish_reason and a model_returned string. If either is unknown, keep it as unknown in your UI later. Guessing is how the refund clause disappears.

Stage 2: Render the tier receipt

Here is the pattern. Put it directly under the draft, above the approve button.

┌ Draft: refund policy summary ─────────────────────┐
│ ...text...                                        │
├ Serving tier ─────────────────────────────────────┤
│ tier        free (hosted)                         │
│ model       <model_returned, or "unknown">        │
│ finished    stop | length | unknown               │
│ latency     1.9s                                  │
│ context     complete | truncated | unknown        │
│ [ Hand back to me ]   [ Approve ]  [ Return ]     │
└───────────────────────────────────────────────────┘

Six fields, no decoration. truncated and unknown are first-class values. A strip that can only say "success" is theatre, and your reviewers will learn to ignore it.

Use plain language in the visible label. "Free shared model" beats "tier_0" every time. Reviewers are not reading your enum.

Verify: force each of the three context values with a fixture and screenshot all three. If you cannot produce unknown on demand, you cannot produce it in production either.

Stage 3: Write stop conditions before the study

A receipt is worthless without a rule for when it stops the flow. Run five scenarios with five people, and give each a stop condition.

Scenario	What you inject	Measure	Stop condition
Context overflow	40-page doc	Did they notice `truncated`?	Two misses in a row
Rate limit	HTTP 429	Did they retry or escalate?	Silent auto-retry seen
Slow reply	12s delay	Did they approve blind?	Any approval before it renders
Refusal	Model declines	Could they recover in one step?	Rage click on Approve
Mid-task failure	Drop the response	Did hand-back work?	Draft lost, no copy

Record clicks, not opinions. When two participants approve a truncated draft, stop the study and fix the strip. Do not finish the sample to be polite.

Verify: after each session, check you have one timestamped event for every stop condition you wrote down.

Stage 4: Draw the recovery path

request ─▶ tier receipt ─▶ reviewer
                │             │
         unknown/truncated    ├─ approve ─▶ publish
                │             ├─ return  ─▶ edit
                ▼             └─ hand back ─▶ human writes
        hand back to human

The hand-back must be one click, and it must keep the draft. If returning a draft costs typing, people approve instead. That is a design decision, not a training problem.

Verify: with the network throttled to offline, click hand back. The draft text must survive.

Stage 5: Check accessibility

The strip carries a decision, so it must be reachable and readable.

Test it with a screen reader in one pass. Confirm the tier is announced as text, not color alone. Confirm truncated is in the accessible name, not only in an icon. Confirm the hand-back button is first in tab order after the draft. Confirm error text is tied with aria-describedby, never placeholder-only. Confirm contrast of the strip at 200% zoom.

One more check people forget. Read the strip aloud. If it sounds like a log line, rewrite it for humans.

Verify: tab from the draft to hand-back without a mouse. If focus skips it, you have a blocking bug.

Stage 6: Decide with evidence, not vibes

The table below is my rule for designing around a free tier. It is a proposal, not a finding.

Evidence you hold	Design around free tier?
Receipt renders tier, model, finish reason	Conditional yes
`unknown` states are visible and testable	Conditional yes
Hand-back works offline, keeps draft	Yes
Stop conditions met, no misses	Yes
Quota still unverified for your volume	No, spike first
Receipt exists only in logs, not UI	No

If the last row is true for you, stop. Logs are for you. The receipt is for the person who clicks.

Limitations, and who should skip this

This harness measures interface honesty, not model quality. It will not tell you whether a free model writes good refund copy. It will only tell you whether your reviewer knew what she was reading.

Skip it if you have no human approval step, or if no draft leaves the system. Skip it if you cannot read your provider's response fields at all. And skip the token arithmetic entirely until someone with the account confirms the current grant.

Want a place to run the probe against something real? MonkeyCode's free model access and free server option make a reasonable test bench, and the project is open source. Try it, then read your receipt out loud. If it cannot say which tier answered, your reviewer is still guessing.

Tutorial: Show the Token Ceiling Before the Agent Runs Unattended

Haley — Tue, 27 Oct 2026 12:24:55 +0000

Last month a two-person team demoed an agent to a room of twelve people.
At minute nine the run stopped dead. Half a draft hung on the projector.
Nobody knew the token ceiling, so nobody could explain the stop.

Who owned that decision? The PM did. She clicked "run twice" ninety minutes earlier.
That click was the point of reversibility, and nobody marked it.

So let me ask you something. Would you ship a form that silently discards input at submit?
Then why are you shipping an agent that silently dies at minute nine?

This tutorial builds the missing evidence. You will produce a capacity rehearsal card.
You will gate the run on a measured ceiling, not a guess.
Every stage ends with a check you can actually run.

I use MonkeyCode's free model access and free server option as the staging surface here.
Disclosure: This article was prepared as part of MonkeyCode's product outreach.
Treat the free allowance as an operator-stated plan detail and re-check current terms first.

Stage 1: Measure one call before you automate ten

Start with one request. Never with a batch.
I want a token delta, a latency, and an exit code.

# example harness, not executed in this post
curl -sS -o out.json -w '%{http_code}\n' \
  -X POST "$BASE_URL/chat/completions" \
  -H "Authorization: Bearer $TOKEN" \
  -H 'Content-Type: application/json' \
  -d @fixtures/short_prompt.json

Then read usage from the response body, not from your own counter.
Your counter is a hypothesis. The provider's number is evidence.

import json, pathlib
usage = json.loads(pathlib.Path("out.json").read_text())["usage"]
total = usage["prompt_tokens"] + usage["completion_tokens"]
print("one_call_total", total)

Check: run it twice with the same fixture. If the two totals differ by more than a rounding step, your ceiling math is already unsafe.

Stage 2: Turn the allowance into a per-run budget

A free allowance is a pool, not a promise. Pools get shared and pools get spent.
So convert the pool into a per-run number you can defend.

Here is the ratio I use for a demo run: one run may spend at most 12% of the measured allowance.
That leaves room for a retry, a bad day, and a second teammate.
Is 12% magic? No. It is a stated design hypothesis you must label as one.

ALLOWANCE_FRACTION = 0.12          # design hypothesis, not a finding
RESERVE_FRACTION   = 0.25          # untouchable floor for recovery
DEMO_RUNS          = 3

def ceiling(allowance_total: int) -> int:
    return int(allowance_total * (1 - RESERVE_FRACTION) * ALLOWANCE_FRACTION)

Check: write the number on the rehearsal card next to the assumption it came from.
A ceiling without its source is a rumour with a decimal point.

Stage 3: Build the gate that refuses the run

Now the interesting part. The gate must say no.
An agent that can only start is not controlled. It is merely fast.

class StopRun(Exception):
    def __init__(self, code, shown, next_step):
        self.code, self.shown, self.next_step = code, shown, next_step

def guarded_run(run, budget, allowance_left, dry_run=False):
    if allowance_left < budget + int(budget * 4):     # reserve check
        raise StopRun("E_ALLOWANCE_FLOOR",
                      "This run needs more room than the free allowance has left.",
                      "Shorten the prompt, or wait for the allowance to reset.")
    if dry_run:
        return {"status": "rehearsed", "budget": budget}
    return run(budget=budget)

Every stop carries three fields. A code, a sentence a human reads, and a next step.
Drop any one of those and you have built a dead end, not a guardrail.

Check: call guarded_run with allowance_left=budget. It must raise, not warn.
A warning in a log is not a stop condition. It is a hope.

Stage 4: Make the stop legible, not decorative

Here is where most agent UIs fail quietly. The stop appears as a toast. It vanishes.
A screen reader user hears nothing, and the run history shows a gap.

So render the stop as a persistent card with text, not colour.

<section role="status" aria-live="polite" aria-atomic="true" id="stop-card" hidden>
  <h2>Run paused: allowance floor reached</h2>
  <p id="stop-reason"></p>
  <p><strong>Next step:</strong> <span id="stop-next"></span></p>
  <button type="button">Review the last complete draft</button>
</section>

Move focus to that heading. Keep the previous draft reachable in one click.
An error without a recovery path is just an apology with better typography.

Check: tab through the paused state with the mouse untouched. If focus stays on the send button, you have work left.

Stage 5: Rehearse the failure before the demo

Now break it on purpose. Kill the run at 80% of its budget.
Then ask a colleague who did not write the code to recover the draft.

If they hesitate for more than ten seconds, your recovery story is fiction.
Write down what they said out loud. That sentence becomes your hand-back copy.

Check: the dry run returns status: rehearsed, the live run returns a real result, and both leave the same audit line.

Stage 6: Decide with a stop table, not a vibe

Evidence on the card	Approve unattended run?	Why
Measured one-call tokens, two runs agree	Yes	Ceiling has a source
Ceiling set, reserve floor intact	Yes	Recovery is funded
Stop card focus-tested, next step present	Yes	Failure is legible
Tokens inferred from character count	No	Arithmetic dressed as measurement
Allowance undocumented at run time	No	Unknown ceiling is no ceiling
Recovery requires reading a log file	No	Users cannot recover

That last row matters most to me. Logs are for maintainers, not for the person whose draft just froze.

What this method cannot tell you

It cannot promise the free allowance stays constant. Plans move, caps change, servers migrate.
It cannot tell you whether a longer prompt is worth its cost. That is a product judgement.
And it cannot replace a real load test if you plan concurrent runs.

Skip this workflow if you are prototyping alone with no second user, or if your provider gives you hard caps at the API layer. The gate is redundant there.
Is it redundant for you? Check before you build it.

Where MonkeyCode fits, honestly

The free model access let me rehearse on a separate surface without touching paid quota.
The free server option meant the gate ran next to the model, not on my laptop.
That separation is the point. Rehearsal should never borrow the demo's keys.

Start with Stage 1 and one fixture. Measure, divide, gate, then break it.
If the free tier is still open when you read this, that is more than enough room to begin.

And when a run stops in front of twelve people, you will have the number ready.
Not an excuse. A ceiling, a card, and a way back.

Tutorial: Prove the Reviewer Read the Card

Haley — Mon, 26 Oct 2026 11:07:22 +0000

Last month I sat in on a review session that lasted eleven minutes.

Twelve agent actions got approved. Nine went through in under two seconds.

Nobody was careless. That is the point. The card was simply easy to click past.

So what did we actually approve? The support lead owned that decision. A refund had gone out against a ledger nobody opened. It was reversible for twenty-four hours, which is the only reason it became a lesson instead of an incident.

This tutorial does not improve the card. It builds the instrument that tells you the card failed. We go from empty folder to a running audit, with a verification step at each stage.

Step 0: Decide what "read" means before you log anything

You cannot measure comprehension. You can measure three observable proxies.

Dwell time is the first. Evidence expansion is the second. Editing before approving is the third.

None of them is proof. A fast approval can be correct. A slow one can be lazy. Write that on the wall before you start.

Ask the harder question early. Which missing evidence should stop approval here? And which extra panel would only add noise?

Step 1: A log format small enough to keep

One JSON event per line. No schema registry, no vendor SDK.

{"session":"pilot-01","review_id":"r-01","event":"card_open","t":0}
{"session":"pilot-01","review_id":"r-01","event":"evidence_expand","panel":"source_ledger","t":1840}
{"session":"pilot-01","review_id":"r-01","event":"decision","verdict":"approve","t":2310}

Three event names. card_open, evidence_expand, decision. The panel field is what makes the later report readable.

Keep t relative to the session, not wall clock. You are measuring intervals, not surveillance timestamps.

Step 2: Wire the card, gently

Here is the sketch I use. It is a sketch, not production code, and I have not run this exact block in your app.

// sketch — point the sink at your own collector
function logEvent(reviewId, event, extra = {}) {
  const payload = JSON.stringify({
    session: SESSION_ID,
    review_id: reviewId,
    event,
    t: Math.round(performance.now()),
    ...extra,
  });
  navigator.sendBeacon("/events", payload);
}

card.addEventListener("open", () => logEvent(REVIEW_ID, "card_open"));

for (const panel of document.querySelectorAll("[data-evidence-panel]")) {
  panel.addEventListener("toggle", () => {
    if (panel.open) {
      logEvent(REVIEW_ID, "evidence_expand", { panel: panel.dataset.evidencePanel });
    }
  });
}

approveButton.addEventListener("click", () =>
  logEvent(REVIEW_ID, "decision", { verdict: "approve" })
);

Use real <details> elements, not clickable divs. Keyboard users get expansion for free. Screen reader users hear the panel state.

And do not put a countdown on the card. A timer turns measurement into pacing. You will get longer dwell times and worse decisions.

Step 3: Get pilot material without writing twelve cards by hand

You need reviews with varied risk. Hand-writing them takes an afternoon you do not have.

Here is where I reached for MonkeyCode. The operator offers free model access, so drafting card text costs me nothing but review time. I wrote the ground truth myself, because model output is material, not truth.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Two caveats, stated plainly. First, the operator describes a free token allowance — reported to me as ten million tokens — and I have not personally verified the current figure or its conditions. Second, free access is an availability claim, not a permanence promise. Check the terms before you build a workflow around them.

My split was simple. The model drafted the visible card text: the action summary, the stated reason, the risk label. I wrote the hidden answer key by hand.

That answer key matters. Without it you can measure speed but never correctness.

Step 4: Run the session somewhere two people can reach

Twelve reviews. Two reviewers. One shared build.

I did not want to host anything, so I used the free server option to run a single small Python process that accepts /events and appends to a JSONL file. One process, one port, one file. If your environment gives you different start mechanics, keep the shape and change the command.

python3 collector.py --port 8080 --out pilot-01.jsonl

Tell both reviewers what you log, before they start. Expansion events and decision times. Aggregate reporting only. No individual scores, ever.

Consent is not paperwork here. It is the difference between a study and a sting.

Step 5: Audit the log with the standard library

Save this as review_audit.py. No dependencies.

#!/usr/bin/env python3
import json
import sys
from collections import defaultdict

READ_MS = 3000  # your threshold, not a universal constant

def load(path):
    events = []
    with open(path) as f:
        for line in f:
            line = line.strip()
            if line:
                events.append(json.loads(line))
    return events

def per_review(events):
    groups = defaultdict(list)
    for e in events:
        groups[e["review_id"]].append(e)
    for rid in groups:
        groups[rid].sort(key=lambda e: e["t"])
    return groups

def audit(groups):
    rows = []
    for rid, evs in groups.items():
        opened = next((e["t"] for e in evs if e["event"] == "card_open"), None)
        decided = next((e for e in evs if e["event"] == "decision"), None)
        if opened is None or decided is None:
            rows.append({"review_id": rid, "status": "incomplete"})
            continue
        expanded = [e for e in evs if e["event"] == "evidence_expand" and e["t"] <= decided["t"]]
        dwell = decided["t"] - opened
        rows.append({
            "review_id": rid,
            "status": "ok",
            "dwell_ms": dwell,
            "panels_opened": len({e["panel"] for e in expanded}),
            "verdict": decided["verdict"],
            "flag": dwell < READ_MS and not expanded,
        })
    return rows

def main(path):
    rows = audit(per_review(load(path)))
    done = [r for r in rows if r["status"] == "ok"]
    if not done:
        print("no complete reviews found")
        return
    dwells = sorted(r["dwell_ms"] for r in done)
    mid = dwells[len(dwells) // 2]
    flagged = [r for r in done if r["flag"]]
    opened = sum(1 for r in done if r["panels_opened"])
    print(f"reviews: {len(rows)}  decisions: {len(done)}  incomplete: {len(rows) - len(done)}")
    print(f"median dwell ms: {mid}")
    print(f"reviews with at least one evidence panel opened: {opened}/{len(done)}")
    print(f"flagged as possible rubber-stamp: {len(flagged)}/{len(done)}")
    for r in flagged:
        print(f"  {r['review_id']}  dwell={r['dwell_ms']}ms  panels=0  verdict={r['verdict']}")

if __name__ == "__main__":
    main(sys.argv[1])

Note what the flag means. Short dwell and no expansion. One condition alone is a rumour.

Step 6: Verify the script against a fixture

The numbers below come from a fixture I wrote by hand. They are not from a real session, and you should not read them as a result.

{"session":"fixture","review_id":"r-01","event":"card_open","t":0}
{"session":"fixture","review_id":"r-01","event":"evidence_expand","panel":"source_ledger","t":1840}
{"session":"fixture","review_id":"r-01","event":"decision","verdict":"approve","t":2310}
{"session":"fixture","review_id":"r-02","event":"card_open","t":0}
{"session":"fixture","review_id":"r-02","event":"decision","verdict":"approve","t":900}
{"session":"fixture","review_id":"r-03","event":"card_open","t":0}
{"session":"fixture","review_id":"r-03","event":"evidence_expand","panel":"missing_fields","t":400}
{"session":"fixture","review_id":"r-03","event":"decision","verdict":"edit","t":6200}

python3 review_audit.py fixtures/pilot-01.jsonl

reviews: 3  decisions: 3  incomplete: 0
median dwell ms: 2310
reviews with at least one evidence panel opened: 2/3
flagged as possible rubber-stamp: 1/3
  r-02  dwell=900ms  panels=0  verdict=approve

Now break it on purpose. Add a review that opens the card, expands nothing, and decides at 9000ms. The script must not flag it, because the dwell is long.

Then delete one decision line and run again. The incomplete count must rise.

Two checks. If either fails, your log format drifted and every later number is fiction.

Step 7: Agree on stop conditions before the real session

Write these down with the decision owner, not after the data arrives.

Signal	Response
Zero evidence panels opened across all reviews	Stop. The card is decoration.
Flagged on more than half of decisions	Pause approvals on irreversible actions until the card changes.
Flags cluster on one verdict type	Inspect that verdict's copy first, not the layout.
Reviewers speed up after you share the numbers	Treat the data as contaminated. Re-run with a fresh group.

The last row is the one teams skip. Show people a speed metric and they will beat it.

What this instrument cannot tell you

Twelve reviews is a smoke test, not a study. You cannot compute significance on it.

Dwell time is a proxy with a known failure: reading is invisible. A reviewer can read carefully in 2,400ms. A reviewer can stare at a card for nine seconds and think about lunch.

The fixture proves the script runs. It says nothing about your reviewers.

And accessibility is a real constraint here. Panels must be keyboard reachable, labelled, and untimed. If your instrumentation makes the card slower for assistive tech users, you are measuring your own bug.

Who should not use this

Do not run this with fewer than five reviewers. You will be able to identify individuals from the log, and that turns a study into a performance review.

Skip it if you cannot act on the result. A number nobody changes anything for is just anxiety with a dashboard.

If your context is regulated, get the logging reviewed before you collect a single event.

Where I would start

Twelve reviews. Two people. One afternoon.

If you want to try this without paying for model calls, the free model access is enough to draft the card text. Draft twelve, then write the answer key yourself.

Start with twelve, not twelve hundred. The first flagged review will teach you more than a quarter of aggregate data.

What would you need to see before you let that approval through?

Tutorial: Make the Agent Show Its Ledger Before It Spends

Haley — Tue, 20 Oct 2026 09:11:28 +0000

Last Tuesday I approved a five-step research run. It broke on step three.

The summary it produced looked fine. The silence underneath it did not.

I could not say which steps had actually run. I could not say what they cost.

So I approved an action with missing evidence. That was my call, not the model's.

Here is the question I keep asking myself now. What would have stopped me?

Not a nicer warning banner. A record, written before the spend.

This tutorial builds that record. Five stages, each with a check you can fail on purpose.

The whole thing is three small Node files and no dependencies. You can run it before lunch.

Why a ledger, and not a dashboard

A dashboard tells you what already happened. You need to know what is about to happen.

That is the decision point. Approve, refuse, or hand back — before the call, not after.

A local ledger does three jobs at once. It proves the run's spend, it carries the stop condition, and it maps recovery after a crash.

I run these calls through MonkeyCode's free model access, and the project is open source. The operator also offers a free server option.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

I treat both availability claims as operator-supplied. I do not restate any allowance figure here. Check the project page for the current numbers, then count the spend yourself.

The dashboard number is not evidence. Your record is.

Stage 1 — Write the record shape first

Design the record before you touch a model. Fields beat prompts here.

// ledger.js — append-only spend and recovery record, no dependencies
const fs = require("fs");
const PATH = process.env.LEDGER || "run.jsonl";

function read() {
  if (!fs.existsSync(PATH)) return [];
  return fs.readFileSync(PATH, "utf8").split("\n").filter(Boolean).map(JSON.parse);
}

function remaining() {
  const rows = read();
  if (!rows.length) return Number(process.env.BUDGET_START || 0);
  return rows[rows.length - 1].budget_after;
}

function append(entry) {
  const before = remaining();
  const row = { ...entry, budget_before: before, budget_after: before - entry.tokens };
  if (!Number.isFinite(row.tokens) || row.tokens < 0) throw new Error("refuse: bad token count");
  if (row.budget_after < 0) throw new Error("refuse: budget would go negative");
  fs.appendFileSync(PATH, JSON.stringify(row) + "\n");
  return row;
}

module.exports = { read, remaining, append };

Every row needs seven things. run_id, step_id, the intent in plain words, an evidence_ref pointing at the source it read, the tokens it spent, the outcome, and whether the step is reversible.

If a field feels awkward to fill, that is the field doing the work. Awkward is honest.

Verify it with a throwaway file:

rm -f tmp.jsonl
LEDGER=tmp.jsonl BUDGET_START=10000 node -e '
const l = require("./ledger");
l.append({ run_id:"demo-01", step_id:"s1", intent:"read source",
           evidence_ref:"doc-17", tokens:120, outcome:"verified", reversible:false });
console.log(l.remaining());'
# expect: 9880

If you see 9880, the arithmetic and the append are both wired. If you see NaN, your last row is missing budget_after, and you just found out before a model did.

Stage 2 — Put a gate in front of every call

The gate is the only part that can say no. Keep it dumb. Dumb gates survive refactors.

// gate.js — run before every model call
const ledger = require("./ledger");
const FLOOR = Number(process.env.FLOOR || 5000);
const step = process.argv[2];
const rows = ledger.read();
const last = rows[rows.length - 1];

if (last && last.outcome !== "verified") {
  console.error(`STOP: previous step unverified — ${last.step_id} (${last.outcome})`);
  process.exit(2);
}
if (ledger.remaining() < FLOOR) {
  console.error(`STOP: remaining ${ledger.remaining()} below floor ${FLOOR}`);
  process.exit(3);
}
console.log(`OK ${step} remaining=${ledger.remaining()}`);

Two exit codes, two meanings. 2 means a human must look at the last step. 3 means the run is too poor to continue.

Now break it on purpose. That is the only way to know it refuses.

LEDGER=tmp.jsonl FLOOR=5000 node gate.js s2; echo "exit=$?"
# expect: OK s2 remaining=9880 / exit=0

LEDGER=tmp.jsonl node -e '
require("./ledger").append({ run_id:"demo-01", step_id:"s2", intent:"draft",
  evidence_ref:"doc-17", tokens:400, outcome:"unverified", reversible:true });'

LEDGER=tmp.jsonl FLOOR=5000 node gate.js s3; echo "exit=$?"
# expect: STOP: previous step unverified — s2 (unverified) / exit=2

Notice what just happened. The run stopped because a draft was unverified, not because it was wrong. You now have a named moment to review.

Stage 3 — Make the first metered call

Pull the endpoint and model name from your own project settings. Do not hardcode them, and do not guess a model name from a blog post.

export MODEL_BASE_URL="<endpoint from your project settings>"
export MODEL_NAME="<model you picked from the available list>"
export MODEL_API_KEY="<your key>"

curl -sS "$MODEL_BASE_URL" \
  -H "Authorization: Bearer $MODEL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"'"$MODEL_NAME"'","messages":[{"role":"user","content":"List three sources for claim X. No summary."}]}' \
  > out.json

Now count the spend yourself. If the response reports usage, trust it and label it. If it does not, estimate and say so.

node -e '
const fs = require("fs");
const res = JSON.parse(fs.readFileSync("out.json", "utf8"));
const u = res.usage || {};
const reported = Number.isFinite(u.prompt_tokens) && Number.isFinite(u.completion_tokens);
const raw = JSON.stringify(res.choices || res.messages || res);
const tokens = reported ? u.prompt_tokens + u.completion_tokens : Math.ceil(raw.length / 4);
require("./ledger").append({ run_id:"demo-01", step_id:"s3", intent:"source hunt",
  evidence_ref:"doc-17", tokens, tokens_source: reported ? "reported" : "estimate",
  outcome:"unverified", reversible:true });
console.log("logged", tokens, reported ? "reported" : "estimate");'

The tokens_source field matters more than the number. An estimate is fine as long as it is labelled. An unlabelled estimate is a quiet lie.

I have not run this against your endpoint. Treat it as a template you adapt, then check the new row with tail -n 1 run.jsonl.

Stage 4 — Rehearse the crash before it happens

Kill the run mid-step. Do it now, while nothing is at stake.

// replay.js — decide what is safe to redo after a crash
const ledger = require("./ledger");
const run = process.argv[process.argv.indexOf("--run") + 1];
const rows = ledger.read().filter(r => r.run_id === run);

for (const row of rows) {
  const redo = row.reversible === true && row.outcome !== "verified";
  console.log(`${row.step_id}\t${row.outcome}\t${redo ? "redo-safe" : "keep-as-is"}`);
}

const last = rows[rows.length - 1];
if (!last || last.outcome !== "verified") {
  console.error("RESUME BLOCKED: last step unverified — hand back to a human");
  process.exit(2);
}

LEDGER=tmp.jsonl node replay.js --run demo-01; echo "exit=$?"

You should see s2 marked redo-safe, s3 blocked, and a 2. The block is the point.

Auto-retry on an unverified step is how drafts get written twice. Recovery is a review decision, not a loop.

Stage 5 — Render an approval card people can actually read

Now the interface. The card must carry the ledger's four facts in words, not colour.

<section role="group" aria-labelledby="card-h" aria-describedby="card-budget card-stop card-undo">
  <h2 id="card-h">Approve step 3: source hunt</h2>
  <p id="card-budget">Recorded budget left: 9,480 tokens. This step is capped at 900.</p>
  <p id="card-stop">Stops if the previous step is unverified, or if budget drops below 5,000.</p>
  <p id="card-undo">Undo: writes to <code>drafts/</code> only. Nothing is sent.</p>
  <p role="status" aria-live="polite" hidden>Cannot approve: previous step is unverified.</p>
  <button type="button">Approve step 3</button>
  <button type="button">Hand back to me</button>
</section>

Run four checks and write down the results. Tab through with no mouse. Confirm the refusal message is announced, not just shown. Confirm the stop condition survives at 200% zoom. Confirm refusal text still reads when colour is removed.

If any check fails, the card is not ready. The ledger is fine. The card is the problem.

Which missing evidence should stop you

This is the table I keep next to the gate.

Missing	Stop approval?	Why
Which step wrote the last artifact	Yes	recovery becomes guesswork
Remaining recorded budget	Yes	you cannot size the run
Whether the next step sends or overwrites	Yes	no undo means no default
A polished summary of step 2	No	noise, not a decision input
The model's self-reported confidence	No	it is not evidence about the world

That last row is the one people argue with me about. Ask yourself what you would do differently with a confidence score. Usually the answer is nothing.

What I can defend, and what I cannot

I can defend the failure I hit. A five-step run broke and nobody could account for steps one through three.

I can defend the checks above. They pass, fail loudly, and produce a row you can read out loud.

The claim that pre-call ledgers reduce unrecoverable runs across teams is a hypothesis. I have not measured it across teams, and I will not pretend otherwise.

Who should not use this

Skip it if you need durable audit storage with retention rules. A JSONL file on a laptop is not compliance.

Skip it if the run executes on a machine you do not control. A gate you cannot enforce is decoration.

Skip it if you need a guaranteed service level. A free tier is a starting point, not an SLA.

Also skip it if nobody on the team will read the ledger. A record nobody reads is just a slower silence.

If you want to try the gate on a scratch run, start on the free server option and keep the ledger local. Then tell me which missing field stopped you first.

That answer is the most useful thing you can send me.

Tutorial: Put a Stop-Check Between the Agent and the Send Button

Haley — Mon, 19 Oct 2026 07:43:55 +0000

Last week I sat in a review where an agent had already rewritten 412 customer emails. Nobody had opened the audience preview. The send window closed in twenty minutes.

Who owned that decision? Unclear. What evidence justified it? A green checkmark. Where was the point of reversibility? Somewhere behind a modal nobody had read.

We stopped it by hand. That is not a pattern. So I built the smallest thing that would have stopped it for us, and this tutorial walks from empty folder to a service that refuses.

What we are building

The gate is a small service between the agent's proposal and the real action. It answers exactly one question: does this action class have the evidence it requires, right now?

The decision is deterministic code. A model may write the explanation later. It never writes the verdict.

That split matters. Judges that can be talked into approving are not judges.

Stage 0: Name the decision, the owner, and the stop

Before any code, write down what a reviewer must see. I use one table per action class, and I keep it in the repo so it gets reviewed like code.

Action class	Required evidence	Evidence max age	Decision owner	Stop if missing
`bulk_email_send`	audience preview URL, reviewer of the template diff	14 days	lifecycle-ops	yes
`refund_issue`	order lookup id, verifier of the amount	3 days	billing-ops	yes

Verification for this stage is social, not technical. Ask one named owner to read the row aloud. If they cannot say who stops the action, the row is not done.

Stage 1: Scaffold and prove your runtime

mkdir evidence-gate && cd evidence-gate
npm init -y
node --version

Verification: node --version prints v20 or newer. If it prints v16, stop here. The gate uses top-level ESM and you will debug the wrong thing later.

Stage 2: Write two fixtures before you write logic

Fixtures are the test evidence. Mine live in fixtures/ and each one is a recorded proposal from the agent.

{
  "action": "bulk_email_send",
  "actor": "agent:support-rewrite",
  "recipients_count": 412,
  "evidence": {
    "audience_preview_url": null,
    "template_diff_reviewed_by": null,
    "captured_at": "2026-10-18T16:20:00Z"
  },
  "reversible_until": "2026-10-19T09:00:00Z"
}

Make a second fixture where both fields are filled. Two fixtures, two outcomes, one file each.

Verification: ls fixtures shows both files. You now have a failing case and a passing case before the gate exists.

Stage 3: Write the gate as pure code

The whole decision fits in forty lines. Note that the model does not appear yet.

// gate.mjs
import fs from "node:fs";

const RULES = {
  bulk_email_send: {
    required: ["audience_preview_url", "template_diff_reviewed_by"],
    maxAgeDays: 14,
    owner: "lifecycle-ops",
  },
  refund_issue: {
    required: ["order_lookup_id", "amount_verified_by"],
    maxAgeDays: 3,
    owner: "billing-ops",
  },
};

const DAY = 86_400_000;
const now = Date.parse(process.env.GATE_NOW ?? new Date().toISOString());

function check(run) {
  const rule = RULES[run.action];
  if (!rule) {
    return { decision: "stop", missing: ["unknown_action"], owner: "platform" };
  }

  const missing = rule.required.filter((field) => !run.evidence?.[field]);
  const captured = Date.parse(run.evidence?.captured_at ?? "");
  const ageDays = Number.isNaN(captured) ? Infinity : (now - captured) / DAY;

  if (ageDays > rule.maxAgeDays) missing.push(`evidence_stale:${rule.maxAgeDays}d`);

  return {
    decision: missing.length ? "stop" : "allow",
    missing,
    owner: rule.owner,
    reversible_until: run.reversible_until ?? null,
  };
}

const run = JSON.parse(fs.readFileSync(process.argv[2], "utf8"));
const result = check(run);
console.log(JSON.stringify(result, null, 2));
process.exit(result.decision === "allow" ? 0 : 2);

Verification, and this is the moment the tutorial earns its keep:

node gate.mjs fixtures/missing-preview.json; echo "exit=$?"
node gate.mjs fixtures/complete-preview.json; echo "exit=$?"

The first prints both missing fields and exit=2. The second prints allow and exit=0. If both exit zero, your fixture is wrong, not your logic.

Stage 4: Let a model write the hand-back card

A stop reason of template_diff_reviewed_by is not a sentence a support lead can act on. So we translate it, and only on stop.

This is where I pointed the drafting step at MonkeyCode's free model access, so I could rehearse the flow without a card.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

curl -s "$MONKEYCODE_BASE_URL/chat/completions" \
  -H "Authorization: Bearer $MONKEYCODE_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{\"model\":\"$MONKEYCODE_MODEL\",\"messages\":[{\"role\":\"user\",\"content\":\"Rewrite this gate result as one plain sentence for a reviewer: $(node gate.mjs fixtures/missing-preview.json)\"}]}"

Two guardrails. The request body is the gate output, never the original proposal. And the response is a suggestion a human edits, never a verdict we accept.

Check the project page for the current endpoint shape, model list, and quota before you wire this in. Do not hardcode a model name you saw in a blog post.

Verification: run it, then read the sentence aloud. If it names the missing field and the owner, keep it. If it says "an error occurred," throw it away.

Stage 5: Put it on the free server option

Wrap the function in a tiny HTTP handler, then deploy it on MonkeyCode's free server option. I keep the gate stateless so restarts cost nothing.

curl -s -X POST "$GATE_URL/check" -d @fixtures/missing-preview.json | jq -r .decision

Verification: the response is stop, and the health endpoint returns 200. Then stop the instance and start it again. If the gate answers differently after a restart, you have hidden state and you are not done.

Stage 6: Break it on purpose

A gate that has never refused in a drill has never been tested. Here are four drills I run before trusting one.

Drill	Input change	Expected
Stale evidence	`captured_at` set to 30 days ago	stop, `evidence_stale:14d`
Unknown action	`action` renamed	stop, `unknown_action`, owner `platform`
Fake completeness	field present but empty string	stop, field listed
Clock drift	`GATE_NOW` set backwards	stop, never allow

Verification: all four exit 2. Then check the coverage question honestly. Run twenty past agent actions through the gate and count the stops. That number is your evidence coverage, and it is more useful than any trust survey.

Stage 7: Check the card, not the color

A reviewer under time pressure reads severity from color first. That breaks for a third of your team if the card is red-on-red or icon-only.

So I check four things on the hand-back card. The stop reason is a real heading. The owner is in the text, not a tooltip. Severity carries a word, not just a hue. And focus moves to the stop reason when the card opens.

Verification: keyboard only, no mouse. Tab until the card opens and confirm you hear or see the missing field before anything else. If you reach the approve button first, reorder it.

What this does not do

The gate only knows the fields you feed it. Missing evidence it was never told about passes silently, which is the failure mode I worry about most.

Rules rot. A fourteen-day window for email previews made sense for us and may be nonsense for you, and nobody will notice until it stops a legitimate send at 2 a.m.

False stops create fatigue, and fatigue creates rubber stamps. That is why the drills matter more than the happy path.

And do not use this if your action class is irreversible and high volume with no on-call owner. A gate without a human at the other end is just a slower button.

Start with one row

You do not need this whole thing to change one decision. Write one row of the table. Name one owner. Record one fixture from last week's worst send.

Then ask the question the gate asks. What evidence would have stopped us, and who was supposed to look at it?

If you want to rehearse the drafting step without a card, MonkeyCode's free model access and free server option are a reasonable place to start. Check their current terms yourself. I did, and I still read every card the model wrote.

Build the Evidence Card Before the Model Writes Anything

Haley — Sun, 18 Oct 2026 06:40:59 +0000

The review that looks like a copy edit

A reviewer opens a pull request at 4:40 on a Friday. The new error copy is shorter and friendlier. It also drops the sentence telling the user what to do next. She approves it, because the diff reads better.

The decision owner was her. The consequence lands on users with no next step. The point of reversibility was the approve button. It looked like a copy edit, so nobody treated it as one.

I keep seeing this shape. A model drafts prose, a human skims prose, and the actual decision disappears into the reading. So I want the review object to be the evidence card, not the sentence.

That is a design hypothesis, not a finding. Here is how I would test it from zero in one afternoon.

Step 1: Write the card before the prompt

Start with the fields. A card that cannot be checked is a mood board. I want each field to fail loudly when it is empty.

The card needs a surface, a person, present evidence, missing evidence, a stop condition, a recovery path, and reader-facing fallback text. Write one by hand before you touch any model.

{
  "surface": "checkout.error.card_declined",
  "decision_owner": "design-review@example.com",
  "evidence_present": ["decline-code map v3", "support macro 118"],
  "evidence_missing": ["what happens after the second failed retry"],
  "stop_condition": "evidence_missing is not empty",
  "reversibility": "copy ships in the app bundle; rollback needs a release",
  "fallback_text": "Your card was declined. Retry, or use a different card."
}

Now verify the hand-written card against the real recovery step. Open the flow. Tap retry twice. If the second tap does nothing, your missing-evidence field is right and your fallback text is probably wrong.

That is verification stage one. The card must disagree with the product before it agrees with you.

Step 2: Make the checker boring

The checker should be too dumb to be interesting. Rules in, exit codes out. No model calls inside it.

import json
import sys

REQUIRED = [
    "surface",
    "decision_owner",
    "evidence_present",
    "evidence_missing",
    "stop_condition",
    "reversibility",
    "fallback_text",
]

NOT_EVIDENCE = ("model said", "the model", "looks fine", "reads better")
NO_PATH = {"", "none", "n/a", "unknown", "rollback later"}


def load(path):
    with open(path, encoding="utf-8") as handle:
        return json.load(handle)


def problems(card):
    found = []
    for key in REQUIRED:
        if key not in card or card[key] in ("", [], None):
            found.append(f"empty or missing: {key}")

    owner = str(card.get("decision_owner", ""))
    if "@" not in owner:
        found.append("decision_owner must name a person or alias")

    evidence = [str(item).lower() for item in card.get("evidence_present", [])]
    real = [item for item in evidence if not any(bad in item for bad in NOT_EVIDENCE)]
    if not real:
        found.append("evidence_present holds no verifiable source")

    if str(card.get("reversibility", "")).strip().lower() in NO_PATH:
        found.append("reversibility has no recovery path")

    if len(str(card.get("fallback_text", ""))) < 20:
        found.append("fallback_text is too short to be actionable")

    return found


if __name__ == "__main__":
    card = load(sys.argv[1])
    found = problems(card)
    for item in found:
        print(f"BLOCK {item}")
    print("PASS" if not found else "FAIL")
    sys.exit(0 if not found else 1)

Run it on the card you wrote by hand.

python3 check_card.py cards/card_declined.json; echo "exit=$?"

Expect exit 0 only if the card is honest. If it passes while evidence_missing is still open, your stop condition is decorative. Fix the rule, not the card.

Step 3: Let the model draft, not decide

The free model access is where this gets practical. I use MonkeyCode's free model access for the drafting step only. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The operator describes a free token allowance and a free server option. Treat both as claims to check on the current plan page. I am not quoting limits, hardware, or duration I cannot verify.

Here is the call sketch. Treat it as unexecuted pseudocode. Field names and paths come from your own workspace docs, not from me.

import json
import os
import urllib.request

payload = {
    "model": os.environ["MONKEYCODE_MODEL"],
    "messages": [
        {"role": "system", "content": "Return JSON matching the card schema. No prose."},
        {"role": "user", "content": open("prompts/error-copy.md").read()},
    ],
}

request = urllib.request.Request(
    os.environ["MONKEYCODE_BASE_URL"].rstrip("/") + "/chat/completions",
    data=json.dumps(payload).encode(),
    headers={
        "Content-Type": "application/json",
        "Authorization": "Bearer " + os.environ["MONKEYCODE_API_KEY"],
    },
)

Verify by failing closed. If the response is not parseable JSON, write nothing and exit 1. A draft that arrives as prose is a draft you will paste by hand, and hand-pasting is where the missing step vanished in the first place.

Step 4: Gate the write

The generator may not touch the copy file. Only the gate may.

#!/usr/bin/env bash
set -euo pipefail

card="${1:?usage: gate.sh path/to/card.json}"
python3 check_card.py "$card"
mkdir -p approved
cp "$card" "approved/$(basename "$card")"
echo "cleared for review: $card"

Verify by attempting a blocked write. Point the gate at a card with an empty evidence_present. Expect a nonzero exit and no file in approved/. If a file appears, the gate is a suggestion.

Step 5: Break it with fixtures

Six fixtures, fixed expectations. This is the whole test plan, and it fits in one directory.

Fixture	Planted defect	Expected
`card_ok.json`	none	pass
`card_empty_evidence.json`	empty evidence list	fail
`card_model_only.json`	evidence is "the model said it reads better"	fail
`card_channel_owner.json`	owner is `#design`	fail
`card_no_path.json`	reversibility is "none"	fail
`card_terse_fallback.json`	fallback text is "Failed."	fail

for f in fixtures/*.json; do
  if python3 check_card.py "$f" >/dev/null 2>&1; then
    printf 'PASS %s\n' "$f"
  else
    printf 'FAIL %s\n' "$f"
  fi
done

Compare the output to the table, line by line. Any mismatch means the gate has a hole, and you found it before a user did. This is the only step I would not skip under deadline.

Step 6: Put the checker on the free server

The free server option is tempting for the model. I would put the checker there instead. The checker holds no key and stores no draft.

from http.server import BaseHTTPRequestHandler, HTTPServer
import json
import subprocess
import tempfile


class Handler(BaseHTTPRequestHandler):
    def do_POST(self):
        body = self.rfile.read(int(self.headers["Content-Length"]))
        with tempfile.NamedTemporaryFile("w", suffix=".json", delete=False) as handle:
            handle.write(body.decode())
            path = handle.name
        result = subprocess.run(
            ["python3", "check_card.py", path],
            capture_output=True,
            text=True,
        )
        self.send_response(200 if result.returncode == 0 else 422)
        self.end_headers()
        self.wfile.write(result.stdout.encode())


HTTPServer(("0.0.0.0", 8080), Handler).serve_forever()

Verify with a passing card and a failing one. Expect 200 then 422.

curl -sS -o /dev/null -w '%{http_code}\n' -X POST http://HOST:8080/ \
  -H 'Content-Type: application/json' --data @fixtures/card_ok.json

curl -sS -o /dev/null -w '%{http_code}\n' -X POST http://HOST:8080/ \
  -H 'Content-Type: application/json' --data @fixtures/card_no_path.json

Now the consent question, and it is a real one. Who else can read that host? If the answer is unclear, keep the model key off it entirely. Deployment is a decision about exposure, not a convenience.

Step 7: Review the card as a person would

A gate nobody can read is a gate nobody trusts. Check the card UI in this order, and stop at the first failure.

Keyboard first. Tab through the card. Focus must follow the reading order: owner, evidence, gap, stop, recovery. If focus jumps to the approve button before the missing evidence, the order is wrong.

Screen reader second. The blocked state must be spoken, not only colored. "Blocked: missing recovery step" beats a red left border every time. Then check the fallback text: if it repeats the original error, a screen reader user gets the same dead end twice.

Last, zoom to 200 percent. Long evidence strings are where cards break. If the missing-evidence field scrolls out of view, the reviewer approves what they can see.

What this does not do

It does not judge copy quality. It does not know your legal review process. It does not prove the model is consistent across runs.

Do not use this if nobody owns the copy surface. Do not use it for regulated text with a mandatory legal gate, because the gate will pass cards that legal would reject. Do not use it if you cannot pin a model version, since a moving target makes your fixtures meaningless.

And do not treat free tiers as a service level agreement. Free is a starting condition, not a promise.

The two questions I would ask in review

Which missing evidence should stop approval here? And which extra information would only add noise?

The first question protects the user. The second protects the reviewer. A card that answers both is short enough to read at 4:40 on a Friday.

If you want to try the drafting step without a budget conversation, the free model access and the free server option are where I would start. Put the checker on the server, though. Let the card hold the decision.

Rehearse the Hand-Back Before You Ship the Agent

Haley — Sat, 17 Oct 2026 05:44:57 +0000

Last week I watched a designer approve an agent's refund draft in four seconds.

The policy clause sat behind a quiet little link. She never opened it. Nothing in the log recorded the skip.

Who owned that decision? She did. What was the consequence? A refund policy might not allow. Where was the point of reversibility? Gone with one click.

I build drills for that moment now. Not demos, not trust surveys. A drill is a small, boring harness that asks one question: can a real reviewer find the missing evidence and act before the agent ships?

Below is the whole thing, zero to working. Every stage ends with a check you can run. If the check fails, stop there. Don't move on and hope.

Stage 1 — Write the missing evidence into a file

Thoughts are not testable. A JSON file is.

Pick one decision, one deliberately missing field, and one correct first action. Here is my refund scenario.

{
  "id": "refund-hold-01",
  "decision": "Approve a 42.00 EUR refund",
  "agent_state": "Draft ready, nothing sent",
  "interrupt_at": "before send",
  "missing_field_on_purpose": "Refund window: 30 days from delivery",
  "correct_first_action": "open_policy_clause",
  "stop_condition": "Reviewer approves without opening the clause"
}

Look at correct_first_action. That is what I measure. Not whether they felt confident. Not whether they liked the card. Whether they opened the clause before approving.

Check: read the file aloud to a colleague. If they cannot say who decides and what a wrong click costs, rewrite the fields.

Stage 2 — Serve it, don't slide it

Slides get skipped. Pages get clicked. I use only the Python standard library, so there is nothing to install.

import json, time, pathlib
from http.server import BaseHTTPRequestHandler, HTTPServer
from urllib.parse import urlparse

SCEN = json.loads(pathlib.Path('scenario.json').read_text())
LOG = pathlib.Path('sessions.jsonl')

PAGE = f'''<!doctype html>
<html lang='en'><head><meta charset='utf-8'>
<meta name='viewport' content='width=device-width,initial-scale=1'>
<title>Hand-back drill</title></head>
<body>
<h1>{SCEN['decision']}</h1>
<p>{SCEN['agent_state']}.</p>
<p><a href='#policy' id='clause'>Policy clause</a></p>
<p id='policy' hidden>{SCEN['missing_field_on_purpose']}</p>
<button id='approve'>Approve refund</button>
<script>
const send = (name) => fetch('/log', {{method: 'POST', body: JSON.stringify({{id: '{SCEN['id']}', action: name}})}});
document.getElementById('clause').addEventListener('click', () => {{ document.getElementById('policy').hidden = false; send('open_policy_clause'); }});
document.getElementById('approve').addEventListener('click', () => send('approve'));
</script>
</body></html>'''

class H(BaseHTTPRequestHandler):
    def send(self, body, ctype='text/html'):
        data = body.encode()
        self.send_response(200)
        self.send_header('Content-Type', ctype)
        self.send_header('Content-Length', str(len(data)))
        self.end_headers()
        self.wfile.write(data)

    def do_GET(self):
        path = urlparse(self.path).path
        if path == '/health':
            return self.send('ok', 'text/plain')
        if path == '/verdict':
            return self.send(verdict(), 'text/plain')
        return self.send(PAGE)

    def do_POST(self):
        n = int(self.headers.get('Content-Length', 0))
        event = json.loads(self.rfile.read(n) or b'{}')
        event['t'] = time.time()
        with LOG.open('a') as f:
            f.write(json.dumps(event) + '\n')
        return self.send('logged', 'text/plain')

def verdict():
    rows = [json.loads(l) for l in LOG.read_text().splitlines()] if LOG.exists() else []
    opened = any(r['action'] == 'open_policy_clause' for r in rows)
    approved = any(r['action'] == 'approve' for r in rows)
    if not approved:
        return 'no decision yet'
    return 'correct' if opened else 'skipped'

if __name__ == '__main__':
    HTTPServer(('0.0.0.0', 8000), H).serve_forever()

Start it and check the health route.

python3 drill.py &
curl -s localhost:8000/health

Check: you must see ok. If not, don't invite a reviewer yet.

Stage 3 — Log the skip, not the sentiment

Every click lands in sessions.jsonl with a timestamp. Two numbers come out: seconds to the correct first action, and skip rate.

curl -s -X POST -d '{"id":"refund-hold-01","action":"approve"}' localhost:8000/log
curl -s localhost:8000/verdict
cat sessions.jsonl | tail -n 2

If the verdict says skipped, the link was too quiet or the copy was wrong. Don't blame the reviewer yet.

Check: reproduce a skip yourself on purpose. If the log cannot catch your own skip, it cannot catch anyone's.

One caveat I learned the hard way. This verdict is global across the file. Delete sessions.jsonl between sessions, or add a session id per run.

Stage 4 — Read a script and keep your mouth shut

Facilitators ruin drills. One hint and the data becomes theatre.

Three lines, read cold. "This is a draft. Nothing is sent yet. Approve when you're ready." Then silence.

Stop the session if the reviewer asks what the clause says and you answer. That is coaching, not evidence.

Check: write those three lines down before the session, then compare. Ad-libbing is a failed check.

Stage 5 — Generate variants, then throw most away

One scenario proves nothing. I generate ten and keep the ones that change the surface without changing the correct first action.

This is where MonkeyCode's free model access helped me draft variant copy at volume. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The free-access and free-server claims are operator-supplied here, so check the current plan page yourself before relying on any limit.

Keep the brief in a file so variants stay comparable.

# prompts/variants.md
- Same decision, same correct first action.
- Vary label wording, link prominence, field order.
- Never paste the policy text into the card.
- Output one JSON object per line.

Then triage with jq instead of reading ten drafts at midnight.

jq -c 'select(.correct_first_action == "open_policy_clause")' variants.jsonl > keep.jsonl
wc -l keep.jsonl

Check: every kept variant must fail in the same direction when skipped. A variant that cannot be skipped tests nothing at all.

Stage 6 — Put it on a real URL

Localhost is a lab coat. People behave differently on a link they can open on a phone.

I put my harness on MonkeyCode's free server option and opened it on my own phone first. Follow that host's own docs for ports and process binding; your deploy command is not mine.

PORT=8000 python3 drill.py

Check: open the URL one-handed on a phone, sound off. If the approve button lands under your thumb and the clause link does not, you just found a real defect for free.

Stage 7 — Accessibility before you count anything

A drill that excludes reviewers produces confident, wrong data. I have shipped that mistake once.

Four checks: keyboard-only path to the clause, visible focus, labels that survive a screen reader, and plain language on the button. "Approve" needs a noun. Approve what, for how much?

Check: tab through without a mouse. If the clause is reachable but focus is invisible, stop and fix it.

The decision table I actually use

After five sessions I put the evidence in one table and decide.

Evidence	Ship the agent?	Action
0/5 skips, clause opened first	Yes	Ship, keep logging
1/5 skips, recoverable before send	Hold	Rewrite the link, rerun
2+/5 skips or one irreversible click	No	Redesign the hand-back
Reviewer can't explain the consequence	No	Cut the autonomy, not the copy

Two skips in five is not a small number. It is a pattern wearing a small sample.

What this drill cannot tell you

Five sessions is a smell test, not a study. I don't compute significance on five people. I look for the same failure twice.

The agent is mocked, so real failures stay messier than my file. This harness tests the hand-back, not the model.

Don't use it for regulated sign-off. Don't use it instead of production telemetry. And don't run it if nobody can watch the sessions, because unsupervised drills become usability theatre with a CSV.

It is also a poor fit for teams with no reversible moment at all. If every action is final, you have a different problem and a drill won't find it.

The question I keep asking

Which missing evidence should stop an approval? Which extra detail only adds noise?

You won't answer that in a design review. You answer it by watching one person, one click, one skipped clause. Then you fix the link and run it again.

Build a Refusal Gate in Five Stages, Then Break It

Haley — Fri, 16 Oct 2026 04:39:07 +0000

Picture a support queue on a Tuesday morning. An agent reads one ticket and drafts a refund promise. The ticket is missing the order state. Nobody notices until the customer replies.

That draft was a decision made with missing evidence. The decision owner was the support lead. The consequence landed on the customer. The point of reversibility was the send button.

So I stopped asking whether the model was trustworthy. I started asking where the refusal should live. A gate is cheaper than an apology. Here is the harness, in five stages, each with its own check.

Stage 1: Write the stop condition as data, not prose

Rules in a wiki get read once. Rules in a file get executed. Put yours in stop_conditions.json.

[
  {
    "id": "refund-promise",
    "trigger": "draft contains a refund commitment",
    "required_evidence": ["customer_id", "order_state", "refund_policy_id"],
    "owner": "support-lead",
    "allowed_action": "draft_only",
    "recovery": "ask_customer_for_order_id"
  }
]

Four fields matter most: the trigger, the evidence, the owner, the recovery. If you cannot name one owner, you have a policy, not a rule.

Verification: python -m json.tool stop_conditions.json > /dev/null && echo "schema ok" should print schema ok and exit zero. If this file is empty, your gate is a rubber stamp.

Stage 2: Stand up the harness on free model access

I keep a scratch server for gate work. The point is a machine you can break without breaking production.

MonkeyCode, an open-source project I use for this kind of experiment, advertises free model access and a free server option. Those are operator-supplied availability claims, and I treat them as availability, not as a promise. Check the current terms before you depend on them.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

ssh you@your-free-server
python3 -V                 # 3.9+ for the stdlib calls below
git clone <your-fork-url> stop-gate && cd stop-gate
python3 -m venv .venv && . .venv/bin/activate
export MONKEYCODE_API_KEY="<from your console>"
export MODEL_BASE_URL="<from your console>"
export MODEL_ID="<pick one your console lists>"

I am not naming models here, because the list changes and my naming it would age badly.

Verification: python -c "import os; print(bool(os.environ['MODEL_BASE_URL']))" prints True. One working transport call is enough for the whole harness.

Stage 3: Put the gate in front of the model

The gate checks evidence before any model call. A refusal therefore costs zero tokens.

# gate.py — reference implementation, stdlib only
import json, os, urllib.request

STOP = json.load(open("stop_conditions.json"))

def transport(prompt, base, key, model):
    """Fill in the request shape your console documents. This is a placeholder."""
    body = json.dumps({"model": model, "input": prompt}).encode()
    req = urllib.request.Request(
        base, data=body,
        headers={"Authorization": f"Bearer {key}",
                 "Content-Type": "application/json"})
    with urllib.request.urlopen(req, timeout=30) as r:
        return json.load(r)

def coverage(case, rule):
    have = set(f for f in rule["required_evidence"] if case.get(f))
    return have, set(rule["required_evidence"]) - have

def decide(case, call_model=None):
    for rule in STOP:
        _, missing = coverage(case, rule)
        if missing:
            return {"decision": "REFUSE", "rule": rule["id"],
                    "missing": sorted(missing), "owner": rule["owner"],
                    "recovery": rule["recovery"], "model_called": False}
    text = call_model(case) if call_model else None
    return {"decision": "DRAFT", "rule": None, "missing": [],
            "output": text, "model_called": call_model is not None}

Now the part people skip. Prove the refusal never reaches the model.

# test_gate.py
from gate import decide

def boom(_):
    raise AssertionError("model must not be called on a refusal")

case = {"customer_id": "c-88", "draft": "we will refund you"}
out = decide(case, call_model=boom)
assert out["decision"] == "REFUSE", out
assert out["missing"] == ["order_state", "refund_policy_id"], out
print("refusal gate ok:", out["owner"], out["recovery"])

Verification: python test_gate.py prints refusal gate ok: support-lead ask_customer_for_order_id. When a refusal still calls the model, you have not built a gate. You have built a comment.

Stage 4: Keep the discarded information in the record

A refusal is data, not silence. Log what was missing, who owns it, and how a human recovers.

def record(path, case_id, rule, missing, decision, owner, recovery, ts):
    line = {"case_id": case_id, "rule": rule, "missing": sorted(missing),
            "decision": decision, "owner": owner,
            "recovery": recovery, "ts": ts}
    with open(path, "a") as fh:
        fh.write(json.dumps(line) + "\n")

Verification: jq -r 'select(.decision=="REFUSE") | .missing[]' decisions.jsonl | sort | uniq -c | sort -rn groups the blockers by count.

 14 order_state
  9 refund_policy_id

Fourteen refusals, one field. Fix the intake form instead of the prompt. That is the whole argument for logging last.

Stage 5: Show the refusal card to a human

The card has four lines, in this order: what the agent wanted to do, which evidence is missing, who can unblock it, what approving anyway means. Focus order must match that reading order. No color-only signals. Plain words, no system names.

My review protocol is small and repeatable. Two reviewers read one card aloud, one of them with a screen reader. I count hesitations and time how long it takes each to name the owner and the recovery path.

Success measure: every refusal card names exactly one owner and one recovery path within ten seconds. Stop measure: any approval that proceeds while a required field is empty. When that happens, halt the release and fix the rule. Never relax the fixture.

What this harness does not prove

It does not prove the model is safe, only that one class of action waits for evidence. Free access is availability, not a guarantee, and terms can change.

Skip this if a regulator requires a contractual model version pinned to a document. Skip it for sub-second interactive flows, because the gate writes a record. Skip it if your team cannot name a single decision owner. Anonymous refusals teach nobody anything.

So which missing piece of evidence should stop your approval this week? Write that one rule, run the fixture, and let the gate refuse for you.

If you want a scratch machine for the fixture, the free server option in MonkeyCode will run this harness end to end.

Show the First Refusal Before You Retry

Haley — Mon, 12 Oct 2026 22:46:50 +0000

Picture this review with me, as a teaching scenario. The design lead's cursor hovered over the Approve button. The second model pass agreed with our preferred label.

The first pass had refused that same recovery modal. I asked who owned the click if a user got stuck. The scenario says the design lead owned the decision.

I asked where we could still undo the approval. The room went quiet in that familiar review-room way. A cheap retry can make agreement feel like proof.

Agreement is not the same thing as evidence. Have you watched a second pass sand down a first no? It shows up inside this kind of teaching scenario.

The dissent vanishes, and the card looks clean. That is the decision I want to slow down. The owner is the design lead, not the model.

The consequence is a recovery path that hides refusal. The point of reversibility sits before the card is marked approved. After that click, engineering may treat the copy as settled.

Can you still walk that approval back after the click? Retry debates keep surfacing in public developer threads. I am not treating those posts as proof of anything.

I am treating the urge to retry as the design problem. A nod is a mood. A stored refusal is a record.

Name the decision before a model speaks

Start from zero, even if the file already exists. Write the decision in one sentence a stranger can read. Do not start this stage by prompting a model.

I use this sentence in the worked scenario. May we replace the refused recovery label with the team's preferred label? The owner, the user, and the harm must all appear.

Here is the card I ask the team to save first.

{
  "decision_id": "recovery-label-014",
  "owner": "design-lead",
  "user_moment": "account-lockout-recovery",
  "proposed_action": "replace refused label with team label",
  "harm_if_wrong": "user cannot tell refusal from success",
  "reversible_until": "review-card-status-approved",
  "first_refusal": {
    "kept_verbatim": true,
    "summary": "Label reads as success. Undo path missing. Reason missing.",
    "source_handle": "pass-1-output.txt",
    "overwritable": false
  },
  "stops": [
    "missing verbatim first refusal",
    "retry output drops the undo gap",
    "retry output drops the refusal reason",
    "no named decision owner"
  ],
  "noise": [
    "model confidence score",
    "a third wording variant",
    "praise for the team label"
  ],
  "evidence": ["stored first pass", "lockout flow notes"],
  "hypotheses": ["team label feels clearer", "accessible name wording"],
  "retry_allowed": false
}

Verification for this stage is blunt on purpose. Read the sentence aloud without the model in the room. If you cannot name the owner, you stop.

If you cannot name the harm, you stop. A missing field should fail that command outright. A pretty sentence should not rescue a missing field.

Did the command exit clean, or did you shrug? A green exit is the only pass I accept here. A shrug means the stage is not done.

jq -e '.owner and .harm_if_wrong and .reversible_until' review-card.json

Here is the flow I want on the wall, not in someone's head. The user is in lockout, then asks for a recovery path. The refusal is stored before any retry is even offered.

lockout
  -> recovery request
  -> first refusal stored and locked
  -> stop check
  -> optional retry challenge on a disposable run
  -> owner approves or hands back

The optional run comes after the stop check, never before it. Approval sits with the owner at the end. A model nod is not a box on this flow.

Lock the first refusal

The first pass said the label sounded like success. It named a missing undo path and a missing reason. That dissent is the evidence, not a draft to polish.

I paste the refusal into the card before anyone hits retry. I do not paraphrase it into something kinder. Kind paraphrases are how eager agreement gets hired.

Now check that the refusal is present and locked. If that check fails, the retry is not a review. It is an eraser wearing a review badge.

jq -e '.first_refusal.overwritable == false and .first_refusal.kept_verbatim == true' review-card.json

Would you approve a design that deletes its own critique? I would not. The deleted line is usually the one the user needed.

I separate evidence from the hunch in the next fields. Evidence is the stored first pass and the flow notes. The hunch is that the team label feels clearer.

Do not merge those two lines into one. A hunch can guide the next research question. A hunch cannot fill an empty evidence slot.

Set the stops before a cheap retry

Free retries tempt teams to sample until the model nods. I was told MonkeyCode offers free model access and a free server option. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

I am not quoting a token total, a model list, or an uptime promise. I have not checked a primary source for those figures this week. Availability is not a quality certificate, and I will not pretend it is.

So the free lane is only the place we might rerun a bounded check. That lane is not the approver on this card. It is not allowed to overwrite the first refusal.

Here are the stops I want on the card. A missing verbatim refusal stops the retry cold. A second pass that drops the undo gap stops approval.

A confidence number without a cited flow is noise, not a key. Which missing evidence should stop this approval cold? For me, a dropped undo gap or a dropped reason stops it.

Which extra line would only add noise here? A score that flatters the preferred label is noise. A third adjective is noise too.

Verify the stops exist before you spend a free run. If you cannot say the stop out loud, do not open the model. Curiosity is not a stop condition by itself.

jq -e '.stops | length >= 3' review-card.json

A deadline is not a stop condition either. Boredom is not a stop condition on this card. Only a named gap in the refusal can stop the run.

Save this checker beside the card. It is a proposed contract, not a lab result I already ran.

# Proposed checker. Not executed against a live product.
import json, sys
card = json.load(open(sys.argv[1]))
required = ["owner", "harm_if_wrong", "reversible_until", "first_refusal", "stops"]
missing = [k for k in required if not card.get(k)]
refusal = card.get("first_refusal") or {}
if refusal.get("overwritable", True):
    missing.append("locked_first_refusal")
if card.get("retry_allowed") and missing:
    sys.exit("retry allowed while evidence is missing")
if missing:
    sys.exit("stop: " + ", ".join(missing))
print("card has the minimum refusal record")

python3 check_review_card.py review-card.json

If the refusal is unlocked, the script should exit on a stop. If the minimum record is present, it should print the minimum line. Treat that print as the expected contract, not a lab result.

Challenge the label, do not rewrite the no

I point the second pass at the stored refusal, not at a blank page. The job is to challenge our label against that refusal. The job is not to make us feel right.

This command is a proposed check, not a live call I executed. Swap in whatever client your team already trusts. Keep the second output in a new file.

# Proposed sequence. Do not overwrite pass-1-output.txt.
cat review-card.json pass-1-output.txt > retry-brief.txt
test ! -e pass-2-output.txt && echo "no second pass yet"
# After your free model lane writes pass-2-output.txt:
diff -u pass-1-output.txt pass-2-output.txt

Run the diff only after the second file exists. Until then, the test line should admit the gap. A missing second file is a stop, not a crash to ignore.

When a reply exists, I diff the claims, not the vibes. I then mark each surviving claim as evidence or hypothesis. If the second pass agrees and drops the undo gap, retry stays blocked.

Agreement with a hole is a failed retry. The jq line below is the failed-retry hand-back, not the happy path. The design lead still owns the actual decision.

jq '.retry_allowed = false' review-card.json > review-card.tmp
mv review-card.tmp review-card.json

The model does not get a quieter vote. Can a free server make this safer by itself? No, a free server cannot make the decision safer alone.

A free server can make the run cheaper to discard. Cheap discard is useful only if the first no stays put. Otherwise the team only bought a cleaner mistake.

If the retry survives, the owner may edit the flag by hand. Do not let a script flip that flag to true. Automated approval is how eager agreement gets the last word.

Read the card the way we read the product

The review card is an interface in its own right. If dissent is only a red dot, some people will miss it. I want the word Refusal in text, early in the reading order.

I tab through the card before I trust the layout. The refusal block must be reachable without a pointer. The Approve control must expose why it is disabled.

I also read it with a screen reader in the scenario. Or I ask a teammate who will actually do it. The stop reason should be announced with the disabled button.

Color alone cannot carry a do-not-retry message. Here is the accessible name I expect on the blocked action. The wording is still a hypothesis until someone hears it.

<button type="button" disabled aria-describedby="stop-reason">
  Approve recovery label
</button>
<p id="stop-reason">
  Stopped. First refusal still names a missing undo path.
</p>

Verify the association before you call the pattern done. If either grep is empty, the block is a visual hint. Visual hints fail quietly for a lot of people.

grep -n 'aria-describedby="stop-reason"' review-card.html
grep -n 'id="stop-reason"' review-card.html

Quiet failure is how bad recovery copy ships. The stored refusal and the flow notes are the only evidence here. The accessible names are still a design hypothesis.

They stay a hypothesis until someone actually hears the stop. I also check the product flow, not only the card. The user hits lockout, asks for recovery, and must see refusal as refusal.

They need a reason, a next step, and a way back. If our preferred label skips any of those, the first no stands. A smoother sentence is not an accessible sentence.

Would you know you had just been refused? If the answer is a pause, the card is not ready. Smooth copy can still abandon someone.

Run three scenarios, then stop generalizing

I do not call this protocol a finding. It is a protocol you can run tomorrow. Run it on three scenarios before you generalize.

In the first scenario, the shorter label still lacks a reason. Success means the reason stays visible or approval stays blocked. Stop if the retry calls brevity a fix.

In the second scenario, the retry adds a reason but drops undo. Success means both the reason and the undo survive. Stop if the card celebrates a partial save.

In the third scenario, the two passes disagree on severity. Success means both statements stay on the card for the owner. Stop if you delete the harsher line to reduce noise.

Noise can look like a third decorative adjective. Noise can also look like a naked score. Noise can sound like a polished model compliment.

None of those tell the locked-out person what to do next. I would not use this approach on a live incident bridge. I would not use it if you cannot store the first pass.

I would not use it as a substitute for someone else's code fix. I would not use it to bless a quota or a chip. I would not use it to bless a forever-free claim.

I was not given those facts in a form I can check. Please do not invent them in the review notes either. Unchecked numbers age fast, and this card has no room for them.

Leave the no on the table

Leave the first refusal in the record even if you later reject it. A rejection is still a real human decision. Deletion looks like the model was right all along.

The flow is simple if you say it slowly. Capture the no, then lock that no in place. Name the stop before you challenge the label.

Challenge the label, and do not rewrite the no. Then let the owner approve or hand the work back. The model can speak twice and still not own the click.

If a free model lane already sits on your bench, use it here. If a free server is there too, keep the run disposable. Keep the first refusal in the file the retry cannot touch.

Then ask whether the missing undo path is still on the card. If it is gone, the retry failed, no matter how smooth the label sounds. Would you hand that card to the locked-out person?

Show the Measured Pair Before You Approve the Page

Haley — Sun, 11 Oct 2026 19:53:40 +0000

I open a Thursday review and the page already looks finished. A side panel holds a free draft of the hero. It claims the contrast passes, and it wrote the alt text.

I ask who measured the color pair on the real screen. Nobody answers, and the publish button is still armed. Would you press it on a sentence you cannot trace?

That quiet is the decision I care about today. I own the call, and a keyboard user may miss contact. I can still hold the URL before it goes public.

People are arguing again about generated pages and real gates. I will not settle that argument here with a slogan. I will stay on the moment where the evidence is missing.

This is a scenario I use to teach the gate. It is not a customer study, and it has no success rate. Each stage below has a check you can fail.

Hold the decision before the draft talks

I write one line before I read any model output. The line says whether this page may go public today. I name the owner, the harm, and the last reversible moment.

{
  "decision": "publish_page",
  "owner": "design_reviewer",
  "harm_if_wrong": "keyboard_user_misses_contact",
  "reversible_until": "before_public_url",
  "status": "blocked"
}

I verify the card by reading every field out loud. The owner must be a person sitting in the review. The reversible moment must name a real page state.

If the owner field is blank, I stop the review. If the reversible moment only says later, I stop. A mood note about the palette would only add noise.

Who is allowed to move status from blocked to ready? If you cannot name them, you do not have a gate. A tool name in that field is not an owner.

Capture the pair a person saw

I refuse the model's contrast sentence as if it were proof. I record the foreground, background, ratio, tool, and human. I also mark the alt source and the focus record.

The numbers below are fixture data for the tutorial. They are not a lab result from a shipped page. Treat them as a shape to copy, not as a finding.

pair: hero_heading
fg: "#1A1A1A"
bg: "#F7F4EF"
ratio: 14.8
tool: contrast_checker
seen_by: reviewer
image_alt_source: missing
focus_path_recorded: no

I verify the file by checking the seen_by field carefully. That name must be a person, not the drafting model. The ratio must come from the tool a person ran.

A calm note about cream paper is only a hypothesis. The stored ratio is the evidence I am willing to defend. Feelings can stay in the notes, but they cannot unlock publish.

I point the room at W3C contrast and text-alternative guidance. I am not asking a model to remember the criterion for me. If your team uses a stricter bar, write that bar in.

Do you already see the two locks in this file? Alt source is missing, and the focus path is unmarked. Those locks matter more than a smoother hero sentence.

Say which gap stops you

I ask one research question before anyone admires the copy. Which missing field should block a publish in this review? Which extra note should we ignore on purpose today?

In this scenario I choose three gaps as hard stops. I stop for no human ratio on the measured pair. I stop for no alt source tied to a seen image.

I stop when nobody has recorded the keyboard path yet. I ignore a second paraphrase of the same hero line. I ignore a confidence score beside the drafted alt text.

I ignore a longer rationale that adds no new fact. Two fixtures make this split something you can repeat. The stop fixture omits the recorded focus path on purpose.

The pass fixture includes owner, ratio, human alt, and path. I verify the protocol by scoring both fixtures before we meet. If the stop fixture can still publish, the question was theater.

Success means the stop fixture stays blocked after the review. It does not mean the page has passed an access audit. A pass fixture only shows that notes exist, not that they are true.

Walk the flow in the order harm happens

I narrate the flow so we do not skip straight to copy. A person opens the page and tabs toward the contact control. Focus dies on the way, or the image alt tells a lie.

They leave, and we cannot undo that visit later. The recovery point has to sit before the public URL. After publish, the repair flow gets slower and colder.

open page -> tab toward contact -> focus dies or alt lies -> visitor leaves
hold the URL before it is public
after public -> unpublish -> restore human version -> tell owner -> rerun gate

I verify the flow by pointing at one state per beat. If a beat has no screen state, it is only a story. Would your current mock survive that kind of pointing test?

Use a free draft for words only

I do use a narrow helper, so I show the relationship first. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I was told the trial includes free model access for drafts.

I was also told a free server option exists for a sandbox. I will not certify a token quota or a machine size. I will not promise how long that free option remains available.

Read the live product page before you plan a study around it. I will not treat license or repo status as settled here either. I place only the evidence file on that free server sandbox.

I do not place private participant notes on a free host. I ask the free model for candidate alt text and nothing more.

task: draft_alt_words
input: evidence_file_only
forbid: new_ratios, pass_or_fail, unseen_claims
mark: candidate_not_evidence

I verify the model reply with a plain text search. I look for any ratio that was absent from the file. I also look for passes, compliant, and the word accessible.

A hit means I discard the draft and keep the file. The sandbox helped only if I can throw the output away. Should a free draft ever flip the card status to ready?

I say no, because the person who saw the screen owns that. Skip this stage when you need a formal conformance audit. Skip it when the page is already sitting on a public URL.

Skip it when your team will treat a free option as uptime. A free helper can suggest words after the locks are visible. It cannot become the evidence those locks were missing.

Run a stop script on the two fixtures

I want the boredom of a check that can fail out loud. This script is a proposal I have not executed for you. It only checks that required notes exist in the files.

It does not judge whether the stored ratio is true. Copy it as a starting gate, then point it at your own fixtures. If your filenames differ, change the command, not the stop rule.

import json, sys

card = json.load(open(sys.argv[1]))
ev = json.load(open(sys.argv[2]))
stops = []
if not card.get("owner"):
    stops.append("owner")
if ev.get("ratio") in (None, ""):
    stops.append("ratio")
if ev.get("image_alt_source") in (None, "", "missing"):
    stops.append("alt_source")
if ev.get("focus_path_recorded") is not True:
    stops.append("focus_path")
if stops:
    print("STOP", ",".join(stops))
    sys.exit(2)
print("EVIDENCE_PRESENT")

I verify this stage with two commands and two expected results. The stop fixture must print STOP and exit code two. The pass fixture must print EVIDENCE_PRESENT and then exit clean.

python3 gate.py stop_card.json stop_evidence.json
python3 gate.py pass_card.json pass_evidence.json

If the first command prints EVIDENCE_PRESENT, the gate is theater. If the second command prints STOP, the pass fixture is incomplete. I fix the files or the script before we discuss tone.

A green result is not an accessibility research finding yet. I say that aloud so a proposal is never filed as proof. The script supports the recommendation only as a field check.

Check access without asking the model

I close the draft and walk the page with a keyboard. I tab from the top until I reach the contact control. I watch whether the focus ring stays visible on each stop.

I compare the alt text with the image I can actually see. A draft that guessed a mood is not a match for me. A decorative image needs an empty alternative, not a poem.

I verify the pass by writing a path note beside the file. The note lists controls in order and confirms a visible ring. If I cannot write that note, the status stays blocked.

Color alone must not carry blocked status on our card. The word blocked must appear as text beside a visible label. Our own gate should be readable without guessing from color.

My tab pass can catch a dead path, and that is all. It cannot replace disabled people testing with their own tools. If we claim a full access review, we are overreaching badly.

What did my eyes miss because I never heard the page? A quiet control name can still block a screen reader user. I write that limit into the review so the claim stays small.

Put the page back when someone shipped

Sometimes the URL is already live when I join the review. I do not start by polishing the model's alt text. I unpublish the page or I hide the broken block first.

I restore the last version a human had actually approved. I archive the bad draft so the miss stays in the record. I name who may have been blocked, and how they retry.

I verify recovery before anyone calls the incident fully closed. The public URL must show the held state, not the draft. The archive must still contain the discarded model wording too.

The gate must fail until the new evidence file is complete. Would deleting the draft make the team feel cleaner today? It would also hide the exact miss we need to remember.

What this gate will not do

This tutorial will not make a generated page accessible by itself. It will not replace W3C guidance or assistive technology testing. It will not prove a portfolio matters because a human published.

I would not use this gate for a legal conformance claim. I would not use it when nobody can see the built screen. I would not store private research notes on a free server.

The recommendation rests on evidence you can point to today. Show the measured pair before you approve the page for publish. Show a seen alt source and a recorded keyboard path too.

Those three gaps should stop the approval in the room. A smoother paraphrase should not unlock the publish decision here. Missing proof stops the call, and extra wording is only noise.

If you hold one unpublished screen, run the card first. Make the bad fixture fail before a free draft writes words. Then decide, with the owner still named, whether it may ship.

Show the Withheld Context Before You Admit a Draft

Haley — Sat, 10 Oct 2026 16:06:51 +0000

Picture a Tuesday critique with six tired people. I am facilitating, and the clock is already rude. Someone pastes a generated empty-state line into the deck.

The line sounds kind, calm, and almost finished. Did anyone record what the model was allowed to read? I ask that, and the room goes quiet.

That quiet is the consent gap I refuse to skip. The decision owner is the critique facilitator, not the model. A false line could enter the shared component notes.

You can still throw the line away before anyone copies it. I treat this scene as a tutorial, not a field study. No metric here comes from a shipped product.

You can replay every step on your own laptop. Bring a public guideline, not a private transcript. If you cannot say the prompt aloud, you are not ready.

Name the decision before any prompt

Start with a paper decision, not a clever prompt. Who may admit this draft into critique today? What actually breaks if the line is wrong?

I write those answers before I open any generator. A missing owner stops the session before any prompt. A vague consequence is also a hard stop.

If you cannot name the undo point, wait. You are collecting hope, not a decision record. Hope is a poor ticket at the kitchen pass.

Create a folder and a ledger file first. Run the commands, then check that the file exists. This check is the first verification, not a formality.

mkdir -p critique-gate/fixtures
cd critique-gate
printf '%s\n' '{}' > fixtures/ledger.json
test -f fixtures/ledger.json && echo "ledger file exists"

The echo line is your verification for this stage. If you do not see that message, stop here. A missing file is already a stop condition.

Keep evidence off the guess pile

Here is the analogy I use with design partners. A critique deck is a kitchen pass, not a junk drawer. The cook may plate a dish, but the expediter checks the ticket.

Evidence is something a person can point to. A hypothesis is what we hope the line will do. Those two must not share a field in the ledger.

That mix is the failure this protocol is built to catch. I keep the ledger shape below as a proposal. It is not a production schema for your company.

Change the words, but do not drop the refusal list. Save that object as fixtures/ledger.json on your disk. Then confirm that the refusal array is present.

{
  "decision_owner": "critique facilitator",
  "consequence": "empty-state line may enter component notes",
  "reversibility_point": "before notes are copied into the system",
  "allowed_context": ["public empty-state guideline"],
  "refused_context": ["support transcripts", "private user quotes"],
  "evidence": ["guideline says name the next action"],
  "hypotheses": ["shorter line may reduce abandonment"],
  "draft_text": "",
  "human_note": "",
  "admit": false
}

node -e "const l=require('./fixtures/ledger.json'); if(!l.refused_context.length) process.exit(1); console.log('refusal list present')"

You should see the words refusal list present. If the script exits with an error, stop. Why require a named refusal before any prose?

Silence about withheld context is not a boundary. It is just an empty chair at the table. Empty chairs do not protect anyone in the room.

Build a local gate with no network

Disclosure: This article was prepared as part of MonkeyCode's product outreach. The outreach brief describes MonkeyCode as an open-source project. It also describes free model access for practice drafts.

The same brief names a free server option. It states a free allotment of ten million tokens. I have not rechecked that number in a live console.

Confirm the current project page before you plan around it. I would use that access only to mint a disposable draft. I would not store the consent ledger on that server.

The ledger stays in the local folder you just made. If the free option vanishes, this gate should still run. A practice bench is not your system of record.

Would you bet a design-system note on a changing limit? The flow I want is short, and it stays local at the ends. Read it once before you write the checker.

public guideline
  -> local consent ledger
  -> gate prints hold or admit-ready
  -> optional public draft on a free bench
  -> human sets admit
  -> critique may still reject
refused context never enters the prompt
ledger never moves to the free server

Write gate.js as a local checker with no network call. It only reads the ledger you pass on the command line. That limit is the whole point of the exercise.

const fs = require("fs");
const ledger = JSON.parse(fs.readFileSync(process.argv[2], "utf8"));
const missing = [];

function blank(value) {
  return value === undefined || String(value).trim() === "";
}

if (blank(ledger.decision_owner)) missing.push("decision_owner");
if (blank(ledger.consequence)) missing.push("consequence");
if (blank(ledger.reversibility_point)) missing.push("reversibility_point");
if (!Array.isArray(ledger.refused_context) || ledger.refused_context.length === 0) {
  missing.push("refused_context");
}
if (!Array.isArray(ledger.evidence) || ledger.evidence.length === 0) {
  missing.push("evidence");
}

const leaked = (ledger.evidence || []).filter((item) =>
  String(item).toLowerCase().includes("may ")
);
if (leaked.length) missing.push("evidence_looks_like_hypothesis");

if (missing.length) {
  console.error("hold", missing.join(","));
  process.exit(1);
}

console.log("admit-ready");

Notice the script never sets admit for you. A complete record is not permission to ship. Check the syntax before you trust a green line.

node --check gate.js && echo "syntax ok"

You should see the words syntax ok printed. If Node prints a parse error, fix the file. Do not paper over a syntax error with a manual note.

This script is a teaching example until you run it. It is not a judge of design quality or tone. It only checks whether a human recorded the boundary.

The may check is only a crude hint, not a parser. A real guideline might say users may retry. That phrase would trip this crude filter today.

Rename the hint or drop it before you rely on it. I would rather a false hold than a silent mix. You can delete that filter after the lesson lands.

Fail the gate on purpose

Copy the ledger and strip the refusal list. Then run the gate and welcome the failure. A green light here would mean the gate is fake.

node -e "const l=require('./fixtures/ledger.json'); delete l.refused_context; require('fs').writeFileSync('fixtures/bad.json', JSON.stringify(l, null, 2));"
node gate.js fixtures/bad.json; echo "exit:$?"

I expect a hold message and an exit code of 1. If you see admit-ready, fix the script before you trust it. A gate that smiles at a missing refusal is theater.

Now run the same command against the good file. Keep admit false until a human reads the line. The script does not need that flag to judge completeness.

node gate.js fixtures/ledger.json; echo "exit:$?"

You should see admit-ready and an exit code of 0. That result means the record is complete enough to discuss. It does not mean the sentence is good, kind, or true.

Ask for one public line

Only now do I ask a model for draft text. I paste the allowed context, never the refused context. I ask for one empty-state line and a non-use note.

If you try the free model access, keep the prompt public. Do not paste transcripts, tokens, or private user quotes. A free server is still a server someone else operates.

Would you read that prompt aloud in the critique? If the answer is no, the prompt is already too private. Shrink it until the answer becomes a bored yes.

When a draft returns, paste it into draft_text by hand. Do not let the generator edit the admit field. The human flips that field only after reading the card.

Rerun the gate after you save the hand edit. Exit code 0 means the record is ready for discussion. Critique can still reject the words without apology.

Put that rejection in human_note on the same file. Discarded wording should stay beside the refusal list. If you delete the loser, the next critique repeats the quiet.

node -e "const p='fixtures/ledger.json'; const l=require('./'+p); if(!l.draft_text){console.error('draft missing'); process.exit(1)} if(l.admit!==false){console.error('admit flipped too early'); process.exit(1)} console.log('draft parked, admit still false')"

You should see draft parked, admit still false. If admit is already true, a tool jumped the human. Put that flag back before the critique starts.

Read the card before the shiny line

The gate output is ugly on purpose, and that is fine. A critique still needs a card a person can read aloud. I speak the pattern instead of hiding status in color.

The card title is the decision, not the draft. The first line names the owner in plain text. The next line names the consequence in the same voice.

Then I read the refusal list before the draft text. Why that order, when the sentence looks finished? Because the shiny line will steal the room.

A screen reader user should reach Refuse without a mouse. The refuse control needs a visible name, not an icon alone. Status must not depend on red or green alone.

I write the hold reason as a full sentence. We did not record what the model must not read. Consent gap may confuse a new teammate in the room.

Keep both the short label and the slower sentence. If the card truncates the refusal list, call it a failure. Hidden withheld context is the same quiet we started with.

Would you approve a ticket you cannot fully scroll? I would not, and I would send the card back. Another person should name the refusal without your help.

That spoken check is the verification for this stage. If they cannot name it, the card failed. Fix the card before you discuss the sentence.

Rehearse three scenes, then stop

I rehearse three scenes before I trust the habit. Scene one uses only a public empty-state guideline. The gate should print admit-ready for that scene.

Scene two omits the owner and keeps everything else. The gate should hold, and the session should stop. No draft is worth an unnamed decision owner.

Scene three puts a private quote in allowed context. I stop by policy even if the script stays green. The checker does not understand privacy, but you do.

Add a human scan for names, tickets, and raw quotes. Do that scan before you set admit to true. A green script cannot see a name you forgot to hide.

Success here is modest, and I want it that way. A facilitator can name the owner, the harm, and the undo. A missing refusal list blocks admission every time.

A hypothesis never sits inside the evidence array. Stop the method if nobody will own the decision. Stop if the draft touches health, finance, or account data.

Stop if your team treats exit code 0 as taste approval. This checker cannot see a bad or biased sentence. It cannot score the accessibility of the product screen.

Pair the gate with a separate read of the real interface. The protocol does not replace that interface read. It only stops a draft from entering critique naked.

What the exit codes do not prove

I am not claiming a usability study or a benchmark. I have no completion rate and no customer quote. Please do not cite this article as either one.

The recommendation rests on a failure you can reproduce. The supporting evidence is the exit codes you just watched. The critique scene is a teaching frame, not a dataset.

One design hypothesis sits beside that evidence. A visible consent ledger may reduce silent pastes into critique. Test that with your own team before you write a rule.

Five facilitated sessions would be a humble start. A hallway opinion is not evidence for a system rule. Label that hypothesis until those sessions actually exist.

Do not use this flow for production audit logs. Do not use it as a legal record of consent. Do not use it where counsel expects a retention proof.

Skip it if you need named models, latency, or a permanent quota. Those facts change, and this draft does not certify them. Who else should leave this method on the shelf?

Skip it when your prompt cannot be read aloud. Skip it when you need a private region or a retention note. The brief's free access is a practice bench, not a vault.

People who want the model to approve its own copy should walk away. Teams without a named facilitator should walk away too. The gate will not invent an owner for you.

If a disposable bench would help, read the current MonkeyCode notes. Try the free model access on public text only. Keep the ledger on your machine either way.