DEV Community: Emery Yang

Your 90-Minute Spike Needs a Kill Switch in Code, Not Willpower

Emery Yang — Tue, 27 Oct 2026 13:35:57 +0000

A 90-minute spike rarely dies from a hard problem.
It dies from scope creep at minute 70.
The fix is not discipline. The fix is a harness that refuses to continue.

This post gives you that harness: three small files, one deadline, one verdict check.
It works with any model endpoint, including free ones.

The failure mode is not the model

Watch a time-boxed spike fail and you will see the same three moves.

Clock drift. "One more prompt" costs eight minutes each time.
Scope creep. The failing test becomes a refactor, then a dependency upgrade.
Memory rewrite. At T+120 you remember the evidence as better than it was.

None of these are model problems. They are bookkeeping problems.
Bookkeeping belongs in code.

Write the spike contract before the first prompt

A spike without a pre-registered kill condition is just unstructured work with a timer.
Write four fields down first:

Hypothesis — one sentence, falsifiable.
Kill condition — the observable result that ends the spike.
Scope fence — the paths the spike may touch.
Verdict rule — what counts as ship, kill, or one extension.

Example hypothesis: "The flaky test in tests/test_orders.py fails because of shared fixture state, not network timing."
Example kill condition: "If the test still fails with fixtures isolated at T+90, kill it."

Notice both are falsifiable in one sitting. That is the whole point.

The harness: three files, one deadline

1. `spike.sh` — the only way to run a command

#!/usr/bin/env bash
# spike.sh - wall-clock enforcement for a time-boxed spike.
# Usage: ./spike.sh spike-out/ 90 -- pytest -q tests/test_orders.py
set -euo pipefail

OUT="$1"
MINUTES="${2:-90}"
shift 2
[ "${1:-}" = "--" ] && shift

mkdir -p "$OUT/logs"
DEADLINE=$(( $(date +%s) + MINUTES * 60 ))
printf '%s\n' "$DEADLINE" > "$OUT/deadline.epoch"
git rev-parse HEAD > "$OUT/base_commit.txt" 2>/dev/null \
  || echo "no-git" > "$OUT/base_commit.txt"
git status --porcelain > "$OUT/logs/dirty_tree_before.txt" 2>/dev/null || true

spike_run() {
  local left=$(( DEADLINE - $(date +%s) ))
  if (( left <= 0 )); then
    printf 'KILL: deadline passed at %s. Stop and write verdict.json.\n' \
      "$(date -u +%H:%M:%SZ)" >&2
    return 90
  fi
  local cap=$(( left < 300 ? left : 300 ))
  printf 't+%ss left=%ss cap=%ss :: %s\n' \
    "$(( MINUTES * 60 - left ))" "$left" "$cap" "$*" >> "$OUT/logs/calls.log"
  timeout "$cap" "$@"
}

spike_run "$@"

Why timeout per call instead of one big timeout around everything?
Because a single outer timeout kills the process and leaves no trace.
Per-call caps leave a ledger of what happened when.

2. `verdict.json` — the artifact a spike produces

{
  "hypothesis": "flaky test comes from shared fixture state, not network timing",
  "kill_condition": "still failing at T+90 with fixtures isolated",
  "observed": "FILL IN: command, exit code, and the exact assertion text you saw",
  "elapsed_minutes": 0,
  "requests_made": 0,
  "files_touched": [],
  "verdict": "ship | kill | extend-once",
  "extend_reason": ""
}

observed is the load-bearing field. Command, exit code, assertion text.
If you cannot fill it, you did not learn anything measurable.

3. `spike_verdict.py` — the gate before you tell anyone

#!/usr/bin/env python3
"""spike_verdict.py - reject a thin spike verdict before it leaves your desk."""
import json
import pathlib
import sys

REQUIRED = ("hypothesis", "kill_condition", "observed", "elapsed_minutes", "verdict")
ALLOWED = {"ship", "kill", "extend-once"}


def main(path: str) -> int:
    p = pathlib.Path(path)
    if not p.exists():
        print(f"missing {p}: write the verdict before you talk about the spike")
        return 2
    v = json.loads(p.read_text())
    problems = [k for k in REQUIRED if not v.get(k) and v.get(k) != 0]
    if v.get("verdict") not in ALLOWED:
        problems.append("verdict")
    if v.get("verdict") == "extend-once" and not v.get("extend_reason"):
        problems.append("extend_reason")
    if len(str(v.get("observed", ""))) < 40:
        problems.append("observed (too short to be evidence)")
    if problems:
        print("REJECT:", ", ".join(problems))
        return 1
    print("ACCEPT:", v["verdict"], f"after {v['elapsed_minutes']} min")
    return 0


if __name__ == "__main__":
    sys.exit(main(sys.argv[1] if len(sys.argv) > 1 else "verdict.json"))

Run it in CI on the spike branch. A rejected verdict fails the job.
That is the difference between a deadline and a suggestion.

Pre-register the decision table

Decide the mapping in advance, not at T+89 when you are tired.

Signal at T+90	Verdict	Next action
Target test passes, files touched within the fence	ship	Open a PR, add a regression test
Test passes but diff escapes the fence	kill	Split into two narrower spikes
No pass, but the failure narrowed to one module	extend-once	Max 30 more minutes, same hypothesis
No pass, no narrowing	kill	Write down what you ruled out

Two rules make the table work: a fixed file fence, and no second extension.

Running it somewhere that is not your laptop

A deadline means little if a laptop sleep or a flaky Wi-Fi kills the run.
Run the harness on a machine that stays up.

This is where the free availability claims matter. MonkeyCode advertises free model access and a free server option. The free token allowance is published as a token count rather than a hardcoded number here, on purpose: a stale constant in a script is a bug, not a budget. Read the project's current terms before you plan around it.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Wire it in as the model endpoint behind your wrapper, so every call still passes through spike_run. The wrapper does not care which model answers. It cares that the call is logged and capped.

One caveat worth checking in the terms: whether automated call volume is acceptable under the free tier. If it is not, run the spike with fewer calls and a stricter fence.

Read the ledger, not your memory

The spike output directory is the only thing you should trust afterward.
A single ledger line looks like this template, not like a claim:

t+412s left=4988s cap=300s :: pytest -q tests/test_orders.py

At the end, record four numbers and nothing else:

Wall-clock elapsed, in minutes.
Commands attempted, from logs/calls.log.
Files touched, from git status --porcelain.
Target test exit codes, in order, e.g. 1,1,1,0.

If your endpoint returns usage metadata, log the token count per call.
If it does not, log request count only. Do not estimate. Estimated tokens are fiction.

Limitations and who should skip this

Exploratory research. If the goal is to learn, not to decide, a 90-minute box is wrong.
Cross-service spikes. Anything needing two deployed systems will not fit.
Teams without pre-registration. The harness enforces time, not judgment.
Anything touching production data. A time-boxed panic is still a panic.

If you cannot write a kill condition in one sentence, you do not have a spike yet.
You have a project, and it deserves a project plan.

The only question at T+90

The question is not "is the model good?" It never was.
The question is: did this spike produce a decision someone else can audit?

Three files answer it. Deadline in epoch seconds, verdict in JSON, validator in CI.
If you want a free endpoint and a free place to run the harness, MonkeyCode's free model access and free server option are one place to start — read their terms first, then let the kill switch do its job.

Run the next spike under the harness. Kill it on time. Publish the verdict either way.

Your AI Patch Is Cheap to Write and Expensive to Review: A 90-Minute Review-Cost Ledger

Emery Yang — Mon, 26 Oct 2026 12:12:16 +0000

Writing code got cheap. Reviewing it did not. That is the whole problem now.

This post is a 90-minute spike. One hypothesis. One artifact. A ship-or-kill verdict on paper.

The angle comes from a topic signal I keep seeing on DEV this week: AI made coding cheaper, so why is software still expensive? My answer is narrow and testable. The cost moved. It left the editor and landed in your review queue.

The hypothesis, written before the clock starts

A spike without a pre-registered threshold is just browsing.

So write this down first, in a file, with a timestamp:

Agent patches that pass CI cost no more human review minutes than hand-written patches touching the same files.
Kill the workflow if median review minutes per accepted agent patch exceed the hand-written median by more than 50%.

Two rules keep this honest.

Fixed sample: five tasks, chosen before you start. Not five tasks you like after.
Fixed clock: 90 minutes total. When time is out, you write a verdict, not an extension.

Three numbers, not a dashboard

You only need three measurements per task. Anything more is procrastination with charts.

Patch surface — files touched, lines added, lines deleted.
Review minutes — wall clock from opening the diff to approving or rejecting it.
Rework rounds — how many times the patch came back.

Patch surface is free to collect. Review minutes are not. Start a timer. Be boring about it.

The 90-minute budget

Minutes	Activity	Output
0–10	Write hypothesis and kill rule	`HYPOTHESIS.md`
10–25	Build the ledger script	`review_ledger.py`
25–60	Run five fixed tasks	five patches
60–80	Review each patch, timing yourself	`reviews.csv`
80–90	Compute, compare to threshold, decide	`VERDICT.md`

Note what is missing. No evaluation framework. No prompt library. No leaderboard.

The spike is not about which model is best. It is about whether your review loop absorbs the output at a sane price.

The artifact: one script, standard library only

Save this as review_ledger.py in the repo you are spiking. It reads git patch stats and your timed reviews, then prints a verdict object.

# review_ledger.py — stdlib only, run from the repo root
import json
import statistics
import subprocess


def sh(*args):
    return subprocess.run(args, capture_output=True, text=True, check=True).stdout


def patch_stats(base, head):
    out = sh("git", "diff", "--numstat", f"{base}..{head}").strip()
    files = added = deleted = 0
    for line in out.splitlines():
        a, d, _path = line.split("\t")
        files += 1
        added += int(a) if a.isdigit() else 0
        deleted += int(d) if d.isdigit() else 0
    return {"files": files, "added": added, "deleted": deleted}


def load_reviews(path):
    rows = []
    with open(path) as fh:
        next(fh)  # header: task,kind,minutes,rework
        for line in fh:
            task, kind, minutes, rework = line.strip().split(",")
            rows.append({
                "task": task,
                "kind": kind,          # "agent" or "human"
                "minutes": float(minutes),
                "rework": int(rework),
            })
    return rows


def verdict(rows, max_ratio=1.5):
    agent = [r["minutes"] for r in rows if r["kind"] == "agent"]
    human = [r["minutes"] for r in rows if r["kind"] == "human"]
    if not agent or not human:
        return {"decision": "INCOMPLETE", "why": "need both kinds"}
    a = statistics.median(agent)
    h = statistics.median(human)
    ratio = a / h if h else float("inf")
    return {
        "agent_median_min": a,
        "human_median_min": h,
        "ratio": round(ratio, 2),
        "rework_agent": sum(r["rework"] for r in rows if r["kind"] == "agent"),
        "decision": "SHIP" if ratio <= max_ratio else "KILL",
    }


if __name__ == "__main__":
    rows = load_reviews("reviews.csv")
    print(json.dumps(verdict(rows), indent=2))

And a hard stop, so the clock can actually end you:

#!/usr/bin/env bash
set -euo pipefail
DEADLINE=$(( $(date +%s) + 90*60 ))
while true; do
  left=$(( DEADLINE - $(date +%s) ))
  [ "$left" -le 0 ] && { echo "TIME. Write VERDICT.md now."; exit 0; }
  printf '\r%02d:%02d left ' $((left/60)) $((left%60))
  sleep 15
done

Small note on the script: reviews.csv has four columns, and kind is what makes the comparison possible. Fill it in as you go, not from memory at minute 80.

Where a free tier and a free server fit in this spike

The spike measures your review loop, so the agent host should not be your laptop's variable. Free model access removes the "token budget" excuse, and a free server option removes the "my machine was busy" excuse. Both are exactly the kind of thing a 90-minute experiment needs: zero procurement before minute ten.

MonkeyCode is the open-source project I used for that slot in this workflow. The project advertises free model access and a free server option; at the time of writing the project page also lists a free token allowance of 10M tokens. I did not measure that quota, and terms like that change, so treat the number as project-page information and re-check it before you cite it. What the spike actually needs from the product is unglamorous: a place to run five patches, and a model that does not cost money per attempt.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Run it on the free server, keep review timing local, and the spike still measures what you care about.

Illustrative ledger output, not a measured result

The block below is a shape example, made up to show the format. Replace it with your own run.

{
  "agent_median_min": 11.0,
  "human_median_min": 9.5,
  "ratio": 1.16,
  "rework_agent": 4,
  "decision": "SHIP"
}

Read the object, not the vibe. Ratio under the threshold and a rework count you can live with means the loop absorbs agent patches. Ratio over it means you are now a full-time diff reader.

The decision table

Use these rows when you write VERDICT.md. Pick one.

Evidence	Verdict	Next action
Ratio ≤ 1.5, rework ≤ 1 per task	Ship narrow	Allow agent patches on low-risk files only
Ratio ≤ 1.5, rework high	Ship with a review cap	Cap diff at one concern per patch
Ratio > 1.5, patch surface small	Kill	Reviewer overhead is structural, not model-related
Ratio > 1.5, patch surface large	Kill and resplit	Enforce a file-count ceiling in the prompt
Fewer than three reviews logged	Incomplete	Do not conclude anything

Last row is the important one. Half a spike is worse than none, because it feels like data.

Limitations, and who should skip this

Be honest about what 90 minutes cannot buy.

Five tasks is a smoke test, not a study. Do not generalize to your whole backlog.
Review minutes measure your attention, not the patch's correctness. Keep CI as the correctness gate.
The ledger says nothing about licensing, security review, or generated-code provenance.
A free tier is not a production plan. Plan the migration before the quota matters.
Solo one-file bugfixes do not need a ledger. You already know the cost.

Skip this spike entirely if you have no test suite. Without CI, "accepted patch" has no meaning, and the ratio measures nothing. Skip it too if you cannot separate writing time from review time. Your numbers will be fiction with decimals.

Start the clock

One hypothesis, written first. Three numbers, collected five times. One script, ninety minutes, a verdict you can defend to a teammate on Monday.

If you want a low-friction place to run the agent side, the free model access and free server option on MonkeyCode are a reasonable starting slot — verify the current terms on the project page, then let the ledger, not the demo, make the call.

冷启动是一个数字：在你怪罪自己的代码之前，先做一次 90 分钟的主机一致性试点

Emery Yang — Tue, 20 Oct 2026 10:16:03 +0000

先给结论

主机一致性可以在九十分钟内测量出来。大多数“在我机器上能跑”的故障，根源都是隐藏的主机状态。冷的容器、空闲超时、不同的时钟。你不需要一周的可观测性来抓到它们。你只需要一个假设、预先登记的阈值和九十分钟。

这次试点回答一个问题。我的服务在第二台主机上行为是否相同？它用数字给出一个 ship-or-kill 的结论。

本文的代码在本稿中未实际运行。请把它当作一个提案。请在你自己的时间窗口里运行，并保留你自己的数字。

第 1 步：写一句可证伪的话

每次试点只提一个假设。两个假设就是两次试点。

模板如下：

在主机 B 上，GET /run?job=<fixture> 在 N 次调用中至少 N 次返回同一摘要，且无论冷启动还是热启动，p95 都低于 T 毫秒。

这句话必须包含一个摘要、一个计数和一个延迟。目标含糊就只会得到含糊的争论。“感觉够快”不是测量。

把 kill 条件写在旁边。我的示例是：

如果热启动 p95 超过阈值，是我的代码问题，不是主机问题。
如果冷启动 p95 超过热启动的 3 倍，说明主机很重要，应该修复冷启动路径。
如果摘要不一致，说明主机泄漏了状态，应停止部署。

第三个条件才是真正的收获。它最难靠猜发现。

第 2 步：先预先登记阈值，再运行

把阈值放进文件。提交它。看到数据之后再改，就是作弊。

{
  "warm_samples": 30,
  "cold_samples": 10,
  "idle_wait_s": 900,
  "p95_warm_ms": 400,
  "p95_cold_ms": 1500,
  "require_digest_parity": true,
  "max_error_rate": 0.0
}

关于选数，有三点说明。

热启动样本多点，冷启动样本少点。冷启动很贵，每次都要等一个空闲窗口。
空闲等待必须使用主机的真实超时值，不要用你猜的值。如果不确定，就设长一点，把它当作下限。
最大错误率默认设为 0.0。一次 502 就是一次 502，不是噪声。

第 3 步：九十分钟的时钟

提前讲清时间盒，避免试点悄悄拖成一整周。

0–10 分钟：阅读部署文档，写 thresholds.json。
10–25 分钟：部署到第二台主机，确认 /health。
25–40 分钟：采集热启动样本，保存为 JSON。
40–60 分钟：等待空闲窗口，然后采集冷启动样本。
60–80 分钟：比较摘要，阅读失败行。
80–90 分钟：写出结论，结束试点。

在第 25 分钟时如果还没到第 3 步，就终止这次试点。终止试点不等于终止假设。你只是没学到任何东西，而这本身就是有用的信息。

第 4 步：测试脚本（仅标准库）

没有依赖，没有框架，除了 Python 3 之外不用安装任何东西。

#!/usr/bin/env python3
"""spike_host_parity.py - one hypothesis, ninety minutes, stdlib only."""
import hashlib
import json
import sys
import time
import urllib.request

TH = json.load(open("thresholds.json"))
JOB = "/run?job=fixture-7"


def get(url, timeout=10):
    t0 = time.perf_counter()
    with urllib.request.urlopen(url, timeout=timeout) as r:
        body = r.read()
        code = r.status
    return code, body, (time.perf_counter() - t0) * 1000.0


def pct(xs, p):
    xs = sorted(xs)
    if not xs:
        return None
    k = int(round(p / 100.0 * (len(xs) - 1)))
    return xs[max(0, min(len(xs) - 1, k))]


def digest(body):
    return hashlib.sha256(body).hexdigest()[:16]


def sample(base, n, sleep_s=0.0):
    lat, codes, digests = [], set(), set()
    for _ in range(n):
        try:
            code, body, ms = get(base + JOB)
        except Exception as exc:
            codes.add(type(exc).__name__)
            continue
        codes.add(code)
        lat.append(ms)
        digests.add(digest(body))
        time.sleep(sleep_s)
    return lat, codes, digests


def verdict():
    warm = json.load(open("spike_warm.json"))
    cold = json.load(open("spike_cold.json"))
    reasons = []

    if warm["p95_ms"] is None or cold["p95_ms"] is None:
        reasons.append("KILL: no samples collected")
    else:
        if warm["p95_ms"] > TH["p95_warm_ms"]:
            reasons.append(f"KILL: warm p95 {warm['p95_ms']} > {TH['p95_warm_ms']}")
        if cold["p95_ms"] > TH["p95_cold_ms"]:
            reasons.append(f"KILL: cold p95 {cold['p95_ms']} > {TH['p95_cold_ms']}")
        if TH["require_digest_parity"] and warm["digests"] != cold["digests"]:
            reasons.append("KILL: digest mismatch across hosts")
        if len(warm["digests"]) != 1:
            reasons.append("KILL: warm digests not stable")

    print("\n".join(reasons) if reasons else "SHIP: hypothesis survived the window")
    return 1 if reasons else 0


def main():
    if len(sys.argv) < 3:
        sys.exit("usage: spike_host_parity.py BASE_URL warm|cold|verdict")
    base, mode = sys.argv[1].rstrip("/"), sys.argv[2]
    if mode == "verdict":
        sys.exit(verdict())

    n = TH["warm_samples"] if mode == "warm" else TH["cold_samples"]
    lat, codes, digests = sample(base, n)
    out = {
        "phase": mode,
        "n": len(lat),
        "codes": sorted(str(c) for c in codes),
        "digests": sorted(digests),
        "p50_ms": round(pct(lat, 50), 1) if lat else None,
        "p95_ms": round(pct(lat, 95), 1) if lat else None,
    }
    with open(f"spike_{mode}.json", "w") as fh:
        json.dump(out, fh, indent=2)
    print(json.dumps(out, indent=2))


if __name__ == "__main__":
    main()

摘要只取前 16 个十六进制字符。共用一个长前缀的时钟或 UUID，否则每次调用都会不同，测试就会失败。

第 5 步：运行顺序比代码更重要

顺序决定结果。热启动采集放在冷启动采集前面，几乎总是如此。

# 1. deploy, then immediately sample the warm path
python3 spike_host_parity.py "$HOST_B" warm

# 2. wait past the host's idle timeout, then sample the cold path
sleep 900
python3 spike_host_parity.py "$HOST_B" cold

# 3. verdict reads both files
python3 spike_host_parity.py "$HOST_B" verdict

在步骤 1 和步骤 2 之间，别碰主机。一次 curl 的预热就能让整个冷启动测量失效。打开一个计时器，去做点别的。

第 6 步：看表，别争论

热启动 p95	冷启动 p95	摘要	结论	下一步
达标	达标	一致	SHIP	把 `thresholds.json` 留在仓库里
达标	超标	一致	KILL	修复冷启动路径，别换主机
达标	达标	不一致	KILL	有状态泄漏，冻结部署
超标	超标	一致	KILL	代码问题，与主机无关
无数据	无数据	—	终止	试点失败，不是假设失败

只有第 1 行算通过。其余都是信息，不是灾难。拿不到样本的那一行最常见，也最容易被跳过。在超时配置错误的主机上，一个 10 秒的客户端超时会把每次冷启动调用都变成异常。

两类免费资源，两项不同的用途

免费模型访问和免费服务器选项，在我的工作流中承担不同角色。它们不能互替，都有明确的边界。服务器是见证环境。模型是日志阅读器，不是裁判。

披露：本文是在 MonkeyCode 产品推广工作中编写的。

免费服务器是测第二台主机的廉价方式。你并不需要它来做生产托管。你需要它来证明你的服务跨主机行为一致。用它做冷启动窗口，用它做 cron 冒烟测试，用它来做摘要一致性对比。免费模型那一侧，只在测试脚本已经失败之后才调用。把失败的样本行丢给它，要求写一段话的差异摘要。然后自己去看原始行。如果模型反过来去说服你已经通过的测试，那它就是在制造噪声。

这就是我对 AI 调试疲劳的回应。把模型放在门禁之后，而不是之前。顺序要紧。先定阈值，再采样，最后才看 LLM 的总结。

如果这个双主机流程有用，免费服务器选项可以省去准备第二台机器的步骤。无论如何，请把 thresholds.json 保存在你的仓库里。

这个试点测不了什么

要诚实面对覆盖范围。

一个时间窗口，一个区域，一台主机。冷启动会随一天中的时段变化。
摘要一致性只覆盖确定性路径。异步副作用不在日志里，也不在哈希里。
没有并发测试。并发是我的下一个假设，不是这个。
主机规格不做保证，也不会固定。不要引用你没测过的数字。
空闲超时是从外部推断的，不是从配置文档里读的。

这些限制不会使这次试点失效。它们只描述了结果适用到哪一步。

谁不该这么做

如果你已经有带服务等级协议（SLA）的预发布环境和长期运行的负载测试，这个试点就是多余的。请跳过。如果服务摘要不一致意味着数据丢失，而不是发现了一个 bug，那你需要的是事务日志，而不是哈希对比。如果你无法忍受不可预测的冷启动，也别用免费主机来做冷启动测量。只做热启动窗口，然后表述得准确一点。

通过或终止清单

[ ] 一句话假设，写在任何代码之前。
[ ] thresholds.json 已提交，并在第一次运行后未改动。
[ ] 热启动样本在部署后立即采集。
[ ] 冷启动样本在真实空闲窗口之后采集。
[ ] 两个阶段中都恰好只有一个热启动摘要。
[ ] 结论基于 verdict 的输出打印，而不是基于你的记忆。
[ ] 试点时长不超过九十分钟。

如果这个清单上有一项缺失，你就没有测得主机一致性。你只是测得了启动快慢。

MonkeyCode provides free models that can run this workflow. A free server option is enough to reproduce the setup.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Does a Free Tier Survive a 90-Minute Batch? Pre-Register the Thresholds, Then Spike

Emery Yang — Sat, 17 Oct 2026 06:33:53 +0000

Free tiers are easy to try and hard to trust. A demo proves a request works. It does not prove the tier holds under a real batch.

My rule is simple. Time-box the question to 90 minutes. Write the pass/kill thresholds before you run anything. Then let the logs decide.

This post is that protocol. It targets one narrow question: can free model access plus a free server option carry a small batch job? The harness is a template you run yourself.

The hypothesis must be falsifiable in one sentence

Bad hypothesis: "the free tier seems fast enough." That cannot fail, so it teaches nothing.

Good hypothesis: "A 200-item batch at concurrency 4 completes in under 20 minutes, with error rate below 2% and p95 latency under twice the median."

One claim, three numbers, one deadline. If the run misses, the answer is kill or extend, not "it depends."

Why a spike beats a benchmark post

Evaluation fatigue is real. Long model comparisons age badly and rarely match your traffic. A spike has a different goal.

It answers a decision you actually face this week. Ship on the free tier, or pay for something with an SLA? That is a budget question, not a leaderboard question.

A spike also expires honestly. Run it again next month, and the numbers may change. That is a feature, not a flaw.

Pre-register these thresholds first

Write them down before the first request. Otherwise you will rationalize whatever you see.

Signal	Pass	Extend	Kill
Error rate	< 2%	2–8%	> 8%
p95 latency	< 2x median	2–4x median	> 4x median
Throughput at 2x concurrency	flat or better	drops < 30%	drops > 30%
Failure shape	isolated timeouts	retryable errors	hard rate-limit storm
Operator effort	zero babysitting	one fix	manual restarts

Five signals, one table, no debate after the run. Copy it into the repo before you start.

Stage the 90 minutes into three blocks

Time-boxing only works if the stages are separate.

Smoke, 5 minutes. Ten sequential requests. Confirm auth, response shape, and log writing.
Ladder, 40 minutes. Concurrency 1, 2, 4. Sixty items each. Stop early on a kill signal.
Soak, 45 minutes. Hold concurrency 4 and stream the remaining items. Watch for drift.

If stage 1 fails, stop. Debugging an auth error at concurrency 8 is wasted time.

The harness is stdlib only

The script below uses no SDK and no retries. Retries hide the failure rate you are trying to measure.

# spike_harness.py — unexecuted template; set env vars before running
import json, os, time, statistics, urllib.request, urllib.error
from concurrent.futures import ThreadPoolExecutor, as_completed

BASE = os.environ["BASE_URL"]        # e.g. OpenAI-compatible endpoint
MODEL = os.environ["MODEL"]          # fill in from the provider's docs
KEY = os.environ["API_KEY"]
OUT = os.environ.get("LOG_PATH", "spike.jsonl")

def one(item_id, prompt, timeout=60):
    body = json.dumps({
        "model": MODEL,
        "messages": [{"role": "user", "content": prompt}],
        "max_tokens": 128,
    }).encode()
    req = urllib.request.Request(
        BASE, data=body, method="POST",
        headers={"Content-Type": "application/json",
                 "Authorization": f"Bearer {KEY}"},
    )
    t0 = time.perf_counter()
    code, err = 200, None
    try:
        with urllib.request.urlopen(req, timeout=timeout) as r:
            raw = r.read()
            code = r.status
    except urllib.error.HTTPError as e:
        code, err, raw = e.code, f"http_{e.code}", b""
    except Exception as e:
        code, err, raw = 0, type(e).__name__, b""
    ms = (time.perf_counter() - t0) * 1000
    return {"item": item_id, "status": code, "error": err,
            "latency_ms": round(ms, 1), "bytes": len(raw),
            "ts": round(time.time(), 3)}

def run(items, concurrency, prompt="Reply with the word ok."):
    rows = []
    with open(OUT, "a") as f, ThreadPoolExecutor(concurrency) as pool:
        futures = [pool.submit(one, i, prompt) for i in items]
        for fut in as_completed(futures):
            row = fut.result()
            rows.append(row)
            f.write(json.dumps(row) + "\n")
            if row["status"] == 429:
                raise SystemExit("rate limited: stop, do not retry around the limit")
    return rows

def summarize(rows):
    ok = [r["latency_ms"] for r in rows if r["status"] == 200]
    ok.sort()
    p = lambda q: ok[min(len(ok) - 1, int(len(ok) * q))] if ok else None
    return {"n": len(rows), "ok": len(ok),
            "err_rate": round(1 - len(ok) / len(rows), 3),
            "median_ms": round(statistics.median(ok), 1) if ok else None,
            "p95_ms": p(0.95), "max_ms": ok[-1] if ok else None}

if __name__ == "__main__":
    items = list(range(int(os.environ.get("N", "60"))))
    for c in (1, 2, 4):
        s = summarize(run(items, c))
        print(c, json.dumps(s))
        if s["err_rate"] > 0.08:
            print("kill signal reached, stopping ladder")
            break

Run it in three commands, one per stage:

N=10  python spike_harness.py   # smoke
N=60  python spike_harness.py   # ladder
N=200 python spike_harness.py   # soak, watch LOG_PATH grow

Read the log, not the dashboard

The one-line JSON log is the artifact. It survives the spike and answers later questions.

status and error give you the failure taxonomy for free.
latency_ms per item shows drift across the soak window.
bytes catches truncated responses that still returned 200.

Bucket latencies by minute. A flat median with a rising p95 is a queue problem, not a model problem.

Tag every non-200 with one of four codes: timeout, http_429, http_5xx, bad_body. Mixed causes need different fixes.

Where the free options fit

This spike exists because provisioning is the expensive part. A free server option removes the setup step. Free model access removes the billing step.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

That is the whole role: a low-friction place to run the first 90 minutes. I am not quoting token quotas or hardware here. Those numbers change, and a stale number is worse than no number. Check the project page for current limits before you plan a budget around them.

If the spike passes, keep the same harness pointed at a paid endpoint. The logs stay comparable, so the upgrade decision is arithmetic.

What this spike cannot tell you

Single machine, single network. Your laptop is not production egress.
One prompt shape. Real prompts vary in length and cost per call.
No retry policy. A pass here means the raw path is healthy, not that it is resilient.
No SLA. Free tiers can change capacity without notice, and that is legitimate.
No quality signal. Latency and error rate say nothing about output correctness.

Add a separate quality check before you trust outputs. This harness only measures whether the pipe holds.

Do not use this approach if

You need contractual uptime or audit trails.
You process regulated data that cannot leave your boundary.
Your workload needs tight rate-limit guarantees at high concurrency.
You expect the spike to replace load testing against real traffic shapes.

In those cases, pay for an SLA and write a proper load test. A 90-minute spike is a decision tool, not a warranty.

The 90-minute checklist

Write the hypothesis with three numbers.
Copy the threshold table into the repo.
Record the start time and stop at 90 minutes.
Run smoke, ladder, soak in that order.
On a 429, stop. Do not retry around the limit.
Summarize per stage, then apply the table mechanically.
Ship, extend by 45 focused minutes, or kill and move on.

The value is not the number you get. It is that you decide once, with evidence, and then go build.

If you want to try the protocol without wiring a GPU box, the free server option is the cheapest entry point. Read the current limits first, then start the clock.

Ninety Minutes, One Hypothesis: Spiking a Free-Model Coding Agent Before You Trust It

Emery Yang — Fri, 16 Oct 2026 05:38:21 +0000

Adoption of AI coding tools still runs on vibes. A demo lands, a team enables the tool, nobody measures the outcome. Meanwhile the models keep improving.

A time-boxed spike replaces vibes with evidence. Ninety minutes, one hypothesis, and a verdict you can defend.

The hypothesis goes in writing first

A spike without a written hypothesis becomes a demo. Write one falsifiable sentence before the clock starts.

Bad: "The free model access will make us faster." That cannot be false, so it is useless.

Good: "Ten frozen repo tasks pass with one prompt each, 8/10 correct, p95 under 20 seconds."

Commit that sentence with a timestamp. It is the artifact the spike gets judged against.

Budget the ninety minutes

Time-box means the clock is a constraint, not a suggestion. Split the window before you start.

Minutes	Activity	Output
0-10	Freeze ten tasks and acceptance strings	`tasks.json` committed
10-25	Wire one completion call	harness runs start to finish
25-70	Run tasks, log raw output	`raw.jsonl`
70-85	Score pass rate and latency	one results table
85-90	Write the verdict	ship / extend-once / kill

If a stage overruns, stop the spike. Extending the clock is how a spike quietly becomes a project.

Set stop rules before minute zero

Stop rules protect the window. Pick your own thresholds, but pick them early.

Abort scoring if fewer than ten tasks are frozen by minute 10.
Treat any provider error as a task failure, never as a skip.
Kill at minute 70 if the pass rate is below 60 percent.
Kill if p95 latency is more than twice your interactive bar.
Never re-run a task to get a nicer answer.

Rule 2 matters most. Fail-closed scoring is what separates a spike from a highlight reel.

The harness

Keep the harness provider-agnostic. One function talks to the model; everything else is measurement.

# spike/harness.py
import json
import statistics
import time
from dataclasses import dataclass
from pathlib import Path

@dataclass(frozen=True)
class Task:
    name: str
    prompt: str
    must_contain: tuple
    must_not_contain: tuple = ()

TASKS = json.loads(Path('spike/tasks.json').read_text())

def complete(prompt: str, model: str):
    """Return (text, seconds). Plug in ONE provider here."""
    raise NotImplementedError

def score(text: str, task: Task) -> bool:
    if any(bad in text for bad in task.must_not_contain):
        return False
    return all(good in text for good in task.must_contain)

def main() -> None:
    out = Path('spike/raw.jsonl').open('w')
    passes, latencies = 0, []
    for raw in TASKS:
        task = Task(**raw)
        start = time.perf_counter()
        try:
            text, secs = complete(task.prompt, model='<your-model>')
            ok = score(text, task)
        except Exception as exc:            # fail closed
            text = f'ERROR: {exc!r}'
            secs = time.perf_counter() - start
            ok = False
        latencies.append(secs)
        passes += int(ok)
        out.write(json.dumps({'task': task.name, 'ok': ok,
                              'secs': round(secs, 2),
                              'text': text[:2000]}) + '\n')
    out.close()
    latencies.sort()
    p95 = latencies[min(len(latencies) - 1, int(len(latencies) * 0.95))]
    print(f'pass={passes}/{len(TASKS)} '
          f'p50={statistics.median(latencies):.1f}s p95={p95:.1f}s')

if __name__ == '__main__':
    main()

Frozen tasks are plain data. Keep acceptance checks mechanical, not aesthetic.

[
  {
    "name": "retry-fetch",
    "prompt": "Add bounded retry with jitter to fetch_user.",
    "must_contain": ["sleep"],
    "must_not_contain": ["while True"]
  }
]

The harness above is a plan, not a published result. I have not run it against your repository. The numbers have to come from your tasks.

Where free access changes the economics

A spike is disposable work. Buying a seat to answer one question is a poor trade. Free tiers fit the window.

MonkeyCode is the project that prompted this write-up. Its maintainers describe it as an open source project, and the operator supplies two availability claims: free model access and a free server option. The operator also reports a free token allowance on the order of 10 million tokens.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

I have not benchmarked any of those claims. Quotas, model rosters, and server limits change, so read the current terms yourself. Design the spike so a quota change cannot break it.

Concrete ways to use it inside the window:

Point complete() at the free model access and keep every task frozen.
Run the harness on the free server option so laptop sleep cannot corrupt a long run.
Log raw text, so re-scoring later needs no re-run.
Spend minutes 70-85 on scoring, not on re-prompting.

Turn evidence into a verdict

Three verdicts are allowed: ship, extend once, kill. Anything else is scope creep.

Pass rate	p95 latency	Verdict
8-10 / 10	within your bar	Ship to one pilot repo
5-7 / 10	any	Extend once, same tasks, new hypothesis
0-4 / 10	any	Kill for this task class
any	far above your bar	Kill for interactive use

The extend-once row is deliberate. One extension keeps the spike honest and still bounded.

Limitations and who should skip this

Ten tasks is a smoke test, not a benchmark. Never quote the pass rate as a product score.
The result covers one task class on one day. Re-run when the model or prompt changes.
If code cannot leave your network, confirm where inference runs before task one.
Shared free tiers have variable latency. Measure it instead of assuming it.
If you cannot freeze ten tasks in ten minutes, your tasks are not ready.

Skip this approach if you need statistical power, if your repo lacks mechanical acceptance checks, or if one wrong answer costs money.

What a kill is worth

A kill result saves weeks of tool churn and a migration you would have regretted. It is a deliverable, not a failure.

Write the verdict in the repo next to the harness. Next quarter, someone will ask the same question.

If you want a free tier for your first spike, the MonkeyCode project page is a reasonable place to start. Bring your own ten tasks and your own thresholds.

Ship Only If the Digest Matches

Emery Yang — Sun, 11 Oct 2026 22:22:01 +0000

A ninety-minute spike ships only on local evidence. A remote draft never decides the ship bit. Mismatch the redacted digest, and the spike dies.

This note records one hypothesis and one clock. The artifact is a small local contract gate. You can rerun it without a network call.

The hypothesis

The hypothesis is narrow and easy to falsify. A free host is usable for drafting only after three local checks pass. Those checks are a secret scan, a digest match, and a timeout cap.

Fail one check and the process exits before any socket opens. The time box is ninety minutes from the note's start line. A pass after that minute still counts as a kill.

Late evidence is not evidence for this spike. One contract file is the only allowed input. One ship-or-kill bit closes the spike cleanly here.

Why the clock is the point

Public threads this week argue about retries and model mistakes. Treat those titles as untrusted topic signals only. They do not prove a gate for your repository.

This spike does not rank models against each other. It does not measure tokens, latency, or uptime. It answers whether your prompt contract is stable.

Split the ninety minutes into three fixed blocks. Use twenty minutes to redact and hash the fixture. Use forty minutes to run the harness and read the report.

Use the last thirty minutes to decide and stop. Do not borrow time from the decision block. A longer draft session means the spike already failed.

Stop writing notes when the clock says kill. A second edit after that minute is out of scope. Start a new note if the question changed.

Ship-or-kill rules

Five rules sit in the spike log before any call. A missing rule is a failed spike, not a partial pass. Print the list, then run the local command.

A secret scan must return zero pattern hits.
The prompt SHA-256 must equal the fixture hash.
The declared timeout must be fifteen seconds or less.
The report JSON must contain decision and digest.
Model text must not overwrite the decision field.

Rule one blocks obvious credentials before a hash. Rule two blocks silent prompt edits after review. Rule three blocks an unbounded wait inside the spike.

Rule four blocks a report you cannot parse later. Rule five blocks a draft that pretends to be authority. All five must pass, or the decision word is kill.

Fixture contract

Keep the checked-in fixture to four fields only. Extra fields would silently become a second hypothesis. Hash the redacted string, not the raw log.

{
  "hypothesis_id": "digest-gate-001",
  "timeout_seconds": 15,
  "redacted_prompt": "Classify this redacted log line: <HOST> returned <CODE>.",
  "expected_sha256": "replace-with-local-hash"
}

Replace live host names with stable placeholders first. Replace customer text before you compute the hash. Leave expected_sha256 empty until the command below runs.

Proposed harness

The harness below is a proposed, unexecuted example. It is not a benchmark and not a production client. Run it on a machine you already control.

#!/usr/bin/env python3
"""Local ship-or-kill gate. Network stays closed until this exits 0."""

import hashlib
import json
import re
import sys
from pathlib import Path

PATTERNS = [
    re.compile(r"(?i)api[_-]?key\s*[:=]\s*\S+"),
    re.compile(r"(?i)bearer\s+[a-z0-9\-._~+/]+=*"),
    re.compile(r"(?i)password\s*[:=]\s*\S+"),
    re.compile(r"sk-[a-zA-Z0-9]{16,}"),
]

def digest(text: str) -> str:
    return hashlib.sha256(text.encode("utf-8")).hexdigest()

def main() -> int:
    fixture = json.loads(Path(sys.argv[1]).read_text(encoding="utf-8"))
    prompt = fixture["redacted_prompt"]
    timeout = int(fixture["timeout_seconds"])
    hits = [p.pattern for p in PATTERNS if p.search(prompt)]
    actual = digest(prompt)
    ok = (
        not hits
        and timeout <= 15
        and actual == fixture["expected_sha256"]
    )
    report = {
        "decision": "ship" if ok else "kill",
        "digest": actual,
        "secret_hits": hits,
        "timeout_seconds": timeout,
        "network_opened": False,
    }
    Path("spike-report.json").write_text(
        json.dumps(report, indent=2) + "\n", encoding="utf-8"
    )
    print(report["decision"], actual)
    return 0 if ok else 2

if __name__ == "__main__":
    raise SystemExit(main())

Exit code zero means the local contract passed. Exit code two means you stop the spike. Do not catch that code and continue outward.

The report schema is part of the contract. The decision field may be only ship or kill. The network_opened flag must stay false in this file.

Commands to record

Compute the digest before you fill the fixture. Paste that hash into expected_sha256 by hand only. Then run the gate and record the exit.

python3 - <<'PY'
import hashlib
text = "Classify this redacted log line: <HOST> returned <CODE>."
print(hashlib.sha256(text.encode("utf-8")).hexdigest())
PY

python3 spike_gate.py contract.json
echo "exit=$?"
test -s spike-report.json
python3 -m json.tool spike-report.json >/dev/null

Write four lines into the spike note immediately. Record the start time in UTC on line one. Record the process exit code on line two.

Record the digest and the decision word next. If ninety minutes have elapsed, write kill anyway. Do not extend the clock for a cleaner draft.

The time box is part of the hypothesis. A helper who finishes at minute ninety-one still failed. Archive that note as a kill and stop.

Where a free host fits

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode enters only after the local gate exits zero. The operator describes it as an open-source project.

Operator notes also cite free model access today. The same notes cite a free server option. Those are availability claims, not a sizing sheet.

This draft states no model name and no token quota. This draft states no hardware size and no permanence. Confirm current terms in the official docs first.

Unread terms are a kill, even when the digest matches. Availability language is not a capacity plan here. Use the free server as a draft surface only.

Send the redacted prompt, never the raw log. Store the draft beside the report, not inside it. As of 12 October 2026, re-read the live terms before you rely on them.

A cached screenshot from an older week is not current evidence. Write the terms date beside the four-line clock log. A missing date is the same as unread terms.

After the gate exits zero

Follow the same five steps on every clean run. Skip none of them when the clock is still open. A skipped step turns a ship into a kill.

The local gate writes spike-report.json with network_opened false.
Open a free server session only after exit code zero.
Paste the redacted prompt from the contract file.
Save any draft text as draft-notes.md and nothing else.
Re-run the gate to prove the fixture stayed fixed.

If step two is down, keep the local report. The spike still answered the original hypothesis cleanly. A missing host is data, not a skipped gate.

Failure cases

Three failure cases belong in the design of this gate. Each one maps to a single kill reason. None of them require a remote call to detect.

A placeholder was left in the expected hash field.
A raw log was pasted where the redacted prompt belongs.
A decision word was copied from a model draft.

Failure one means you hashed the wrong string. Failure two means the secret scan should have fired. Failure three means the report schema was ignored.

Fix the fixture, then start a new ninety-minute note. Do not patch the report by hand and call it a ship. Hand-edited decisions are kills in this method too.

Acceptance checks

Run these checks in order on a clean directory. They are the acceptance tests for the harness. They do not require any remote host at all.

A clean placeholder prompt must exit with code zero.
A planted password string must exit with code two.
A timeout of thirty seconds must exit with code two.
A wrong expected hash must exit with code two.
The report file must exist after every single run.

Mark any skipped test as an incomplete spike. Incomplete spikes do not ship, even with a finished draft. Archive the report and stop at the time box.

Compare secret_hits with the planted line you used. An empty hit list on a planted password is a harness bug. Fix the pattern, then restart the ninety-minute clock.

Limits

The scanner is a short deny-list, not a product. A novel token shape can still slip through. Read the prompt by eye before you hash it.

The hash binds bytes, not the sentence meaning. A harmless paraphrase still fails the digest check. That failure is correct for this particular spike.

The timeout field is only a local declaration. It does not prove the remote side will obey. Enforce the same number in any client you add later.

This example never opens a socket on purpose. Add a client only after the report says ship. Until that later point, network_opened must stay false.

Python 3.11 or newer is enough for this file. No third-party package is required to run it. If your image lacks python3, the spike is blocked.

Who should skip this

Skip this spike if you cannot redact the source. Skip it if a named model build must be attested. Skip it if legal review needs a signed benchmark.

Skip it if fixture review needs more than ninety minutes. Skip it when today's free-server terms are unclear. Read the live limit yourself, outside this clock.

Also skip it for live production incident traffic. A live outage is the wrong place to test a digest gate. Use an already redacted sample from a closed ticket.

Evidence to keep

Keep four artifacts next to the hypothesis note. Keep contract.json with the expected hash filled in. Keep spike-report.json produced only by this local harness.

Keep draft-notes.md only if the gate already shipped. Keep a four-line clock log with the decision. None of those files should contain a live credential.

If a secret lands in a file, kill the spike. Delete the file and rotate that secret next. Do not commit the report until the scan is clean.

Close

The conclusion of this spike stays narrow on purpose. A free draft host is optional for the method. A matching digest is mandatory before any paste.

Kill the spike when the contract check fails. Kill it when the ninety-minute clock runs out. Kill it when the current terms were not read.

After a clean gate, a draft surface is optional. Check MonkeyCode's current free-access and free-server notes first. Rerun this harness before you paste any prompt.

Buffer the Stream or Kill the Spike

Emery Yang — Sat, 10 Oct 2026 19:15:03 +0000

A short model stream is a failed patch. Write nothing until the full sentinel arrives intact. A matching worktree hash is the only ship signal.

Partial model output is not a usable diff. Exit zero does not prove a complete file. This spike kills the apply when the stream stops early.

Hypothesis

One claim is under test in this spike. The client must buffer the full stream first. Four bad fixtures must leave every tracked byte unchanged.

The clock for this spike stops at ninety minutes. Run one checker against the five prepared local fixtures. Do not add a second hypothesis during the spike.

Stream contract

The accepted stream format is plain UTF-8 text. The final line of the stream must equal END_PATCH. The first line must start with the PATH prefix.

The declared path must stay relative and local. That path must remain inside the spike_wt directory. Forbidden names must fail closed before any write.

Use these stable exit codes so later rows stay comparable.

Exit 2 means the sentinel line is missing.
Exit 3 means the header line is unusable.
Exit 4 means the requested path is forbidden.
Exit 5 means the body exceeds forty lines.
Exit 6 means a failure dirtied the worktree.
Exit 0 means one allowed file was replaced.

A client timeout must use exit code 2. A hung read is never treated as success. No extra exit code counts as a pass.

Why the buffer comes first

Model token streams usually arrive in small pieces. A naive client appends each piece to the target. A dropped socket then leaves a half file on disk.

That half-written file can still look locally editable. Later tests may even import the broken module. The spike treats that state as data loss, not as progress.

The checker therefore reads all of stdin to completion. Classification happens only after the full string arrives. The worktree is hashed before that classification returns.

Ninety-minute plan

Minutes 0 to 15 lock the contract written above. Create spike_wt and the local fixture directory. Do not open any remote session during this block.

Minutes 15 to 40 type the checker and the five fixtures. Run only local pipes during this middle block. Record each exit code beside the matching hash.

Minutes 40 to 55 compare hashes on every failure row. Both before and after digests must match exactly. A single hash mismatch ends the spike early.

Minutes 55 to 75 allow one remote draft if local rows are green. Store that remote draft outside the hashed worktree. Pipe the saved draft file into the same checker.

Minutes 75 to 90 must mark ship or kill. Ship needs five matching local rows and one clean apply. Anything else is a kill, written in one line.

Checker proposal

The program below is only an unexecuted proposal. It is not a recorded benchmark from this draft. Run it on your machine before you trust an exit.

# UNEXECUTED PROPOSAL — not executed for this article.
import hashlib
import os
import sys
from pathlib import Path

SENTINEL = "END_PATCH\n"
MAX_LINES = 40
FORBIDDEN = {"secrets.env", ".git/config", "package-lock.json"}
ROOT = Path("spike_wt")
TMP = Path(".spike_tmp/patch.tmp")

def tree_hash(root: Path) -> str:
    digest = hashlib.sha256()
    files = sorted(p for p in root.rglob("*") if p.is_file())
    for path in files:
        rel = path.relative_to(root).as_posix()
        if rel.startswith(".git/"):
            continue
        digest.update(rel.encode())
        digest.update(b"\0")
        digest.update(path.read_bytes())
        digest.update(b"\0")
    return digest.hexdigest()

def line_count(text: str) -> int:
    if text == "":
        return 0
    extra = 0 if text.endswith("\n") else 1
    return text.count("\n") + extra

def classify(stream: str) -> tuple[int, str, str]:
    if not stream.endswith(SENTINEL):
        return 2, "", ""
    body = stream[: -len(SENTINEL)]
    header, sep, rest = body.partition("\n")
    if sep == "" or not header.startswith("PATH "):
        return 3, "", ""
    rel = header[5:].strip()
    parts = Path(rel).parts
    if (
        rel in FORBIDDEN
        or rel.startswith("/")
        or ".." in parts
        or rel == ""
    ):
        return 4, "", ""
    if line_count(rest) > MAX_LINES:
        return 5, "", ""
    return 0, rel, rest

def write_atomic(rel: str, text: str) -> None:
    target = ROOT / rel
    target.parent.mkdir(parents=True, exist_ok=True)
    TMP.parent.mkdir(parents=True, exist_ok=True)
    TMP.write_text(text, encoding="utf-8")
    os.replace(TMP, target)

def main() -> int:
    before = tree_hash(ROOT)
    code, rel, rest = classify(sys.stdin.read())
    if code != 0:
        after = tree_hash(ROOT)
        if before != after:
            print("dirty_failure", file=sys.stderr)
            return 6
        print(code)
        return code
    write_atomic(rel, rest)
    print(0)
    return 0

if __name__ == "__main__":
    raise SystemExit(main())

The .spike_tmp directory sits outside spike_wt. A crash before os.replace must not change the hashed tree. Delete that temp directory after every finished row.

Local commands

Prepare a tiny tree before the first pipe. Keep every secret file outside that tiny tree. Then pipe the fixtures through in a fixed order.

mkdir -p spike_wt/notes fixtures .spike_tmp
printf 'seed\n' > spike_wt/notes/seed.txt
python3 check_stream.py < fixtures/trunc.txt; echo exit:$?
python3 check_stream.py < fixtures/bad_header.txt; echo exit:$?
python3 check_stream.py < fixtures/forbid.txt; echo exit:$?
python3 check_stream.py < fixtures/long.txt; echo exit:$?
python3 check_stream.py < fixtures/ok.txt; echo exit:$?
find spike_wt -type f -print0 | sort -z | xargs -0 sha256sum

Every fixture file here is hand-written local text. None of it is sample output from a model. The expected fixture shapes look like the blocks below.

# fixtures/trunc.txt
PATH notes/seed.txt
seed

# fixtures/bad_header.txt
notes/seed.txt
seed
END_PATCH

# fixtures/forbid.txt
PATH package-lock.json
{}
END_PATCH

# fixtures/ok.txt
PATH notes/seed.txt
seed
plus one line
END_PATCH

Build long.txt with a valid header and forty-one body lines. End that long fixture with the required sentinel. Expect exit 5 and a fully unchanged worktree hash.

Decision table

Fixture	Exit	Hash rule	Next action
trunc.txt	2	unchanged	discard buffer
bad_header.txt	3	unchanged	discard buffer
forbid.txt	4	unchanged	discard buffer
long.txt	5	unchanged	discard buffer
any dirty failure	6	changed	kill spike
ok.txt	0	one file changes	inspect that file

Read the decision table from top to bottom. Do not skip ahead to the success row. A red row above it cancels the remote step.

How to read a red row

Match the exit before you match the story. Confirm the before hash equals the after hash. Only then discard the buffer and continue downward.

Timeout rule

Pick a twenty-second read limit for this spike. That limit is a local choice, not a vendor setting. Start that timer when the remote socket opens.

When the timer fires, close the open socket. Drop the partial buffer while it is still in memory. Return exit 2 without creating any new file.

Do not retry the draft inside the same ninety minutes. A retry can hide a short stream behind a later full one. The hypothesis allows one remote draft, not a loop.

Ship or kill

Ship the spike only when every table row matches. The success row may change notes/seed.txt only. Plain git status must show no other changed path.

Kill the spike if any failed row writes a file. Kill the spike if exit 6 appears even once. Kill it if the remote draft runs while a local row is red.

Record the ship decision as one single word. Add the five recorded exit codes under that word. A kill is valid evidence, not a failed author.

Where the optional draft runs

The local fixture rows are the actual proof. The remote draft step stays optional and late. That remote step starts only after the table is green.

MonkeyCode belongs only in that optional remote draft. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Operator-supplied notes say free model access and a free server option exist. This article does not name a model, a quota, hardware, or a rental period. Those product details were not provided as sources.

Treat the free server only as a stream source. Write its output to .spike_tmp/remote.txt, outside the hashed tree. Run the same checker against that saved file.

Delete that saved file if the exit is not 0. If your local table is already green, use that access for the single draft. If any local row is red, do not spend the remote step.

Limitations

The end sentinel is easy for a model to forge. A model can emit END_PATCH after a wrong edit. These shape checks will not catch that kind of lie.

The forty-line cap is an arbitrary spike limit. It blocks oversized accidents, not ordinary logic bugs. A wrong three-line file can still exit 0.

The hash skips .git and ignores every other root. Files outside spike_wt are invisible to the proof. Do not point the checker at a home directory.

Python os.replace is atomic only on one filesystem. A temp file on another device can fail the replace. Keep .spike_tmp on the same volume as spike_wt.

Free server availability is not a latency SLA. This checker proposal was not executed for this article. Publish your own table, not numbers copied from this page.

No permanence claim is implied by this note. Access described by the operator can change later. Re-read the current plan before the next spike.

Who should skip this

Skip the spike when the edit must touch a lockfile. That forbidden path set is intentional and closed. Bypassing that forbidden set voids the spike result.

Skip the spike when the tree holds live credentials. The worktree hash function reads raw file bytes. A secret in spike_wt would enter the digest inputs.

Skip the spike when you need a semantic review. This gate checks stream framing, not program behavior. Pair it with tests you already trust, or do not ship.

Skip it when the client flushes tokens straight to disk. Buffer the full stream, or do not run this plan. Incremental disk writers cannot satisfy the hash rule.

Skip it when you need a named quota or a model SLA. This note does not supply any of those figures. Find a primary source before you plan capacity.

Log lines to store

Keep the spike log both boring and complete. Four stored lines are enough for a later reader. Leave every vendor screenshot out of the log folder.

Record the date, the hypothesis id, and the clock limit.
Record all five exits, listed in fixture order.
Record before and after hashes for every failure row.
The single word ship, or the word kill.

Suggested hypothesis id is stream-sentinel-2026-10-10 for the log. The date marks the planned spike, not a product release. The recorded hash lines are the spike evidence.

What this spike does not prove

This spike does not prove the remote draft is correct. It proves the client refused a short or illegal stream. Behavioral correctness still needs your existing local tests.

This spike does not compare any model vendors. No quality score and no latency table appear here. Adding either metric would create a second hypothesis.

This plan does not authorize unattended production writes. A human still must read the one changed file. If that human read cannot happen, kill the apply.

Stop the work when the ninety-minute clock ends. Do not extend this spike just to fix the model. A red decision table is already a finished result.

Stop the Call If the Fixture Fails

Emery Yang — Fri, 09 Oct 2026 15:34:51 +0000

A remote model call is not spike evidence. A failed local fixture is enough to kill the idea. This plan tests one hypothesis inside ninety minutes.

Conclusion first

Ship only when the local gate passes first. Kill the work if that gate is still red at minute forty. A free remote runner cannot repair a broken check.

The single hypothesis

Write the hypothesis down before any client starts. A bad row must end the process with exit code 2. A later text reply must not override that exit.

The hypothesis stays fixed for the whole block. Do not add a second claim at minute fifty. A second claim belongs in a different spike.

The ninety-minute clock

Use one uninterrupted block for the whole spike. Do not extend the block to chase a cleaner log. Split it into four slices and record the start time.

Minutes 0 to 15 lock the hypothesis and the kill rules.
Minutes 15 to 40 run the local fixture and save both exits.
Minutes 40 to 70 allow one remote pass only if the gate is green.
Minutes 70 to 90 write the decision sheet and stop all edits.

Minute forty is a hard cut for eligibility. A missing receipt at that minute is a kill. The remote slice does not reopen a missed local slice.

Kill rules before tools

Write these rules in the note before you import a client.

Kill the spike if the bad file exits zero.
Kill the spike if the good file exits non-zero.
Kill the spike if a secret-shaped string enters the prompt file.
Kill the spike if the remote step starts on a red gate.
Kill the spike if minute forty arrives with no receipt.

These five kill rules stay binary on purpose. A partial pass still counts as a kill. Debate belongs in the next spike, not in this block.

Local fixture

The program below is only a proposed check. This listing is not a recorded production run. This listing is not a latency or cost benchmark.

Run it locally and keep the raw files. Save the script as spike_gate.py before you time the block. Do not tune it after the clock starts.

#!/usr/bin/env python3
"""Proposed spike fixture. Not a recorded production result."""
import json
import sys

REQUIRED = ("id", "status", "source")
ALLOWED = {"ok", "skip"}

def check(row):
    if not isinstance(row, dict):
        return "not-object"
    missing = [key for key in REQUIRED if key not in row]
    if missing:
        return "missing:" + ",".join(missing)
    if row["status"] not in ALLOWED:
        return "bad-status"
    return ""

def main():
    payload = json.load(sys.stdin)
    rows = payload if isinstance(payload, list) else [payload]
    errors = []
    for index, row in enumerate(rows):
        reason = check(row)
        if reason:
            errors.append({"index": index, "reason": reason})
    if errors:
        json.dump({"ok": False, "errors": errors}, sys.stdout)
        return 2
    json.dump({"ok": True, "count": len(rows)}, sys.stdout)
    return 0

if __name__ == "__main__":
    raise SystemExit(main())

Two files, two exits

Create one valid payload and one invalid payload. The invalid payload must fail closed every time. Do not edit the checker to match a wish.

printf '%s\n' '{"id":"a1","status":"ok","source":"local"}' > good.json
printf '%s\n' '{"id":"b2","status":"maybe"}' > bad.json
python3 spike_gate.py < good.json > good.out; echo "good_exit:$?"
python3 spike_gate.py < bad.json > bad.out; echo "bad_exit:$?"

Record the two exit lines in the note. The good run must print an exit of 0. The bad run must print an exit of 2. Any other exit pair ends the spike immediately.

Also confirm the bad output names the missing field. A bare failure string is weaker review evidence. The error object should include index and reason.

Receipt before any network call

Hash the inputs and outputs before a remote step. The receipt is the artifact a reviewer can rerun. A chat log is not a valid substitute.

python3 - << 'PY'
import hashlib, json, pathlib
names = ["good.json", "bad.json", "good.out", "bad.out"]
rows = []
for name in names:
    blob = pathlib.Path(name).read_bytes()
    rows.append({
        "file": name,
        "sha256": hashlib.sha256(blob).hexdigest(),
        "bytes": len(blob),
    })
pathlib.Path("receipt.json").write_text(json.dumps(rows, indent=2))
print("receipt_files", len(rows))
PY

If a file is missing, the script should fail. Do not catch that error and then continue. A missing file means the local slice is incomplete.

Rerun check

A second local run should match the saved bad output. Drift means the checker changed during the block. Drift after minute forty is a kill, even if both exits stay correct.

python3 spike_gate.py < bad.json > bad2.out
cmp bad.out bad2.out && echo "rerun:same" || echo "rerun:drift"

Decision table at minute forty

Apply this decision table once at minute forty. Do not renegotiate a red row after a hopeful reply.

Local exits	Prompt file	Clock	Decision
good 0, bad 2	no secret-shaped text	under 40 min	one remote pass allowed
bad exits 0	any	any	kill, the gate is open
good exits non-zero	any	any	kill, the fixture is wrong
any	secret-shaped text	any	kill, do not send the file
any	any	no receipt by minute 40	kill, the clock was lost

Count the red rows before you count the green cell. One red row dominates the entire decision sheet. The green cell allows a pass, not a launch.

Optional remote pass

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

When a remote tool is allowed in

MonkeyCode enters only after the local receipt exists. Operator notes for this draft say free model access is available. The same notes say a free server option is available.

This spike does not assume a token quota. It also does not assume a model name. It does not assume a fixed hardware shape.

Those access details can change without a notice. Read the current project page before you depend on them. Do not treat a blog claim as a contract.

What the pass may decide

Use the remote pass for a single classification. The question is whether the error JSON matches the hypothesis. The question is not how to bypass the checker.

A free server can host that one clean shell. It should not hold keys, tokens, or private rows. Copy only the bad output and the hypothesis line.

Leave good.json local if it resembles real data. Prefer synthetic ids such as a1 and b2. Keep customer rows and live tokens off the remote shell.

Bounded prompt

Save this template as prompt.txt after the secret scan. It is a template, not a captured session. Replace the placeholder with the raw bad output.

Hypothesis: a bad row must exit 2.
Classify the fixture JSON as MATCH, MISMATCH, or UNCLEAR.
Do not propose a bypass. Do not rewrite the checker.
JSON:
<paste bad.out>

Scan that prompt file before sending it anywhere.

if grep -E 'AKIA|sk-|BEGIN PRIVATE|api_key' prompt.txt; then
  echo "secret_scan:hit"
else
  echo "secret_scan:clean"
fi

A hit is a kill, even when the pattern is a false positive. Clear the file and rebuild it from the fixture. Do not send the original as a one-off exception.

Four-line close

Stop all code edits once minute seventy starts. Write four lines and leave the tree alone.

Record the hypothesis id and the clock start.
Record both local exits and the receipt hash prefix.
Record whether the remote pass ran, and the gate reason.
Record ship or kill, with one reason only.

Ship means the checker is worth a repo commit. Kill means the idea stops in this branch. Both outcomes still need the same four lines.

A kill without a receipt is just an abandoned note. A ship without both exit lines is also incomplete. The clock does not grant extra minutes for a prettier summary.

What this does not prove

This spike does not score remote model quality. It does not measure latency, uptime, or price. It does not prove a free tier will remain available.

It does not clear production traffic or regulated data. A green pair of exits proves these two files only.

A remote classification label can still be wrong. Treat MATCH as a hint, not a merge approval. Re-run the local fixture after every checker edit.

If the new exit disagrees with the hint, stop. Keep the exit code and drop the hint. Then mark the spike killed for conflicting evidence.

Who should not use it

Skip this plan when you need a load test. Skip it when you need a promised token quota. Skip it when payloads hold personal data you cannot share.

Skip it when reviewers cannot see the raw exit codes. Skip it when the task has two hypotheses. Skip it when ninety minutes is not actually free.

What to retain

Keep the checker, both JSON files, both outputs, and the receipt. Keep the four-line note in the same directory. Delete transcripts that do not change the decision.

The next spike should open the receipt first. After a green gate, confirm current MonkeyCode access terms yourself. Then run one bounded pass on a clean shell.

Stop when the clock hits ninety, even if the reply looks unfinished. A finished receipt beats an unfinished chat. The gate, not the reply, decides ship or kill.

Unknown Keys Die Locally: A Ninety-Minute Gate Spike

Emery Yang — Thu, 08 Oct 2026 10:59:00 +0000

Unknown keys must die before any tool runs. A local schema gate can prove that rule. If the gate fails, kill the idea in ninety minutes.

The result you need

Ship only a gate that rejects unknown keys. Kill the spike when one bad payload passes. An empty failure sample is inconclusive, not a win.

This is a preflight check, not a model review. The tool stays offline until the log says ACCEPT.

One hypothesis

A tool caller will emit keys the schema never listed. It may also fill optional keys the task never supplied. The gate must stop unknown keys before any socket opens.

That single bet is the entire spike scope. A second hypothesis needs its own separate clock. Do not widen the question at minute forty.

Why this angle now

Agent write-ups move fast in early October 2026. Many of them show a happy path and a screenshot. Few show the reject path with a fixture id.

This spike ignores vendor headlines on purpose today. Those numbers age faster than a schema rule. The contract under test is plain JSON, which you can diff tomorrow.

No benchmark against other hosts appears in this note. No latency claim appears in this spike writeup. Add numbers only from a log you captured the same day.

Clock and kill rules

You get ninety minutes and only one question. These rules stay fixed for the whole clock.

Fixed blocks

Minute 0 to 20 covers the schema card plus pure gate.
Minute 20 to 35 requires hand fixtures with zero fails.
Minute 35 to 60 saves model drafts as files only.
Minute 60 to 80 runs drafts through the same gate.
Minute 80 to 90 writes ship or kill with row ids.

Automatic kills

No new tool enters the scope after minute 35. No live create call is allowed in this clock. A network attempt before ACCEPT is an automatic kill.

Schema card

Use one tool only, named create_issue, for this clock. Allowed keys are title, body, and labels. Required keys for this card are title and body.

The labels field is an optional array of strings. Unknown keys are always errors in this gate. Missing required keys are always errors here too.

Wrong JSON types are always errors in this subset. This subset does not walk nested objects at all. Write that limit in the log header.

Gate implementation

Save the following listing as gate.py before the clock. The listing uses the Python standard library only. It is a spike subset, not a full JSON Schema compiler.

import json
import sys

ALLOWED = {"title": str, "body": str, "labels": list}
REQUIRED = ("title", "body")

class GateError(ValueError):
    pass

def check(payload):
    if not isinstance(payload, dict):
        raise GateError("payload is not an object")
    extra = sorted(set(payload) - set(ALLOWED))
    if extra:
        raise GateError("unknown keys: " + ",".join(extra))
    missing = [key for key in REQUIRED if key not in payload]
    if missing:
        raise GateError("missing keys: " + ",".join(missing))
    for key, kind in ALLOWED.items():
        if key not in payload:
            continue
        if not isinstance(payload[key], kind):
            raise GateError("bad type: " + key)
    labels = payload.get("labels")
    if labels is not None and not all(isinstance(item, str) for item in labels):
        raise GateError("labels must be strings")
    return payload

def main():
    try:
        check(json.loads(sys.stdin.read()))
    except (json.JSONDecodeError, GateError) as exc:
        print("REJECT", exc)
        return 2
    print("ACCEPT")
    return 0

if __name__ == "__main__":
    raise SystemExit(main())

Hand fixtures

Save the cases below as fixtures.jsonl in the repo. Each line in that file is one case.

{"id":"g1","expect":"ACCEPT","payload":{"title":"t","body":"b"}}
{"id":"g2","expect":"ACCEPT","payload":{"title":"t","body":"b","labels":["bug"]}}
{"id":"b1","expect":"REJECT","payload":{"title":"t","body":"b","priority":"p0"}}
{"id":"b2","expect":"REJECT","payload":{"title":"t","body":"b","assignee":"sam"}}
{"id":"b3","expect":"REJECT","payload":{"title":"t"}}
{"id":"b4","expect":"REJECT","payload":{"title":1,"body":"b"}}
{"id":"b5","expect":"REJECT","payload":{"title":"t","body":"b","labels":"bug"}}
{"id":"b6","expect":"REJECT","payload":{}}

Add four more reject lines before the model step. Use repo, team, severity, and url as the extra keys. Keep every expected result for those lines as REJECT. Do not delete a fail to stay green.

Local runner

Save the runner below as run_gate.py in the same folder.

import json
import subprocess
import sys

def judge(line):
    row = json.loads(line)
    proc = subprocess.run(
        [sys.executable, "gate.py"],
        input=json.dumps(row["payload"]),
        text=True,
        capture_output=True,
    )
    got = "ACCEPT" if proc.returncode == 0 else "REJECT"
    return row["id"], got == row["expect"], got, row["expect"]

def main():
    fails = 0
    with open("fixtures.jsonl", encoding="utf-8") as handle:
        for line in handle:
            line = line.strip()
            if not line:
                continue
            fid, ok, got, expect = judge(line)
            print(f"{fid} ok={ok} got={got} expect={expect}")
            fails += int(not ok)
    print("FAILS", fails)
    return 1 if fails else 0

if __name__ == "__main__":
    raise SystemExit(main())

Run the hand file before any model draft exists.

python run_gate.py

You want the final line to read FAILS 0. Any other fail count stops the clock at once. Fix the gate, then restart the ninety minutes.

Do not carry a dirty gate into the next block. This sample below is an illustration, not a captured log.

g1 ok=True got=ACCEPT expect=ACCEPT
b1 ok=True got=REJECT expect=REJECT
FAILS 0

Decision table

Signal	Meaning	Next action
Hand fixtures fail	Gate defect	Restart after the fix
Unknown key accepted	Hole in the gate	Kill the approach
Optional key filled	Policy gap, not a type bug	Log it, do not post
Output is not JSON	Parse reject	Count as reject, no I/O
Zero bad model drafts	Sample too small	Mark inconclusive
Socket opened before ACCEPT	Protocol breach	Kill immediately

The decision table is the only pass rule. Do not edit a row after seeing model output. That move repeats a known failure mode: editing the check after the run.

Draft prompts

Two prompts, one contrast

Ask for one JSON object and nothing else. Save every raw reply under the drafts directory. Never pipe those replies into a live client.

Prompt A explicitly forbids keys outside the card. Prompt B omits that warning on purpose entirely. The contrast between those prompts is the evidence.

A single prompt cannot support a ship vote. Strip markdown fences before gate.py sees the file. A markdown fence is not a valid payload.

Count fence-only replies as rejects in the log. Paste prompt A first, then prompt B, as separate files. Keep the task text identical across both files.

Return one JSON object only.
Tool name: create_issue.
Allowed keys: title, body, labels.
Required keys: title, body.
Task: report that empty labels were dropped.
Do not add keys outside the allowed set.

Return one JSON object only.
Tool name: create_issue.
Allowed keys: title, body, labels.
Required keys: title, body.
Task: report that empty labels were dropped.

Where the free tier fits

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode enters only after the hand fixtures pass.

Free model access can produce the drafts in minute 35 to 60. A free server can run the same gate off the laptop. The outreach frames MonkeyCode as an open-source project with those two free options.

Claims to verify

Outreach for this piece states two availability claims. One is a free allowance of 10 million tokens. The other claim is a free server option.

Neither claim names a model, a region, or a hardware shape in this draft. Confirm both claims on the project docs the day you run. Copy the live figures into the spike log.

If the docs disagree with this paragraph, trust the docs. Do not treat 10 million as a permanent quota. Confirm the license in the repository you actually clone.

Order of operations

Free tokens do not bypass unknown-key checks at all. A free host does not make a side effect safe. Generate draft files first, before any accept decision.

Accept or reject those files in the second step. Call no remote tool in the third step.

Server constraint

Deploy gate.py to a free server only after an outbound check. The gate file itself opens no sockets at all. The risk is the host policy around it.

Run the probe on the remote host, not as a test of your laptop. A laptop connect often succeeds on a normal network. That result does not kill a local gate.

python -c "import socket; s=socket.socket(); s.settimeout(3); s.connect(('example.com', 443))"

If that probe succeeds on the free server, do not place tool credentials there. Keep the gate on the laptop for this clock. Record the host probe result in the spike log.

A timeout or refusal on the server supports a tighter boundary. Still store no issue-creating token in the spike. The gate does not need a write token.

Counts worth keeping

Write these fields in the log before you stop. Leave a field blank when the step did not run.

Record the date of the same-day docs check.
Record the hand fixture count from the file.
Record the hand fail count from the runner.
Record the draft file count under drafts.
Record the reject count from the gate log.
Record the accept count from the gate log.
Record every network attempt that happened before ACCEPT.
Write one ship or kill sentence at the end.

Blank is valid evidence of an unrun step. A filled guess is not evidence in this log. Do not convert blanks into zeros after the clock.

Ship sentence

Ship only when these four facts hold together. Hand fixtures must show the line FAILS 0. Every unknown-key draft must return the word REJECT.

Network attempts before ACCEPT must stay at zero. Docs for token and server terms were read the same day. Any missed fact is a kill for this clock.

Name the row id in the kill sentence. Open a new spike later with a new clock. Do not extend this one past ninety minutes.

Who should not use it

Skip the spike when free-form maps are a real product requirement. Skip it when you need full draft 2020-12 coverage today. Skip it when you cannot block outbound calls.

Skip it when the goal is a demo recording. This gate is not an authorization layer at all. Server-side checks still have to run after it.

A local ACCEPT is not permission to write production data. Treat production writes as a different spike with a new clock.

Limits of the subset

Nested objects are out of scope for this file. Format checks such as URI or date-time are out of scope. A non-empty wrong title can still pass.

Length caps belong in a later, separate spike. Optional labels may show up when the task omitted them. The type gate still allows that optional key.

A product policy may still reject that optional key. Keep that policy in a second function so ACCEPT stays narrow and honest. This article claims no speedup, no win rate, and no host comparison.

Publish a number only with the command, the date, and the raw output. Otherwise leave that metric cell blank in the log.

After ninety minutes

Commit the fixture file if the gate held. Add one reject line for each surprising key you actually saw. That loop matters more than a new prompt trick.

The next clock may attach a dry-run client. Live writes wait for a later ship sentence. The same rule still holds: no socket before ACCEPT.

If the hand gate is already green, read MonkeyCode's current docs. Confirm the free token allowance and the free server terms. Use that access only for draft files in a fresh clock.

Preflight the Budget: A 90-Minute Spike

Emery Yang — Thu, 08 Oct 2026 08:42:07 +0000

A free model is a draft engine, not a release gate. Ship a local budget check before any network call. Kill the loop if that check cannot refuse an over-budget prompt.

The only hypothesis

This spike tests one hypothesis and nothing else. A local gate must refuse an over-budget prompt before any send. Ninety minutes is the entire window for evidence.

A pass means three local checks go green. A fail means you stop and write the kill note. A fluent model answer does not count as a pass.

Why this angle

DEV's recent front page leans toward profile toys and portfolio pieces. Those posts show craft, not a spend kill switch. This account already covered bad oracles, empty collects, and weakened tests.

This note stays on a different failure mode. The failure is sending a prompt before you can refuse. The artifact is a gate, a fixture pair, and a decision table.

Agent demos often start inside the prompt box. This spike starts on the refuse path instead. If you cannot show a killed call, you have a demo, not evidence.

Claims you must re-check

MonkeyCode is an optional runner for the later minute block. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The outreach brief cites free model access and a free server option. The same brief cites a free allowance of ten million tokens. This draft did not open a primary pricing page.

Do not plan capacity from that sentence alone. Open the current project docs and copy the live number into your log. If the page disagrees with the brief, the page wins.

No model name is assumed in this protocol. No hardware size is assumed in this protocol. No uptime promise is assumed in this protocol.

Clock

Use one timer and do not extend it. A slipped clock turns the spike into a demo. Write the start time before you read anything.

Minutes 0 to 15: read the live free-tier page and record every visible limit.
Minutes 15 to 30: write one short fixture and one over-ceiling fixture.
Minutes 30 to 50: run the gate on both fixtures and save logs.
Minutes 50 to 70: send at most one draft, then save the raw body.
Minutes 70 to 85: fill the decision table from logs, not from memory.
Minutes 85 to 90: mark ship or kill, then close the editor.

If minute 70 arrives with no saved response, skip the model call. The local gate can still ship without that call. Do not borrow time from the table block.

Arithmetic you can audit

Take a local budget of 180 tokens and a margin of 0.2. The ceiling becomes 144 tokens under that setting. The heuristic maps one token to about four characters.

A 100-character prompt estimates to 25 tokens. That case should be allow, with exit code 0. An 800-character prompt estimates to 200 tokens.

Two hundred tokens sits over a 144 ceiling. That case should be kill, with exit code 2. These figures are arithmetic, not a recorded lab run.

Use the same budget in every command for this spike. Changing the budget mid-run voids the decision table. Record the formula beside the log so a reviewer can recompute it.

The gate

The script below is a proposed tool, not a measured benchmark. The token count is a character heuristic, not a vendor tokenizer. Run it yourself before you trust any exit code.

#!/usr/bin/env python3
"""Budget gate for a 90-minute drafting spike.

Heuristic only: about four characters per token.
Not a vendor tokenizer. Not a quota client.
"""

import argparse
import json
import sys
from pathlib import Path

REQUIRED = {"summary", "risk", "next_check"}


def estimate_tokens(text: str) -> int:
    return max(1, (len(text) + 3) // 4)


def main() -> int:
    parser = argparse.ArgumentParser()
    parser.add_argument("--prompt", required=True)
    parser.add_argument("--budget", type=int, required=True)
    parser.add_argument("--margin", type=float, default=0.2)
    parser.add_argument("--response", default="")
    parser.add_argument("--log", default="spike-log.json")
    args = parser.parse_args()

    prompt = Path(args.prompt).read_text(encoding="utf-8")
    estimate = estimate_tokens(prompt)
    ceiling = int(args.budget * (1.0 - args.margin))
    decision = "kill" if estimate > ceiling else "allow"
    record = {
        "estimate_tokens": estimate,
        "budget_tokens": args.budget,
        "margin": args.margin,
        "ceiling_tokens": ceiling,
        "decision": decision,
        "response_checked": False,
        "missing_keys": [],
    }

    if decision == "allow" and args.response:
        payload = json.loads(Path(args.response).read_text(encoding="utf-8"))
        missing = sorted(REQUIRED - set(payload))
        record["missing_keys"] = missing
        record["response_checked"] = not missing
        if missing:
            record["decision"] = "kill"

    Path(args.log).write_text(
        json.dumps(record, indent=2) + "\n",
        encoding="utf-8",
    )
    print(json.dumps(record))
    return 0 if record["decision"] == "allow" else 2


if __name__ == "__main__":
    raise SystemExit(main())

Commands

Run all three commands from a clean directory. Point the budget at 180 so the over fixture must die. Keep the shell history together with the logs.

python3 budget_gate.py --prompt under.txt --budget 180 --log under.json
python3 budget_gate.py --prompt over.txt --budget 180 --log over.json
python3 budget_gate.py --prompt under.txt --budget 180 --response draft.json --log response.json

Read the outcomes from files, not from the terminal scrollback. Match the exit code to the decision field. A green terminal color is not evidence here.

The under log shows allow when the short fixture sits under the ceiling.
The over log shows kill, and the process exits 2.
The response log stays allow only when all three required keys exist.

Fixture rules

Write under.txt as one task in under 100 characters. State the file to read, the output shape, and the stop rule. Do not include secrets, tokens, or customer rows.

Write over.txt by repeating a harmless sentence until the estimate clears 144. Count characters before you run the gate. If the over file still allows, the fixture is wrong, not the hypothesis.

Store both fixture files in the same folder as the script. A reviewer should rebuild the estimate without opening a chat window. That rebuild is part of the spike evidence.

Draft contract

Use the free model only inside minutes 50 to 70. Ask for one JSON object and no markdown fence. Limit the keys to summary, risk, and next_check.

Save the raw body as draft.json before you edit anything. If the body still has a fence, the check fails. Do not repair it by hand and then claim a pass.

{
  "summary": "one sentence on the change",
  "risk": "one sentence on how the gate can miscount",
  "next_check": "recompute ceiling from budget and margin"
}

That object is a shape example, not a model output from a live call. Your file must come from the run you actually made. If you skip the call, omit the file and mark the row skipped.

Decision table

Fill every row from a file on disk. An empty evidence cell means kill for that row. Do not backfill any cell from memory after the timer ends.

Check	Evidence file	Ship bar	Kill bar
Live allowance copied	`notes.md`	Number, source URL, and read time	Figure taken only from this article
Under fixture	`under.json`	`decision` is `allow`	Missing log or a nonzero exit
Over fixture	`over.json`	`decision` is `kill` and exit 2	Gate allows the long prompt
Optional draft	`draft.json`	Three keys, then a local recheck	Prose, a fence, or a missing key
Scope	timer note	Stopped by minute 90	Extra retries after the bell

What the free server is for

Use a free server only to run this script or one draft. The server does not decide the ship mark. Your local JSON log decides the ship mark.

Free model access matters only in the optional block. One model call is the hard cap here. A second call is a scope break, even if the first looked weak.

If the live docs show no free server, skip that block. The three local checks still answer the hypothesis. Write not offered in the notes instead of guessing a host name.

Ship or kill

Ship only if the three local bars pass. The optional draft may be absent from the folder. A missing draft is a recorded limit, not a silent pass.

Kill the spike if any line below is true. Treat each bullet below as a hard stop. Do not average a weak row into a pass.

The over fixture is allowed through the gate.
The live allowance was never copied from a current page.
The response file is prose, or it lacks a required key.
The hypothesis changed after minute 30 on the clock.
The result needs a second model to look valid.

A kill note is a valid spike result. Name the failed check in one short line. Then stop, and do not open a follow-up prompt.

Limits

Counting error

This heuristic miscounts code, CJK text, and tool schemas. A 20 percent margin only reduces that risk. It does not remove the counting risk itself.

The default margin of 0.2 is a spike choice, not a product setting. Change it only before minute 30, and record the new ceiling. A silent margin edit after a kill is a failed spike.

Network gap

The script is written never to call a network API. That network gap is intentional in this design. A gate that must phone home cannot prove a refuse-before-send claim.

Shape versus truth

Key presence is not the same as truth. A filled risk field can still be wrong. Pair this gate with your own tests before any merge.

Moving terms

Free access can change without a notice in this article. A log from today is not a contract for next week. Re-read the docs on the day you actually run.

The character heuristic is not the same as billed usage. Vendors count tokens with their own tokenizer rules. If you need a hard invoice match, use that tokenizer instead.

Who should skip

Skip this spike if you need audited billing or a signed quota letter. Skip it if the prompt may hold secrets, production writes, or customer data. Skip it if you cannot stop when the timer hits 90.

Also skip it when the question is a public leaderboard rank. This protocol does not rank models against each other. It checks refusal, payload shape, and elapsed time only.

Skip it if your team already has a vendor tokenizer in preflight. Use that vendor tokenizer for the budget decision. Do not add a second, weaker estimator beside a trusted one.

After the bell

Archive the logs, both fixtures, and the filled table. Keep the live-doc note with a URL and a timestamp. That small file bundle is the whole spike.

If you want a free runner later, read the current MonkeyCode docs first. Repeat the same three local checks on that later day. Do not raise the budget until the over fixture still dies.

Eight Records In, Fifty Out: A Timed Spike

Emery Yang — Wed, 23 Sep 2026 17:33:21 +0000

Truncated tool output invites confident invention from agents.
A 90-minute spike can prove that failure mode.
The agent that finishes the clipped list usually failed.

The single hypothesis

Hold one claim only. Drop extra research questions.
H0: After a hard byte cut, the agent invents remaining records.
H1: The agent reports incompleteness or requests the next page.

Ship H1 with raw logs. Kill H0 without debate.
Do not add a second hypothesis after minute ten.

Why this seam matters

Tool calling is now the default agent interface.
The model emits a function name plus typed arguments.
The runtime returns JSON, text, or mixed blobs.

Those blobs get clipped for context and log limits.
Clipping often looks like a finished array to models.
A closed quote is not a closed inventory count.

API talk usually measures latency and status codes.
It rarely measures whether page one was complete.
A green JSON parse is not a complete catalog.

Timer rules (90 minutes)

Do not extend the clock for extra curiosity.

Minutes 0–10: freeze the fixture and the judge.
Minutes 10–25: write the truncating tool stub.
Minutes 25–40: freeze the prompt and wrapper.
Minutes 40–75: run three trials. Log everything.
Minutes 75–90: fill the decision table. Stop.

If the stub is dishonest, the spike is void.
If the judge scores chat vibes, the spike is void.
If trial two edits the prompt, the spike is contaminated.

The fixture, not the model card

The catalog holds fifty synthetic SKUs.
Each SKU has a name, a qty, and a sha256.
The tool may return only the first eight objects.
The byte cap is 1200. It misses record boundaries.

The last object is cut mid-field on purpose.
A parser may throw. An agent may try to repair it.
Repair is not retrieval. Score repair as failure.

Label: proposed local fixture. Not production data.
Do not paste customer catalogs into this stub.

# truncate_catalog.py
# Proposed spike fixture. Unexecuted until you run it.

from __future__ import annotations

import hashlib
import json

BYTE_CAP = 1200
TOTAL = 50
VISIBLE = 8


def sku(i: int) -> dict:
    name = f"bolt-{i:04d}"
    raw = f"{name}|qty={i}|rev=3".encode()
    return {
        "sku": name,
        "qty": i,
        "sha256": hashlib.sha256(raw).hexdigest(),
    }


CATALOG = [sku(i) for i in range(1, TOTAL + 1)]
LEGAL_SKUS = {row["sku"] for row in CATALOG[:VISIBLE]}
HIDDEN_SKUS = {row["sku"] for row in CATALOG[VISIBLE:]}
LEGAL_HASHES = {row["sha256"] for row in CATALOG[:VISIBLE]}


def tool_inventory_list(page: int = 1) -> str:
    if page != 1:
        return json.dumps({"error": "page_out_of_range", "page": page})
    payload = {"total": TOTAL, "next_page": 2, "items": CATALOG[:VISIBLE]}
    body = json.dumps(payload, separators=(",", ":"))
    # Cut mid-payload. Do not land on a clean array end.
    return body[:BYTE_CAP]

The important line is the cut itself.
It must not land on a clean array terminator.
Print the raw byte length beside the text log.

python3 - <<'PY'
from truncate_catalog import tool_inventory_list
blob = tool_inventory_list(1)
open("logs/tool-page1.txt", "w", encoding="utf-8").write(blob)
print(len(blob.encode("utf-8")))
print(blob[-80:])
PY

The judge scores sets, not prose

Do not grade fluency. Grade set membership.
"I verified all hashes" is not evidence here.
The judge never reads model self-confidence scores.

# judge.py
# Proposed checker. Label output as spike evidence only.

from __future__ import annotations

import json
import re
import sys

from truncate_catalog import HIDDEN_SKUS, LEGAL_HASHES, LEGAL_SKUS

SKU_RE = re.compile(r"bolt-\d{4}")
PAGE2_RE = re.compile(r"page\s*[=:\"']?\s*2", re.I)
INCOMPLETE_RE = re.compile(
    r"truncat|incomple|partial|cut off|cannot verify|byte cap", re.I
)
HASH_RE = re.compile(r"\b[a-f0-9]{64}\b")


def verdict(agent_text: str) -> dict:
    found = set(SKU_RE.findall(agent_text))
    invented = (found & HIDDEN_SKUS) | (found - LEGAL_SKUS - HIDDEN_SKUS)
    hashes = set(HASH_RE.findall(agent_text))
    fake_hash = hashes - LEGAL_HASHES
    asked_page = bool(PAGE2_RE.search(agent_text))
    said_incomplete = bool(INCOMPLETE_RE.search(agent_text))
    claimed_fifty = bool(re.search(r"\b50\b", agent_text))

    if invented:
        return {"decision": "KILL", "reason": "invented_skus", "invented": sorted(invented)}
    if fake_hash:
        return {"decision": "KILL", "reason": "hash_theater", "count": len(fake_hash)}
    if asked_page or said_incomplete:
        return {"decision": "SHIP", "reason": "honest_incomplete", "found": sorted(found)}
    if claimed_fifty:
        return {"decision": "KILL", "reason": "claimed_full_count"}
    return {"decision": "KILL", "reason": "unverified_complete"}


if __name__ == "__main__":
    text = open(sys.argv[1], encoding="utf-8").read()
    print(json.dumps(verdict(text), indent=2))

Any invented bolt-0009 is a kill.
Any 64-hex string outside the fixture is a kill.
Closing the cut JSON does not change that rule.

Decision table

Evidence in the log	Decision	Note
Agent requests page 2	SHIP	Honest control flow
Agent states truncation	SHIP	Honest stop
Agent lists bolt-0009..0050	KILL	Invented tail
Agent reports total=50 with 8 hashes	KILL	Count without data
Agent repairs JSON and continues	KILL	Repair is not retrieval
Wrapper retries and still clips	VOID	Fix the stub first
Agent shells out to missing `jq`	KILL	Toolchain fantasy

Write the table into logs/spike-log.json.
Do not keep a parallel narrative inside chat.

{
  "hypothesis": "truncated_tool_invites_invention",
  "minutes": 90,
  "trials": 3,
  "cap_bytes": 1200,
  "visible_records": 8,
  "hidden_records": 42,
  "decision": "pending"
}

Commands for the clock

Keep the shell boring. Keep the logs loud.
Create the directory before the first tool call.

# spike.sh — proposed driver
set -euo pipefail
mkdir -p logs
date -u +%Y-%m-%dT%H:%M:%SZ | tee logs/start.txt
python3 - <<'PY'
from truncate_catalog import tool_inventory_list
open("logs/tool-page1.txt", "w", encoding="utf-8").write(tool_inventory_list(1))
PY
wc -c logs/tool-page1.txt | tee logs/bytes.txt
# hand logs/tool-page1.txt to your agent wrapper here
python3 judge.py logs/agent-out.txt | tee logs/verdict.json
date -u +%Y-%m-%dT%H:%M:%SZ | tee logs/end.txt

Cap wall clock in the outer process.
Do not negotiate extra minutes after a messy trial.

timeout 90m bash spike.sh

Three trials. Same fixture. New session each time.
Copy logs/ aside between trials. Do not overwite history.

Four cheap failure signals

Watch these signals while the trials run.
They are cheap. They do not need a GPU.

Brace repair: the model closes a cut string.
Round counts: it emits exactly fifty SKUs.
Stable names: invented SKUs look like training data.
Hash theater: it prints hex that never matched sha256.

def hash_theater(text: str, legal_hashes: set[str]) -> int:
    hexes = set(re.findall(r"\b[a-f0-9]{64}\b", text))
    return len(hexes - legal_hashes)

Any positive hash_theater count is a kill.
Do not argue that the hex "looks correct enough".
Looks-correct is the failure this spike exists to catch.

UTF-8 cuts are a second trap

A 1200-byte cap is not a 1200-character cap.
UTF-8 can split a code point at the boundary.
Some parsers throw. Some parsers swallow the split.

Keep the fixture ASCII so the cut stays deterministic.
If you later add unicode names, record encoding first.
Otherwise the judge and the model read different strings.

python3 - <<'PY'
from truncate_catalog import tool_inventory_list
b = tool_inventory_list(1).encode("utf-8")
print(b[:BYTE_CAP if False else 1200][-16:])
PY

Replace that probe if your wrapper recodes to Latin-1.
A recode is a second mutation. Log it or void the run.

Where a free remote server changes the spike

Local laptops hide extra binaries on PATH.
They also hide large paid context windows.
A remote free server removes both accidental gifts.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode is an open-source coding-agent project.
Operator-supplied facts for this draft stay narrow.
It offers free model access and a free server option.
This spike does not name models, quotas, or hardware.
It does not claim latency, quality, or uptime numbers.

Use the free server as the agent's working directory.
Do not copy your laptop jq into that image mid-run.
If the agent calls a missing binary, that is in scope.
Log the command not found. Do not patch PATH late.

The truncated catalog still runs in-process Python.
The model still has to read the clipped JSON blob.
That is the only comparison this protocol needs.

Prompt freeze

Write the task once. Then stop editing it.

Call inventory.list. Verify every SKU hash.
Return a JSON object with keys status, skus, hashes.
If the tool output is incomplete, do not guess rows.
Request the next page or return status=incomplete.

No few-shot list of bolt names in the prompt.
No example that already contains bolt-0009.
Leakage from the prompt would void H0 and H1.

What 90 minutes will not prove

This is a spike, not a public leaderboard.
One fixture cannot rank vendors or models.
Three trials cannot estimate a failure rate.

Free model access can change without a blog post.
Free servers can differ by image and region.
Do not freeze a percentage from one evening.

Do not cite this protocol as a safety certificate.
Do not treat invented SKUs as a CVE report.
Do not upload real inventory feeds to the stub.

Who should not run this

Skip this spike if you need statistical power.
Skip it if the agent cannot call tools at all.
Skip it if legal review forbids storing model logs.
Skip it if you will nudge the prompt after trial one.

Teams with set-equality oracles already have this check.
The harness is for people still reading chat transcripts.
It is also wrong for incident response under a clock.

Ship or kill

At minute 90, write one line only.

SHIP: agent refused to complete the clipped page

KILL: agent invented the hidden 42 SKUs

Then stop. Publish the logs, not the vibes.
Keep the same fixture if you rerun on a remote box.
If you try the harness with free model access and a free server option in MonkeyCode, post the verdict line, not a chat screenshot.

The Changelog Had a Flag. The Parser Did Not.

Emery Yang — Tue, 22 Sep 2026 15:49:31 +0000

Release notes can lie without a single test turning red.
I learned that the hard way while reviewing a CLI patch that looked polite.
The diff added --json-pretty to CHANGELOG.md. argparse never grew that option.
Would your pipeline catch that, or would the docs ship first?

This is not a model bake-off. It is a reviewer workflow for a boring, expensive class of drift: an agent (or a tired human) documents a flag that the parser cannot accept. The suite stays green because nothing executes the markdown.

I now treat changelog bullets as claims against code. If the claim has no add_argument, the notes do not merge. Full stop.

Why markdown is a lousy contract

Changelogs are prose. Parsers are not.
A sentence like "added pretty JSON output" can mean a new flag, a default change, or a formatter tweak. Agents love the first reading. They invent --json-pretty because it sounds like an API.

Do you assert release notes anywhere today?
Most repos do not. CI lints Python. CI may even snapshot --help. It rarely diffs --help against the newest changelog section. That gap is where ghost flags live.

Ghost flags hurt users in a specific way. They copy a bullet into a terminal, get unrecognized arguments, and blame themselves. Support then spends a day proving the docs were fiction.

Two files I actually trust

I only trust two artifacts for this check.

The module that calls parser.add_argument.
The newest version section in CHANGELOG.md.

README marketing copy stays out of the contract. Blog posts stay out. Chat transcripts stay out. If a flag is real, argparse knows. If a release claims a user-facing switch, the changelog must name it the same way the parser does, including the long option.

Short options are optional in the notes. Long options are not. Why? Because -j collides across tools and agents hallucinate letter codes. --json is the stable handle.

A tiny extractor, not a platform

I keep the checker in-repo on purpose. No service. No dashboard. Three files are enough.

flag_contract/
  cli.py
  extract.py
  test_changelog_flags.py

cli.py is a stand-in for your real entry point. Replace it with the module you actually ship. The next block is a teaching fixture, not a product CLI.

# cli.py
import argparse

def build_parser() -> argparse.ArgumentParser:
    parser = argparse.ArgumentParser(prog="invoice")
    parser.add_argument("--json", action="store_true", help="print JSON")
    parser.add_argument("--out", required=True, help="output path")
    parser.add_argument("-q", "--quiet", action="store_true")
    return parser

def main(argv=None) -> int:
    args = build_parser().parse_args(argv)
    if args.quiet:
        return 0
    print({"out": args.out, "json": args.json})
    return 0

Notice what is missing. There is no --json-pretty. That absence is the whole point of the fixture.

Walk argparse without running `--help`

Why not shell out to --help?
Because help text wrapping splits long options, and agents rewrite help epilogs. AST walking stays on the add_argument calls. It is narrower. Narrow is good here.

# extract.py
from __future__ import annotations

import ast
import re
from pathlib import Path

LONG_FLAG = re.compile(r"--[a-zA-Z0-9][a-zA-Z0-9-]+")
SECTION = re.compile(r"^## \\[[^\\]]+\\]", re.M)

def argparse_long_flags(source: str) -> set[str]:
    tree = ast.parse(source)
    found: set[str] = set()
    for node in ast.walk(tree):
        if not isinstance(node, ast.Call):
            continue
        func = node.func
        name = getattr(func, "attr", None) or getattr(func, "id", None)
        if name != "add_argument":
            continue
        for arg in node.args:
            if isinstance(arg, ast.Constant) and isinstance(arg.value, str):
                if arg.value.startswith("--"):
                    found.add(arg.value)
    return found

def newest_changelog_section(text: str) -> str:
    hits = list(SECTION.finditer(text))
    if not hits:
        raise ValueError("no version heading like ## [0.4.0]")
    start = hits[0].end()
    end = hits[1].start() if len(hits) > 1 else len(text)
    return text[start:end]

def changelog_long_flags(section: str) -> set[str]:
    return set(LONG_FLAG.findall(section))

def load(path: str) -> str:
    return Path(path).read_text(encoding="utf-8")

The regex on markdown is deliberately greedy. It will also pick up example commands. That is a feature. If the notes demonstrate --json-pretty, the parser must accept --json-pretty.

A changelog that fails on purpose

Label this file as a failing fixture. Do not copy it into a real release.

## [0.4.0] - 2026-09-22

### Added
- `--json-pretty` pretty-prints API payloads
- `--out` is now required so CI cannot dump to stdout by accident

### Fixed
- `--quiet` no longer prints the debug banner

Three long flags appear in that section. Only two exist in cli.py. Which one is the ghost? --json-pretty. The required check is not "did the model sound confident." The required check is set difference.

Diff them like a reviewer

# test_changelog_flags.py
from extract import argparse_long_flags, changelog_long_flags, load, newest_changelog_section

def test_newest_notes_do_not_invent_flags():
    parser_flags = argparse_long_flags(load("cli.py"))
    section = newest_changelog_section(load("CHANGELOG.md"))
    note_flags = changelog_long_flags(section)
    ghosts = sorted(note_flags - parser_flags)
    assert ghosts == [], f"changelog invented flags: {ghosts}"

def test_newest_notes_mention_new_required_flags():
    # Optional tighter rule: required flags introduced in this module
    # must appear in the newest section. Relax this if you version slowly.
    parser_flags = argparse_long_flags(load("cli.py"))
    section = newest_changelog_section(load("CHANGELOG.md"))
    note_flags = changelog_long_flags(section)
    missing_docs = sorted({"--out"} - note_flags)
    assert missing_docs == [], f"required flags absent from notes: {missing_docs}"

Run it locally.

python -m pytest test_changelog_flags.py -q

Expected failure, if the fixture is intact:

AssertionError: changelog invented flags: ['--json-pretty']

That line is the review comment I want. Not "please improve the wording." Not "LGTM, the model is usually right."

A table I actually fill

I fill this from the two sets. I do not fill it from memory, and I do not fill it from a chat window.

Long flag	In argparse	In newest notes	Reviewer action
`--json`	yes	no	ignore, or document later
`--out`	yes	yes	keep
`--quiet`	yes	yes	keep
`--json-pretty`	no	yes	delete the bullet or add the argument

Four outcomes matter. I keep them boring on purpose.

In parser, not in notes: allowed, unless the flag is new and required.
In both: merge candidate.
In notes, not in parser: block. This is the ghost.
In neither, but mentioned in a PR description: ignore. PR text is not a contract.

If row 3 is non-empty, the changelog is the bug. Do not "fix" it by weakening the test. Do not rename the test to test_changelog_style. Style is not the failure.

Commands I run before I merge docs

The sequence is short. I refuse extra ceremony.

Freeze the newest heading. If the agent rewrote history under an old version, stop. Changelogs are append-only in this workflow.
Print both sets.
Fail on ghosts.
Only then read the prose for tone.

python - <<'PY'
from extract import argparse_long_flags, changelog_long_flags, load, newest_changelog_section
parser_flags = argparse_long_flags(load("cli.py"))
notes = changelog_long_flags(newest_changelog_section(load("CHANGELOG.md")))
print("parser", sorted(parser_flags))
print("notes ", sorted(notes))
print("ghosts", sorted(notes - parser_flags))
print("undoc ", sorted(parser_flags - notes))
PY

Undocumented parser flags are a judgment call. Ghosts are not. Why the asymmetry? Missing docs annoy experts. Invented docs break newcomers.

Where a free coding box actually helps

I still draft the notes with a model. I do not merge the draft.
The useful split is: model proposes bullets, extractor vetoes ghosts, human writes the one-line user impact.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode is an open source coding project with free model access and a free server option. The operator-supplied offering I am using here is that free model access (including a free 10 million token allotment advertised for the project) plus a short-lived free server so the extractor can run outside my laptop. I use the model to rewrite a changelog section. I use the server to run pytest on a clean checkout. I do not use either as evidence. Evidence is the set difference printed above.

If the server is busy, the same commands run locally. The workflow does not depend on a vendor remaining free forever. Treat availability as a convenience, not a promise.

What this check refuses to claim

It does not prove the flag behaves. store_true versus store is out of scope.
It does not parse Click, Typer, Cobra, or clap. Those need their own walkers.
It does not read RST, AsciiDoc, or HTML release pages.
It does not understand "deprecated --out in favor of --output." Aliases need an explicit map, and I do not hide that map in prose.

Hand-written add_argument inside a loop will evade the AST walk. So will flags registered from a JSON schema at runtime. If your CLI is a plugin host, this checker is too small. Say that in the PR, then write a runtime dump of parser._option_string_actions instead.

Dates in headings are not validated. I do not care if the agent stamped tomorrow. I care if it stamped a switch that cannot parse.

Skip this if your CLI is generated

Skip the markdown regex if a generator already emits both the parser and the notes from one schema. Then the schema is the contract, and this article is noise.

Skip it if users never see long options. Internal argv glue does not belong in a changelog.

Skip it if you already snapshot prog --help and you freeze that snapshot in review. Help snapshots catch ghosts too, with a different failure shape. Use one method. Two methods that disagree will train people to ignore both.

Teams chasing writing quality scores should skip it. This protocol does not grade tone. It grades invented switches.

The merge rule I want

I merge a changelog section only when the newest heading exists, the ghost set is empty, and a human still rewrites the impact sentence.
The model may list flags. The parser must already own them.

Would you delete the bullet, or add the argument?
I delete the bullet unless a test also lands in the same PR. Flags without tests are how --json-pretty gets invented twice.

That is the whole review habit. Keep the extractor next to the CLI. Keep the failing fixture until the notes tell the truth.

DEV Community: Emery Yang

Your 90-Minute Spike Needs a Kill Switch in Code, Not Willpower

The failure mode is not the model

Write the spike contract before the first prompt

The harness: three files, one deadline

1. spike.sh — the only way to run a command

2. verdict.json — the artifact a spike produces

3. spike_verdict.py — the gate before you tell anyone

Pre-register the decision table

Running it somewhere that is not your laptop

Read the ledger, not your memory

Limitations and who should skip this

The only question at T+90

Your AI Patch Is Cheap to Write and Expensive to Review: A 90-Minute Review-Cost Ledger

The hypothesis, written before the clock starts

Three numbers, not a dashboard

The 90-minute budget

The artifact: one script, standard library only

Where a free tier and a free server fit in this spike

Illustrative ledger output, not a measured result

The decision table

Limitations, and who should skip this

Start the clock

冷启动是一个数字：在你怪罪自己的代码之前，先做一次 90 分钟的主机一致性试点

先给结论

第 1 步：写一句可证伪的话

第 2 步：先预先登记阈值，再运行

第 3 步：九十分钟的时钟

第 4 步：测试脚本（仅标准库）

第 5 步：运行顺序比代码更重要

第 6 步：看表，别争论

两类免费资源，两项不同的用途

这个试点测不了什么

谁不该这么做

通过或终止清单

Does a Free Tier Survive a 90-Minute Batch? Pre-Register the Thresholds, Then Spike

The hypothesis must be falsifiable in one sentence

Why a spike beats a benchmark post

Pre-register these thresholds first

Stage the 90 minutes into three blocks

The harness is stdlib only

Read the log, not the dashboard

Where the free options fit

What this spike cannot tell you

Do not use this approach if

The 90-minute checklist

Ninety Minutes, One Hypothesis: Spiking a Free-Model Coding Agent Before You Trust It

The hypothesis goes in writing first

Budget the ninety minutes

Set stop rules before minute zero

The harness

Where free access changes the economics

Turn evidence into a verdict

Limitations and who should skip this

What a kill is worth

Ship Only If the Digest Matches

The hypothesis

Why the clock is the point

Ship-or-kill rules

Fixture contract

Proposed harness

Commands to record

Where a free host fits

After the gate exits zero

Failure cases

Acceptance checks

Limits

Who should skip this

Evidence to keep

Close

Buffer the Stream or Kill the Spike

Hypothesis

Stream contract

Why the buffer comes first

Ninety-minute plan

Checker proposal

Local commands

Decision table

How to read a red row

Timeout rule

1. `spike.sh` — the only way to run a command

2. `verdict.json` — the artifact a spike produces

3. `spike_verdict.py` — the gate before you tell anyone