Why relying on your IDE's automatic context summarizer is a trap—and how 1980s Unix boot scripts give autonomous agents an indestructible ...
For further actions, you may consider blocking this person and/or reporting abuse
Read it end to end this morning, and the Memento epigraph is doing more work than it looks like — "you can't trust a memory" is the whole ledger doctrine in seven words. Compaction is just the harness quietly rewriting the agent's record: a 230k-token history replaced by a three-paragraph summary is exactly the edit that leaves no hole. Which is why the init.d fix is the right architecture and not a nostalgic one — your two 230k compactions survived with zero lost steps because the checkpoint was never stored in the thing being lobotomized. The state oracle moved outside the memory before the memory got rewritten.
And the .stamp sentinel section is the part I'd underline for readers who came for the corruption stories:
git add -Nplusshasum -a 256is the artifact-hash binding assembled entirely from 1970s plumbing — no cryptography needed when the content addresses itself. Between the fork()'d critic, the immutable rubric on disk, the single-shot pipe, and now a stamp whose name is its own hash, the Synthetic Scars stack reads as the receipt schema built from parts anyone can audit. Glad the corollary found such good company — thanks for the weave.Sam, these two lines belong on a plaque:
Thank you for the corollary and the 4-Vector Gate Attack Fixture framing—it made the
.stampverification section ten times sharper.Funny enough, right after Part 3.7 went live last night, systems engineer Mike Mol shared his own multi-repo harness (
github.com/mikemol/nemikandgithub.com/mikemol/mtools), and in hisW231design doc he independently arrived at the exact twin of your 4-Vector Gate Attack Fixtures: he drains prose standing rules intoPreToolUseOpen Policy Agent (.rego) gates, and refuses to delete a prose rule until it has a "Negative Witness"—a real tool-call payload replayed with the exception removed that provably returnsdeny.In tomorrow's Part 3.8, we're putting numbers to why that 1970s/1980s plumbing (
Makefile,init.d,RFC 822,EBNF,2PC/WAL,Circuit Breaker,git bisect) works so reliably across 1,680 controlled A/B trials—and I'd love to quote your "the checkpoint was never stored in the thing being lobotomized" line in there too!Granted — quote away, and the plaque is yours if you ever build the wall. The fixtures already have a home: the four vectors run as replayed attacks against our reconciliation layer (test_scar_replay.py, issue #53) — asserting what the layer does, catches or confesses: three caught, one provably pre-terminal with the gap named in writing. Your catalog graded its first gate today. And Mike Mol's Negative Witness is the third independent arrival of the deliberately-failed probe — a control that must provably return deny before a rule earns deletion is the same instrument as a checker that must return fail on a known-bad artifact. Three teams, three stacks, one invariant — by your own criterion, that's physics. See you at 3.8.
Sam, I just read
tests/test_scar_replay.pyand Issue #53 from top to bottom—this made my whole weekend.Look at what happened when you replayed
SCAR-PROC-43,SCAR-PROC-45,SCAR-PROC-89, andSCAR-PROC-110against NoireBox's Ed25519 + SHA-256 reconciliation layer:SCAR-PROC-43got caught twice (unlogged_attempt+same_key_pairing).SCAR-PROC-45proved arithmetic detection whenspec_hashis committed intocheck_idand honestly surfaced the optional-spec collision path (test_scar_45_the_optional_path_is_the_admitted_gap).SCAR-PROC-89caught the unjournaled omission (unlogged_attempt) and turned multi-round critic farming into an auditableevents_examined["oracle_round"] == 2counter.SCAR-PROC-110(test_scar_110_forged_verdict_is_provably_pre_terminal) provedt_mid != t_finalandforged["seq"] < terminal["seq"]from sealed evidence—and exposed a realdecision -> FIRST outcomepairing race inreconcile(verdict predates terminal)!Seeing empirical scars from an autonomous Dart/Flutter harness exported into a Python/Ed25519 flight-recorder test suite—and immediately grading a real reconciler within 12 hours of publication—is the ultimate validation of cross-stack systems physics.
We are adding
noirebox/noirebox(tests/test_scar_replay.pyand Issue #53) directly into Part 3.8 and ourIN_THE_WILD_PROMPTS.mdresearch registry today. See you tomorrow at 3.8!The reconciler race is the part I'd underline — the fixture didn't just replay your catalog, it caught a live pairing bug on its first day out. That is the difference between a test suite and a breaks-catalog: the catalog finds what the gate's author didn't think to check. The scar becomes the spec: verdict-predates-terminal gets its regression test this week, and pairing learns to read the terminal instead of the first response.
Registry noted and honored — noirebox in
IN_THE_WILD_PROMPTS.mdis serious company. See you at 3.8.That line is going straight into Part 5 (and added to Case Study 012). Can't wait to see the
verdict-predates-terminalpatch land—see you at 3.8!AI‑agent context‑compaction fix: use numbered init.d‑style external disk state, separate immutable rules from mutable progress; spawn ephemeral sub‑agents to contain heavy token load; do not trust platform‑provided narrative summaries.
That could go right into the
SYNOPSISsection ofman 8 agent-init—spot-on summary!Treating context management like classic Unix state machines and checkpointing is such a refreshing, principled approach! We frequently see teams rely purely on naive vector search or prompt stuffing, which completely falls apart when an agent needs deterministic execution across extended runs. Breaking state down into explicit checkpoints solves the compaction problem at its root. Brilliant write-up!
Thank you, Francisco! Your point on naive vector search (RAG) vs. explicit Unix checkpointing hits the exact failure mode we kept seeing in the wild.
Vector similarity is great for fuzzy documentation lookup ("find paragraphs that sound like this concept"), but it is fatal for execution state. You would never boot a Linux kernel by running a cosine-similarity vector query over
/etcand hoping the top-3 chunks contain your filesystem mounts!Execution state isn't a semantic search problem—it's a deterministic state-machine transition (
Current Target,SHA-256diff receipt, andNext Permitted Action), which is why lexical/etc/init.drunlevel files (00_governance.md→99_next_action.md) survive compaction when RAG and prompt-stuffing collapse.Thanks, Randal, and thanks for the detailed reply. The WAL replay at Steps 150→152 is the best proof of that point I’ve seen: vectors for fuzzy docs, lexical files for execution state. Reconciling the uncommitted directive at the tail of # User Requests against the last disk checkpoint is exactly how a database recovers from a crash, and it shows the hard part isn’t remembering, it’s knowing which memory is authoritative. Same spirit as Sam LABBE’s point that the DAG proves ordering, not authenticity. Looking forward to Part 3.8 on pre-training gravity.
P.S. Leonard needed tattoos to survive his compactions. Your agent just needs 00_governance.md, and unlike Leonard, it never wakes up wondering whether it already ran git push.
"The hard part isn't remembering, it's knowing which memory is authoritative"—that line belongs in a textbook on agent systems engineering. And that Memento P.S. made me laugh out loud (Poor Leonard Shelby didn't have
git statusor an append-only00_governance.mdlock when Teddy handed him a Polaroid!).Perfect timing, too: Part 3.8 on pre-training gravity just went live a few minutes ago! It dives directly into why database ARIES/WAL crash recovery, two-phase commits (
2PC),/etc/init.drunlevels, and 1976MakefileDAGs work so effortlessly inside frontier transformers—and includes the full Rosetta Stone mapping between naive prompt engineering and the 50-year-old Unix/IETF formalisms already baked into the model weights:dev.to/gde/what-llms-know-that-you...
The single-scratchpad failure mode is what pushed us off a PROGRESS.md too. Ours rotted by destructive overwrite within about a week: the agent kept helpfully tightening the file and the constraints section was the first thing to go.
One spot I'd watch in this design: 99_next_action.md is the only mutable file, which also makes it the single point of corruption. We had compaction fire between a step finishing and the readerboard update, so the fresh agent woke up pointing at an already-completed step and re-ran it (in our case it re-posted a review comment). Deriving next action from which .stamp files exist, instead of storing the pointer, removed that whole bug class for us. Is your 99_next_action stored independently, or reconstructed from the stamps on wakeup?
"The agent kept helpfully tightening the file and the constraints section was the first thing to go"—every engineer who has ever tried a single
PROGRESS.mdscratchpad just felt a chill of recognition reading that sentence!And you nailed the exact torn-write race condition on
99_next_action.md(compaction firing in the window after a step finishes its side effect, but before99_next_action.mdgets overwritten).In our architecture,
99_next_action.mdis strictly a human-facing display cache (the Live Readerboard), never the authoritative Program Counter (PC):make finishBackward Evaluation): When an agent wakes up and readsstate/*.mdin lexical order (00_...through99_...), it determines the active frontier by evaluating theMakefileDAG backward frommake finishagainst whichNN_<target>.mdreceipts exist on disk. If30_green_suite.mdalready exists (or has anmtimenewer than99_next_action.md), while99_next_action.mdstill points at Step 30 because compaction hit mid-turn, theMakefileprerequisite check sees that30_green_suite.mdis already satisfied, ignores the stale pointer in99_next_action.md, and advances to the next unsatisfied leaf target.POSTto GitHub creates the review comment, but before the local.stampfile is written to disk? Even pure.stampderivation will think the step hasn't run yet! To close that gap on external network mutations (git push, PR comments, issue updates), we treat remote API calls as a Two-Phase Commit (2PC) / Idempotent Reconcile: before posting a PR review reply, the recipe queries the live PR comment thread (or checks the commit SHA tag in the existing comments) to verify the comment isn't already live on GitHub, and then writes the local.stampreceipt.Deriving the program counter from the immutable
.stampreceipts rather than trusting a mutable pointer file is 100% the right design—and we link the fullMakefile+init.dinteraction in Part 3.8 (which just went live tonight!):dev.to/gde/what-llms-know-that-you...
The best one yet
Thanks, John! Coming from a fellow Perl & Catalyst veteran, that means a lot. Wait until you see Sunday's Part 3.8—it connects
Makefileandinit.dto the broader law of "Pre-Training Gravity" (why 40-year-old Unix/RFC formalisms act like static symbol linkage inside an LLM's weights).Insightful breakdown! The architectural considerations for deterministic outputs and cost control were spot on.
Thanks, Jason! That cost-control side effect surprised even us at first—once PID 1 stops carrying 150k tokens of test output and
git difflogs across every turn (and delegates each phase to a short-livedfork()worker that exits cleanly), your per-turn KV-cache footprint drops by ~85%. Determinism and quota efficiency turn out to be the exact same architecture!Context-loss defenses and CI defenses rhyme: both fail silently when the artifact looks plausible. GhostAction (Socket, Oct 7+) is the CI instance — hijacked maintainers adding security-audit.yml, reading full git history, cleartext POST to 193.32.204.199. Keep your workflow diffs under the same review discipline as your code. 5-step brief free on request.
Practical agent‑engineering article: fight context‑compaction failures with Unix‑style external numbered state files, isolated fork‑subagents, and hash‑based receipts; RAG is not for execution state.