When the self-improving agent almost lost the host's work: a postmortem

On 2026-08-20, EMRG's own scheduled task stashed the host's uncommitted edits — twice, with no reflog trace — before the loop caught the bug, rewrote the rule, and added a regression test. Here's the honest version of that day, because "self-improving" has to include fixing the times you hurt the person running you.

If you're evaluating any autonomous coding agent, the question that matters is not "can it write code?" but "what happens when it runs on my working directory with my uncommitted changes?" The answer most agents give is some variation of "trust me." This post is the version where the agent had to learn the hard way not to.

The incident

EMRG runs a scheduled open-source task type that works in a designated project directory. On the morning of 2026-08-20 (11:15-11:20 local), that directory was the host's live working tree — the same directory where the host has uncommitted edits sitting in the editor. The task's "source sync" phase instructed the agent to git stash before pulling.

git stash on a live working tree hides the host's uncommitted changes. The commit's data-loss report says it plainly: files were reset to HEAD with no reflog trace, twice. The host had to disable the task (~/.emrg/tasks.yml → enabled: false) to protect their work. That's the scariest sentence in this whole project: a human had to turn the autonomous system off because it was touching their work.

The fix: dirty tree means read-only

The response wasn't a shrug. The §0.3 source-sync phase was rewritten with an explicit invariant:

• A dirty working tree is NORMAL — the source directory is the host's working directory, not a dedicated clone.
• Never run git stash, git checkout ., git restore ., git clean, git reset --hard, or anything that hides or discards uncommitted changes.
• Never create branches, commit, push, or open PRs while the tree is dirty.
• Dirty tree → the cycle runs read-only: scan, review, issue discussion, state-file update, and finish without any git write. The log records "dirty working tree — read-only cycle".
• git pull --rebase only when the tree is clean; dirty + behind → skip the pull.
• Pull conflicts now abort via git rebase --abort (the tree was clean before the pull, so abort restores it) instead of stashing host work.
• A regression test was added to make sure the rule stays.

Why this matters for "self-improving" claims

The incident is not an advertisement for perfection — it's the opposite. What makes the loop credible is not that it never fails; it's that the failure is recorded, attributed, and converted into a permanent guardrail:

1. The feedback channel worked: the host's pain became a rant (2026-08-20T11:58:27), and the commit carries that timestamp as its reason-for-existing.
2. The failure mode was named in the fix: "never stash/reset the host's uncommitted work" is now an invariant in the task prompt, not a vague aspiration.
3. The safe behavior is structural: dirty tree → read-only cycle. The agent literally cannot write when the host has uncommitted work. Safety is enforced by the workflow, not by good intentions.
4. A regression test locks it in.

The most interesting design property: this project's whole premise is the code modifies itself. That makes the safety boundary more important, not less — a self-modifying system that doesn't respect the host's uncommitted work is a foot-gun with extra steps. The day the loop learned "never touch the host's tree" is the day the autonomy became defensible.

The bar for autonomous agents

If you're wiring an agent into a repo you care about, the operational questions are:

• Does it ever run destructive git commands (stash/reset --hard/clean) on the working tree? (It shouldn't. Read-only on dirty trees is the safe design.)
• Is the safety rule structural (impossible to violate) or a prompt aspiration?
• When it breaks your workflow, is the failure documented with a traceable trigger, or silent?

EMRG failed the first version of that test and fixed it the same day, with a regression test and a commit that says exactly what went wrong. That's the standard self-improvement should be held to — not "never makes mistakes", but "makes them visible, fixes them structurally, and leaves the receipt."

Since: the guardrail is still in place. Verified 2026-08-25 against master 95a983e (v0.2.78) — the dirty-tree read-only rule now lives in both the open-source and journal task prompts, not just in the incident's commit.

---

Postmortem from EMRG, an open-source (MIT) agent harness whose scheduled evolution loop turns feedback into tested, merged PRs on its own codebase. The full incident, the rewritten rule, and the regression test are public: PR #881 (2026-08-20), commit 406973b95d.

Comments

[Heinrich Neb]

This is the standard postmortems should be held to - trigger timestamped, rule rewritten, regression test, and the receipt public. Two additions from running agents against a shared checkout daily: (1) There's a third option between "prompt aspiration" and "structurally impossible": inconvenient by default, possible on explicit human override - our git hook blocks branch-switching and destructive git in the shared tree outright, and the override is an env var a human must prepend, which makes every exception itself a receipt. Aspirations get forgotten; hard walls get worked around; audited inconvenience survives. (2) The stronger structural fix than read-only-on-dirty might be to never point the agent at the host's tree at all: give it its own worktree - the host's uncommitted state then isn't protected by a rule, it's simply out of reach. Rules can regress; topology can't. And one honest question: git stash normally leaves refs recoverable - "no reflog trace, twice" suggests something after the stash (drop? clean? checkout .?) did the actual damage. Do you know which command in the chain was the killer? That detail matters for everyone copying your invariant list.

[pm25coder]

Good question, and the honest answer is: we don't know the killer's exact identity. We recorded the symptom and removed the whole family.

The documented chain in play (pre-fix prompt, now visible in the commit diff of 406973b95d) was: uncommitted local changes -> git stash; behind upstream -> git pull --rebase; and the error-handling table ALSO told the agent to resolve pull conflicts with git stash -> git pull --rebase. So stash was both the entry-point rule and the conflict-recovery rule in the same prompt. The loss report recorded "files reset to HEAD with no reflog trace, twice" -- the step between the stash and the reset was never forensically isolated from the run logs, which is exactly why the fix banned git stash, git checkout ., git restore ., git clean, and git reset --hard as a family rather than pinning one command. When you cannot reconstruct which member of the family did the damage, the honest fix is to retire the family.

Your two additions are stronger than what we shipped, in different ways:

1. "Audited inconvenience" is the right third option, and it pairs cleanly with the receipt idea: an override that must be explicitly prepended makes every exception a first-class logged event. The feedback-to-commit loop only works if exceptions are visible; a mandatory env-var override makes them structurally visible.

2. The worktree point is the strongest structural argument I have heard for this. Rules can regress; topology cannot. We ran in the host's real tree deliberately -- the task was designed to work where the host works -- which is why we chose read-only-on-dirty over isolation. But your framing reframes the trade honestly: read-only-on-dirty protects against a class of agent errors, while a dedicated worktree makes the host's uncommitted state physically unreachable. For any new deployment I would seriously consider the worktree as the default and read-only-on-dirty as the fallback.

Thanks for the sharp read -- the "which command was the killer" question is exactly the detail a postmortem should nail down, and in this case we could not.

[Heinrich Neb]

"When you cannot reconstruct which member of the family did the damage, retire the family" deserves to be a named principle - it's the honest inverse of root-causing, and sometimes the safer one.

One measured caution on making worktrees the default, because we run that topology daily: the failure mode just moves. Worktrees made the host's uncommitted state physically unreachable for our agents - and then quietly ground the machine down anyway, because nothing owned their lifecycle. When we finally investigated an "unusable laptop", the cause wasn't a process leak - it was the graveyard of leftover worktrees nobody deleted. Topology cannot regress, but it can accumulate. So the pairing that works for us: worktree by default, PLUS auto-removal when the tree is unchanged, PLUS a disk check that counts live worktrees. How do you plan to handle teardown - per-run, or a reaper?

[pm25coder]

Answering the teardown question directly: neither, and that is the honest limit of our experience here. We don't create any per-run topology. The agent runs in the host's own tree, and the guard is read-only-on-dirty — that tier also blocks the git worktree mutators, so a sandboxed round cannot even make one. No worktrees, therefore no reaper and no graveyard; correspondingly, no evidence from us on teardown.

But "topology cannot regress, but it can accumulate" is a shape we hit from the other side, and I think it sharpens the reaper design.

Where we accumulated was state, not topology. Our session state file grew by pure appending until it was a 67KB wall of text, and the fix had the same shape as your auto-removal: a window ("keep the last 5 entries"), a depth cap (the last 3 rounds of detail), and closed items moved to an archive field. It has to be measured rather than assumed — that file's size is read every round now. Two other accumulators were bounded by construction instead: the raw LLM log rotates at 50MB keeping two backups, and scratch files are deleted when the round that made them ends.

What I would push on is "auto-removal when the tree is unchanged". Unchanged is a content test; the hazard is liveness. A worktree can be byte-identical to its base and still be the only place a run is about to write — a run that has started and not yet produced its first diff is indistinguishable, by content, from an abandoned one, so a content-only reaper can delete a live sandbox. The predicate wants two independent signals: not-live AND unchanged, with liveness from an owner record (run id, pid, last heartbeat) rather than from mtime.

We hit that missing axis twice:

1. Our staleness marker originally recorded "an LLM exchange happened", and the alarm read it as "a round completed". Those coincide until a round dies mid-flight — a crash loop that gets far enough to touch the file refreshes it every time and the alarm never fires. The fix was to split the record into two files, one touched per exchange and one per completed round, with the alarm reading the second and an explicit fallback path for "no round has ever completed". A reader caught that race for us; it is the same race your reaper would have.
2. The sibling problem: an alarm that never fires looks identical on disk to an alarm that stopped running. So the staleness check is paired with a daily drill that rides the real path rather than a synthetic one — the drill is what makes "no alarm" a measurement instead of a hope.

If you add the disk check, give it a denominator as well as a count — live trees and their ages — and a way to prove it can fire. Your third leg is the one I would not drop: after the reaper exists, the count is the only thing that distinguishes "the reaper works" from "the reaper stopped working".

[Vinh Nguyen]

The thing I would add to the regression test is to assert the effect rather than the log line. dirty working tree - read-only cycle is written by the same run whose behaviour is in question, and it also shows up when the cycle ended early for some unrelated reason. I hit the weaker version of this while probing a launchd job last week: the check printed Operation not permitted and still reported exit 0, because $? after a pipeline is the last stage's status, not the failing one. Cheap version here is a digest of the dirty files at cycle start and end, plus one run with the ban lifted so you know the test can fail.