The reply looked finished: which function to change, why the current path fails, what the fallback should do, and a short sketch of the new behavior. The reviewer read it, agreed, and typed the change into the editor by hand from the thread. No branch had been pushed. No pull request existed. The agent's session closed and the working copy it had described was gone.
Nothing in that exchange was false. The reasoning was sound. The problem is that intent written in prose and code that actually landed in the repository are two different artifacts, and only the second one gets reviewed by CI, tested, and merged.
A written explanation is not a shipped patch
A paragraph can carry the central idea and still drop the parts that make a change correct: the import that moved, the error path the sketch never mentioned, the test that belongs with the fix, the existing caller that depends on the old shape. When a human copies from chat, they reconstruct those pieces from memory, and the reconstruction diverges from what the agent actually reasoned through.
There is also no test run against the real artifact. The agent described a change it may never have written into a file, so no suite executed it, no build checked its types, and no CI run produced evidence. The reviewer approving prose treats confidence the prose reads well as evidence the change works.
Chat is ephemeral. When the session ends, the branch, the partial edits, and the exact diff vanish together; only the summary survives, and the next person who needs the fix starts from a paragraph rather than reproducible code.
Make a committed artifact the exit condition
Separate exploration from implementation before the agent starts. When the ask is advice or an approach, a written answer with no code is appropriate, and it should be labeled as such so nobody treats the sketch as a patch to apply. When the ask is to make the change, done means a pushed branch and an open PR with the real diff attached.
Require the agent to leave behind what a reviewer can check: the branch, the commits, the tests it ran, and the PR body describing risk rather than restating the diff. The answer to "what changed" should point at a reviewable diff, not a long message in the thread. If only prose exists at the moment of done, the task is still open, no matter how clear the explanation reads.
Hand-applying fixes from chat feels fast once, then quietly becomes a place where untested, unreviewed code enters the tree. Where does your implementation work switch from "explain it" to "land it as a diff"?
Top comments (0)