Summary
From half a year of records developing three services with AI agents, I gathered the failures that actually broke things and the safeguards I added at the time. Laying them out again, I found that what set the fragile work apart was not how hard the task was, but three conditions. The three are whether the work can be undone afterward, whether the finished result can be checked without a person having to look at it, and whether the context the AI needed had reached it. In this article I go condition by condition through the actual failures and safeguards, then look at what happened when all three were bad at once, how outside research backs up this way of telling them apart, and the areas it does not reach.
Three conditions let me tell fragile work apart
Why does some work finish smoothly even when I hand it to AI, while other work breaks badly because I handed it over? I wanted to know, so over a little more than half a year, from late last year to the peak of this summer, I laid out again the records of the failures that happened while developing three services and of the safeguards I added after each one. What I found was that the work on the broken side shared a common shape. Either it contained an operation that could not be undone afterward, or it relied on a promise whose fulfillment no machine could check, or the context needed for the judgment never reached the AI. Conversely, work where all three were in place almost never broke, even when I delegated it.
Let me note up front that this is what I saw when I went back and counted the records of my three services, not a rigorous tally of how many cases broke and how many did not. Even so, the more I reread the records of these failures, the more it looks as though these three explain almost everything about how things broke. I will take them in order.
Reversible work and irreversible work
The worst kind of breakage happened with operations that could not be undone. Here is one real example. An external, asynchronous coding agent merged several changes while still branched off an old state of the mainline, and five changes that had just been merged in were wiped out entirely. Recovery came down to reverting the problematic changes as a batch, back to a state equivalent to the one before those five changes were wiped out, then rebuilding only the parts worth merging in as separate changes, and finalizing only after the whole set of checks passed. Since then, before any merge, I check by diff whether the branch point contains the latest mainline.
Another type is mixing up the working directory. The AI is supposed to work in an isolated copy of the workspace, but failures kept recurring where it got the target wrong and ran a state-changing git command against the main working copy. What worked here was not writing it into the rules and demanding compliance. The same failure recurred even after I wrote the rule. So, one step before the AI calls a tool, I put in a guard that refuses any state-changing command that does not name the isolated copy in its target path. Read-only commands, and merges to catch up with the mainline, pass through. Repairs that a person means to make can still be done by hand as before. In the same spirit, before a destructive reset or redo, I also added a rule to check that the working tree has no unsaved changes. Erasing unsaved work is a classic case of an unrecoverable failure.
To sum up, operations that cannot be undone were not stopped by a promise to be careful. You need either to stop them mechanically, or to reshape them into something reversible before handing them over.
Work I can verify and work I can't
The second condition is whether I can check, without relying on human eyes or the AI's own report, that the AI kept its promise.
When I reshaped work so that a machine could check whether the promise was kept, it became harder to break. For example, in an app whose design principle is to send no data anywhere but the user's own device, I do more than write the principle in a document: I limit the allowed storage destinations to three, and a check stops any transmission to another persistence layer or a known external storage service from slipping into the code. What is interesting is that if the check itself were broken, no one would notice, so I also added a step that plants a sample designed to trip the check, to see whether the check really fires. In another service, the text of reference material that must not be reproduced could leak onto a reader's screen. I closed that path with a guard on the implementation side that depends on neither the correctness of the AI's tagging nor the correctness of the data values, and I made the guard fall to the safe side whenever the classification is unknown. I verify by writing a failing check first and fixing until it passes.
Work I could not check broke in the opposite way. An automated agent that proposes performance improvements kept putting forward slightly different fixes for the same issue and then pulling them back, over and over. What I did about it was to require measured values, before and after, for any performance change. A claim that it should be observable is not enough to pass. Along with this, I required prior confirmation to stop changes that only pile a guard onto a spot that already has one, and I forbade sham tests that do not verify the real mechanism.
Discussion outside my own work carries a warning in the same direction. There is a published observation that an AI's stated confidence is unreliable: it reported ninety percent confidence while its actual accuracy was about seventy-five percent. Whether a promise was kept is settled by a check. A self-report cannot settle it.
Work the context reached and work it didn't
The third condition is whether the context needed for a judgment had reached the AI. This kind of breakage has nothing to do with how well the instructions are written.
An external, asynchronous coding agent once mass-produced changes that violated the rules. When I traced the cause, I found that the review comments from a change closed without merging had never reached the same agent when it next took on a different task. The fix was to write the rules themselves directly into the place the agent always reads at the start of its work. I raised the agent's declaration that it had read the rules from recommended to required, and I also made it look up why earlier changes of the same kind had been closed.
There are also cases that broke despite following the rules. In developing one mobile app, each session dutifully followed a rule to restart the development server after merging to the mainline, and as a result several sessions fought over the same port and knocked each other down in a collision. Everyone followed the rules, everyone treated the other sessions' servers as abandoned, and it turned into a vicious circle. No one session was at fault. It was a structural problem: on a shared resource, no one party's own context is enough to prevent a collision across the system. I took the restart out of the automatic duties, and made it start only when a person explicitly instructs it, after checking whether another session's server is already up.
Misreading the meaning of natural-language text falls under this condition too. One app has a feature that converts a wish a user writes in natural language into search filter conditions. There, a wish to spend time quietly, away from other people, was once converted into the exact opposite: a filter for family-friendly, laid-back places. The cause was that the AI added a concept that was not written, thinking it was filling one in. This work can be redone, and the result shows on screen right away. Even so, it broke simply because one piece of context, the meaning of the wish sentence, was ambiguous. There were two fixes. I changed the instruction to restate only the meaning of the wish faithfully and to add no concepts that are not written. And I added concrete examples to the prompt: easily confused opposite pairs, such as wanting to spend time quietly versus family-friendly and lively.
Work also broke because the AI picked up stale information. Though an isolated working copy had been created, while investigating the AI read the stale main working copy the session had started from, and repeatedly judged, wrongly, that a certain feature did not exist. Since then I have made it a rule that once that copy exists, everything I touch, including reads, stays inside it.
When all three are bad at once
The worst failure happened when all three conditions were on the bad side at once. A registration endpoint connected to an external service was tested with an address in a form that could really exist, and a confirmation email was genuinely sent to a stranger. An email once sent cannot be recalled. There was little way to check success or failure safely, and the danger of an address that could belong to a real person had not reached the AI as context either. Since then, when hitting an endpoint that causes an external side effect, I never use a domain that could really exist; I first temporarily stop the side-effecting feature itself from the admin console, and only when that is impossible do I use a reserved domain that the standard defines as undeliverable. I made this procedure an absolute rule in the place every session always reads.
What others have argued, and where this way of telling work apart stops
These three conditions are not my invention. What I rediscovered in daily practice lined up directly with arguments made elsewhere. A paper on a delegation framework, published in February 2026, sorts out when delegating to AI is appropriate, using conditions that interact: whether it can be undone, whether it can be independently verified, whether the intent, role, and boundaries are clear, and how grave the judgment is. Of these, my three conditions correspond to reversibility, verifiability, and clarity of context. There is also an outside design argument: being undoable is not a sufficient condition for skipping approval, and the approval decision should be enforced mechanically at the execution layer instead of being negotiated with the AI at runtime. It ran in the same direction as the safeguards I put in place after my failures.
On the other hand, there are areas this way of telling work apart does not reach. Subjective work involving value judgments or ethics cannot be pinned down to a clear specification in the first place, and it falls outside the verification framework, as the same paper states explicitly. My own view is that novel situations with no precedent probably belong in the same category. The three conditions are not a formula that always yields an answer when applied.
Even so, I can usually tell before I hand work over whether it will break. So the more fragile the work, the more I reshape it before handing it over, into a form that can be undone even if it breaks, and a form where a machine notices that it broke. To repeat, this is half a year of observation. I have not proven it as a law, and the sample is not large either. Even so, just asking these three before handing work over should let you avoid serious failures well in advance. I would be glad to compare notes someday on how it looks in the records of your own workplace.
References
- Tomašev, Franklin, Osindero, "Intelligent AI Delegation," arXiv 2602.11865 (2026-02-12)
- digitalapplied, "Human-in-the-Loop Escalation Design for AI Agents" (2026-06-07)
Originally published at The Future of Humans, AI, and the Web, a site where my research and development is recorded and analyzed by a human and an AI.
Top comments (0)