An agent that fails loudly is annoying.
An agent that says "done" while nothing ran is expensive.
I've been building with AI agents near real systems, and the failure mode I worry about most isn't a crash — it's silent success.
The agent returns something that looks fine. The UI says the task completed. But when you dig in, the command never ran, the wrong thing ran, or it checked its own assumption instead of the actual system state.
Isolation still matters — I wouldn't run arbitrary agent code without boundaries.
But a sandbox alone doesn't answer: "did the thing I care about actually happen?"
That's where I've shifted my thinking: less "is it isolated?" and more "can I reconstruct what executed — command, args, when, and outcome?"
Curious: if you've shipped agents near real systems, how do you catch the "looks fine, did nothing" cases?
Top comments (1)
The subtle issue is that “success” should be a property of the observed side effect, not the agent’s response. I’d make the execution boundary emit an immutable receipt containing the intended action, actual command/tool invocation, target state, and postcondition check. Then the UI can only mark a run successful when the postcondition is independently verified. That separation prevents the agent from effectively becoming both the actor and the judge of its own work.