There's a discussion running on Product Hunt right now about where an AI agent should stop and hand a decision back to a person. The framing that stuck with me: "can this be undone?" is the wrong gate. It over-fires on things nobody cares about, like writing a log line, and under-fires on the ones that actually hurt. A mass email. A migration you can only roll back with downtime.
The proposed replacement is reach. Not "is this safe" but "how far does this go if I'm wrong." And the sharpest point in that thread is that the agent is the worst possible judge of its own reach, because it doesn't know there are 50,000 people on the list.
I've been building a browser with an agent in it, and the browser case makes that concrete in a way the terminal case doesn't.
A coding agent's blast radius is usually one person: you, reading a diff. A browser agent doesn't start there. It inherits every session you're already signed into. The reach of a click isn't a property of the agent's plan, it's a property of the cookie jar it's holding. The same "click the blue button" step is harmless on a docs site and irreversible on a payments dashboard, and nothing in the agent's own summary distinguishes them.
So I stopped trying to make the agent classify risk. Here's what I do instead, with the caveat that this is alpha and I've been wrong about it before.
Some things never reach the model. Credential fields are filtered out of the page snapshot before the agent sees it. That means password inputs, and anything with cc-number, cc-csc, cc-exp, or one-time-code autocomplete semantics. A fill targeting a password field refuses outright. The agent cannot misjudge the reach of a control it was never shown, and that's a stronger guarantee than any classifier, because it doesn't depend on getting a judgment right.
The model's judgment is a floor, never a ceiling. For everything that does reach it, the agent can say "this needs approval" and be believed. It cannot say "this is routine" and be believed. A structurally sensitive control still escalates to a native confirmation even when the model called the action ordinary. The agent can raise the gate; it can't lower it.
Sensitivity is read off the DOM, not off the plan. What marks a control sensitive is its type and autocomplete semantics, which is evidence in the page itself, rather than the agent's description of what it means to do.
A timeout is a denial, not a wait. A stalled job is a silent failure; a denied one is loud, and loud failures get fixed.
The part I haven't solved is denied-and-resumable. Discarding the work is clean and useless, because people want to come back twenty minutes later and continue. But that means storing the world as it was when the question was asked, and the world moved. In a browser that's especially unkind: the DOM you asked about doesn't exist anymore. I don't have a good answer for it yet.
Top comments (5)
I like putting the hard gate outside the agent. A reversible click can still have a huge audience if the session is sitting on the wrong account, and the model usually cannot see that boundary. The resumable-denial piece feels like the hardest part. Do you snapshot the DOM only, or the action queue plus browser state?
Neither, and that turned out to matter. What I keep is not the DOM, it's a flat list of visible labelled controls: kind, label, url, disabled, selected, sensitive. Capped at 200, each one handed to the model as a ref like e12.
The action path doesn't trust that ref. It re-runs the same enumeration against the live page and checks the control at that index still has the same label and kind. If either moved, it refuses rather than clicking.
So there's no stored browser state to go stale. There is a narrower hole: a control whose label and kind match but whose meaning changed still passes.
Reach over reversibility is the right axis I think.
The credential-filtering move generalises past inputs. The same guarantee works on the way out: any secret an agent handles gets written to a file and referenced by path, never echoed into the transcript. A value that never enters the context can't be leaked by a summary, a log, or a screenshot of the session, and that holds without the agent having to decide to be careful. Same shape as your snapshot filter, opposite direction.
On reach being a property of the cookie jar: that also means reach is selectable before the run starts. We keep two browser paths on purpose. One drives the real profile with live sessions, for the tasks that genuinely need them. The other launches a fresh, empty Chromium, and that's the default for anything that doesn't. The cost is real, you script the login instead of already being logged in, and it has overruled the convenient choice more than once. It's still cheaper than gating well, because a fresh profile has no reach to misjudge.
On denied-and-resumable, the reframe I'd try: attach the approval to a claim about the world rather than to a DOM node. "Click this button" can't be resumed. "Pay invoice X, amount Y, recipient Z" can be re-checked twenty minutes later against a page that has moved, by re-locating the target semantically and re-verifying the claim. Claim still holds, approval still applies. Claim doesn't hold, it's a new question rather than a resumed one, which is the right answer and not a limitation. It doesn't cover everything, an approval whose claim was never precise enough to re-verify is just unresumable, but that's a narrower problem than storing the world.
Agreed, and the return path turned out to be the cheap half. My action results only ever name the control, never the value: a fill comes back as Filled "Card number". So nothing the tool writes back into the transcript carries the secret, and a password input refuses outright.
Inbound is the part I have not solved. Filtering the snapshot keeps a password field out of the model's view, but if a value reaches the model any other way it is already in the context by then, and reference-by-path is the only thing that actually fixes that.
My agent treated every stop as terminal, so an ad playing or a sign-in screen killed a run exactly like an impossible task did.
Now the stop carries one flag: can a person clear this in the tab? If yes it pauses, you clear it, and it resumes from a fresh snapshot. Two handoffs, then it really stops.