There are two ways to give an agent access to your software. One is structured. The other is screen-scraping. They look similar in a demo. They behave nothing alike in production.
Screen-scraping means the agent reads the UI the way a human does: parsing pixels, clicking buttons, filling forms. It works until the UI changes. A redesigned button, a moved field, a new modal — and the agent breaks silently, doing the wrong thing with full confidence.
Structured access means the agent talks to the product's actual capabilities: typed actions with defined inputs, outputs, and permissions. When the UI changes, nothing breaks, because the agent was never looking at the UI.
The deeper problem with scraping isn't fragility. It's blindness. An agent clicking through a UI can't tell which actions are safe and which are irreversible. It has no concept of permissions — it just does what the screen lets it do.
Structured access makes the boundaries explicit. Every action declares what it does, what it needs, and whether it needs approval. The agent operates inside a contract, not a guessing game.
If your agent broke the moment someone redesigned a page, it wasn't an agent problem. It was an access problem.
What's the most fragile UI-dependent workflow you've seen an agent try to handle?
Top comments (0)