A few days ago I wrote about a tool that lets agents drive your real browser. It is a good tool. Browser automation exists because most software does not expose an API you are allowed to use, so teaching an agent to click is the only way in.
Then people take that pattern and point it at their own application. That is where it stops making sense.
You are making the agent reverse-engineer your own server
Think about what happens when an agent clicks a button in an app you built.
Your server has the record. Structured, typed, validated. It serialises it to JSON, ships it to the browser, React turns it into DOM nodes, and then the agent reads those DOM nodes and tries to reconstruct the object your server had in the first place.
Every layout change breaks it. Every loading state is a race. You pay tokens for a screenshot or an accessibility tree to recover data you already had in the right shape one hop earlier.
You did not need an agent that can use your interface. You needed an interface and an agent that call the same thing.
One schema, six callers
Builder.io's Agent-Native is built on that. You define a capability once as an action: a zod schema and a run function. Then the agent gets it as a tool, React calls it with a hook, and it is simultaneously an HTTP endpoint, an MCP tool, an A2A endpoint and a CLI command.
The detail I like most is where the prompt engineering lives. The action's description, and .describe() on each schema field, become the tool description and parameter docs the model reads. They sit in the same file as the implementation, so they cannot drift from it. Anyone who has maintained a tool manifest in a second file knows exactly how that drift goes.
The schema is doing triple duty: generating the JSON Schema the model sees, validating what the UI sends, and validating what the agent sends. If you are hand-writing tool definitions today, a JSON Schema generator makes it obvious how much boilerplate that single declaration is replacing.
Where this does not apply
Say the limit plainly. This only works for software you build this way. It has nothing to offer for the SaaS tools you do not control, and there the browser-driving approach is still correct.
It is also not generative UI, which is a different thing that keeps getting conflated with it. Generative UI means the model produces the interface at runtime. Here the interface is hand-authored and the agent gets the same actions, not the ability to invent screens.
The rule I would take away: drive the browser for other people's software, share the action layer for your own. If you are doing the first for the second, you built an API and then decided to consume it through a picture of itself.
Fuller write-up, including the nine open-source reference apps and a licensing footnote worth knowing: Agent-Native.
Top comments (0)