Web performance has traditionally always been about human user experience. In recent time, that has evolved to include AI agents as well. AI agents can be thought of as anonymous and agentic systems that browse, consume and interact the web on behalf of users. As more and more people continue to use AI agents or search engines pushing AI mode, it is recommended to optimize for agentic browsing as well.
Google has recently included an agentic browsing category for both lighthouse and pagespeed insights, although they are currently experimental.
How do we do that? Broadly speaking, there are a few basic steps that can be covered here:
1. Accessibility Tree:
When agents browse your page, they do not typically "see" the page as a human would. They parse the DOM structure to analyze element roles, names and relationships. If any part of the tree is incomplete, it may lead to broken data and checkout flows. Google Lighthouse today measures this by the following methods:
- Every interactive element (button, input, link) needs to have a text node or
aria-labelassigned to it so they are "visible" to agents. - Parent - child relationships or sibling relations must be clear in the tree. For example, a label must be clearly tied to an input and so on. This is for better context of the general page structure and executing workflows.
- Interactive elements should NOT be hidden - A clickable Submit button hidden via CSS or
aria-labelconfuses agents.
2. Proper Metadata:
A llms.txt present at the root of your domain summarizes key pages and sections for the agent to browse for any given task without having to hit every page unnecessarily. There are more advanced techniques here like schema.org and llms-full.txt for a more enhanced summary.
3. Cumulative Layout Shift (CLS):
Cumulative Layout Shift is already a part of traditional web performance metrics but for agents, it is even more crucial as layout stability ensures that the agent does not misclick or misfire anything in a given workflow or task.
4. WebMCP Adoption:
With WebMCP, agents are provided with APIs that enable them to directly take any action based on the user prompt. There are currently two ways to implement this - a declarative API, defined directly with HTML forms or imperative API, with JavaScript. This is currently available under an origin trial - meaning you can actually try this out today in production with real users. This has practical use cases - like ecommerce, travel and general automation of day-to-day tasks.
Closing thoughts
I'm not sure the future of the web will be fully autonomous (or if we would have evolved to some new interface) but for the moment, the best strategy is: build your page for humans and optimize the basics first - accessibility, structure, metadata, stability of the page. Then if and when the need arises, WebMCP and other technologies could be gradually introduced.
Top comments (7)
I like the broader message that good engineering for humans usually benefits AI agents as well. Semantic HTML, accessibility, stable layouts, and structured metadata improve robustness regardless of who—or what—is interacting with the page. One nuance I’d add is that the agent ecosystem is still very heterogeneous. Some agents rely heavily on the accessibility tree, others inspect the DOM, execute JavaScript, use visual reasoning, or interact through APIs. Likewise, llms.txt and WebMCP are promising directions, but they are still emerging rather than universal standards. For me, the safest strategy today is: optimize for web standards first (accessibility, semantics, performance, structured data), then add agent-specific capabilities where there is clear demand. That way the site remains resilient even as agent ecosystems evolve.
Makes sense - just curious, what kind of web interfaces do you think will be in demand in the future?
That’s a fascinating question. I don’t think we’ll end up with a single dominant interface. Instead, we’ll probably have multiple interaction layers over the same application.
Humans will still prefer rich visual UIs for exploration and complex tasks. AI agents, on the other hand, will increasingly rely on structured interfaces—well-defined APIs, MCP-like protocols, semantic metadata, and machine-readable workflows—because they’re more reliable than interpreting pixels or arbitrary HTML.
My guess is that successful applications will expose both: a great human experience and a first-class machine interface. Much like responsive design became standard for different screen sizes, “agent-ready” interfaces may become another capability rather than a separate product.
I like the point about it being a add on than a separate product. I have revised the post with my updated stance now. Thanks for weighing in!
Thanks for being open to the discussion. I think that small shift in framing makes the article even stronger. Technologies will evolve, but building on web standards first keeps applications useful for both humans and AI agents regardless of which protocols ultimately win.
Thank you. Now I know how to make things difficult
Well, I agree it seems a bit complex but these are all experimental and new APIs. Plus, as I said, who knows the future of the web? We'll just have to see how it shakes out