AI is getting better at reasoning.
It is still surprisingly bad at using software.
That gap is not because the models are weak. It is because the web was never designed to be legible to machines in the first place.
Most modern interfaces are optimized for human perception. We use typography, spacing, button placement, color, and copy to communicate meaning. A person can glance at a checkout screen and understand where they are, what fields matter, what the primary action is, and what will probably happen next.
An AI agent does not get that for free.
When an agent lands on a page, it usually has to reverse-engineer the interface from unstable clues. It scrapes the DOM. It follows brittle selectors. It guesses that the button labeled “Save” is probably the one it wants. It hopes the gray button is disabled and not just styled that way. It infers structure from wrappers, classes, and visible text that product teams change every week.
That is not intelligence. That is survival.
And it breaks constantly.
A small redesign can move one button, rename one label, or insert one extra wrapper div and suddenly the entire automation flow starts failing. A CSS selector that worked yesterday stops working today. A form field the agent identified by label text is now ambiguous. A modal opens with no explicit contract that says, “you are in a confirmation step now.”
This is why so much AI-powered UI automation feels flaky in production. The model is being asked to operate an interface that only humans were ever meant to understand.
That is the real problem.
We act like the browser is a universal interface. It is not. It is a universal visual interface.
For humans.
If AI is going to become a real operator of software, there is a missing layer we need to add between the rendered UI and the agent trying to use it. Not another prompt. Not better XPath. Not smarter screenshot parsing.
A contract.
That is the idea behind CortexUI.
CortexUI starts from a simple but important premise: a modern interface has two audiences now. The first is the human using it visually. The second is the machine trying to interact with it programmatically.
Most design systems only serve the first audience.
CortexUI is built for both.
A CortexUI component does not just render a button, a field, or a table. It also emits a machine-readable contract alongside the visual layer. Identity, role, action, state, screen context, section context, entity context. The UI stops being a pile of elements and starts being self-describing.
A button stops being “something blue near the bottom of the form.”
It becomes this:
<button
data-ai-id="save-profile"
data-ai-role="action"
data-ai-action="save-profile"
data-ai-state="idle"
>
Save Profile
</button>
That difference is bigger than it looks.
Now an agent does not need to guess. It can ask: what actions are available right now? Which one is save-profile? Is it idle, loading, disabled, or finished? Which screen am I on? What entity is this form editing?
In CortexUI, the interface can answer those questions directly.
That changes the architecture of AI interaction completely. Instead of DOM scraping and heuristics, you get deterministic access to the meaning of the page. The UI becomes less like a screenshot the model has to interpret and more like an API the model can query.
That is the shift.
Not “AI in the UI.”
UI built so AI can actually use it.
The web was built for humans. That was the right decision for the last era.
But the next era includes operators without eyes.
If we keep building interfaces that only humans can truly understand, AI will remain trapped in a layer of brittle guesswork.
If we build interfaces that declare what they mean, what they do, and what state they are in, something more interesting happens.
The UI stops being a visual artifact.
It becomes a contract.
And that is where CortexUI begins.
If you want to dig deeper, explore CortexUI on GitHub or read the docs at cortexui.llcortex.ai.
Top comments (1)
it’s wild how much we take for granted about a button being clickable. even with the newest models, i spend half my time explaining why a simple dropdown won’t trigger—ai sees the code but misses the human intent. maybe we need less 'shiny' and more 'semantic' if we want these tools to actually handle the grunt work for small business software. austin taught me: just start the thing, but it’s hard when the ai is still trying to find the door handle.