No API? No Problem!

#automation #ai #mcp #api

Why Computer Use Agents (CUAs) Fail

When we set out to build AI tools for healthcare, we had to integrate with a maze of legacy systems that either had no public APIs or charged absurd fees to access their internal ones. Multiple customers were ready to pay us, as long as our AI could work with their EHRs, CRMs, and internal portals. Out of desperation, we turned to Computer Use Agents — the tech everyone promises will “solve browser automation.”

Here’s what we actually found:

Too slow — they reload entire pages just to extract a single field
Too expensive — you’re burning tokens just to click a button
Break constantly — especially when confronting date pickers 📅🤦 (I once looked through the reasoning logs and the LLM appeared to have briefly contemplated its own existence.)

CUAs promise generalizability, but modern web apps aren’t built for systems that take discrete screenshots and spend seconds reasoning before each click. They’re designed for a human’s continuous perception, attention, and reaction time. By the time a CUA decides what to do, the DOM has shifted, the element has re-rendered, and the button it planned to click has already disappeared.

Our Insight: Web Apps Already Reveal Their Own APIs

The breakthrough for us was realizing that every modern web app must communicate with backend services — and that communication is inherently structured. The browser already knows how to authenticate, fetch data, submit forms, paginate results, and trigger workflows. And if the browser knows it, we can know it too.

By observing the full browser state, we can decode the hidden contract between the frontend and backend:

HTML structure and DOM mutations
JavaScript bundles and the logic they execute
Cookies and authentication tokens
Local and session storage
All network traffic: XHR, fetch, GraphQL, REST, streaming
Runtime signals from the JS process itself

Once you can see all of this, the structure of the app’s communication reveals itself.

And once you understand that structure, you can replicate it without scraping pixels or driving a synthetic user.

Introducing Vectorly: Turn Web Apps into APIs

Vectorly doesn’t try to “be a human in the browser.” Instead, it instruments the browser and turns what it sees into reusable automation recipes.

Step 1 — Interact with any website in the Vectorly Console

You browse the site naturally—click, search, scroll, and explore as you would. While you interact, Vectorly captures everything the browser sees: network traffic, cookies, storage, HTML, and JS.

Step 2 — Tell the agent what you want

Once you’re done with the interaction, you simply describe what you need:

The data you want extracted (e.g. “all upcoming bookings for this account”)
The API behavior you want replicated (e.g. “search flights with these parameters”)

Our AI agent then reviews all the captured assets and infers the underlying structure of the app’s internal APIs and flows.