Everyone benchmarking computer-use agents measures the same thing: task success rate. Did it fill the form correctly? Did it find the file? Useful numbers, and they miss the variable that actually determines whether an agent is worth running — wall-clock occupancy of the host machine.
I ran two agents side by side for a month against real work, on infrastructure I administer. Here is what I learned about where the time goes, and why that number, not accuracy, is the one that should drive your deployment topology.
The two subjects
- Claude, via its computer-use capability.
- An open-source agent driven by Grok.
One commercial, one open, deliberately. I wanted to separate product-specific behaviour from category-level behaviour. Almost everything I observed turned out to be category-level.
Where the wall clock actually goes
Break an agent's task into phases and the distribution is not what you would guess from a demo reel:
1. Settling waits. After every navigation and every click, the agent waits to confirm the resulting state. It cannot trust that a page finished loading, so it polls until it is confident. This is correct engineering. It is also the single largest consumer of wall clock in every trace I watched.
2. Action start-up latency. There is a non-trivial gap between "decision made" and "pixel moves." Per action it is small. Multiplied across a multi-step task it stops being small.
3. Disambiguation on unexpected state. When the screen does not match the model's expectation, the agent halts and re-establishes position. Sensible. Expensive.
4. Self-verification. The behaviour I did not anticipate. The open-source agent, filling a form, enumerated every <select> in turn, read the options, then captured a screenshot to persist the values so it could assert against them later. Read-back verification, essentially — the same pattern you would write into a flaky integration test.
That last one is the tell. The agent is optimising for correctness under no supervision, and it has an effectively unbounded time budget to spend on it.
The asymmetry nobody prices in
Here is the thing worth internalising:
Time is free for the agent and expensive for you.
The agent is not bored. It is not aware of the queue behind the current task. Spending four minutes verifying a form costs it nothing. You, meanwhile, have a support queue and a migration to finish.
This asymmetry is why "just wait for it" fails as an operating model. It is not that agents are unacceptably slow in absolute terms — it is that their tolerance for slowness is infinite and yours is not.
Why this forces a topology decision
A computer-use agent does not run beside your applications the way a background process does. It drives the same input primitives you do. Concretely:
- It has your browser — the real profile, with your sessions.
- It has focus. Window activation is part of its action space.
- Intervening mid-task is genuinely unsafe. You cannot reliably predict the next element it will target, and a stray click changes the state it believes it is in.
So the host is not busy. The host is claimed for the duration.
Combine claimed-host with unbounded-verification-time and you get the actual finding: a computer-use agent and a human cannot timeshare one desktop. Not a preference. A structural consequence of the agent's action space overlapping completely with yours.
The fix is a topology change, not a config change
Give the agent its own machine. That is the whole intervention.
The agent's twenty minutes stops being your twenty minutes. Its verification loops become irrelevant to you, because you are not watching them.
What matters for the target machine is mundane: a reliable Windows environment, the applications your agent drives, and availability whenever the agent runs. No GPU. Nothing exotic.
One deployment note that is easy to get wrong: billing model matters more here than instance size. An agent's retry-and-verify behaviour generates far more billable minutes than the same task performed by a human, so metered compute makes your costs a function of how thorough the agent decides to be. Flat-rate hosting removes that variable entirely. I run mine on Infosaic desktops — flat rate from $14.95/month, provisioned instantly on signup — but the general point stands whatever you pick: on this workload, predictable beats cheap-per-hour.
What I would measure next
If you are evaluating agents, add these to your harness alongside success rate:
- Host occupancy — wall-clock minutes the agent holds input control.
- Verification ratio — fraction of actions that are read-back rather than mutation.
- Recovery cost — time from unexpected state to resumed progress.
Success rate tells you whether the agent can do the job. These tell you what it costs you to let it.
This is the developer-facing version of a longer field report. More virtual desktop material: infosaic.com/virtual-desktop-resources
Infosaic Technologies has delivered managed Windows desktops in the cloud since 2001. infosaic.com
Top comments (0)