AI can draft instructions in seconds. But when the actual work is in a Windows application-Excel, a CAD tool, a file explorer, an admin console, or a browser tab-the human is still doing the clicking.
And if the job spans three different PCs, a chat that can operate only one browser tab is not enough.
That's why I built YourHand: a bridge that lets compatible AI assistants interact with the Windows computers you explicitly authorize, using one conversation.
This is a practical look at the architecture, the problems that surprised me, and how you can try the hosted beta.
The missing layer is execution
Imagine asking an AI assistant:
"On WORK-PC-01, open the project folder and tell me which reports were updated today. On WORK-PC-02, check which application windows are running. Then report the findings here."
That requires more than generating a shell command or suggesting a step-by-step tutorial. The assistant needs to reach the right machine, observe its state, choose an appropriate interaction method, and report what actually happened.
The flow looks like this:
Your AI chat (compatible MCP client)
|
v
YourHand MCP / authorization layer
|
v
Account-scoped device routing
|
+--> WORK-PC-01: apps, files, windows
|
+--> WORK-PC-02: browser, UI, processes
|
+--> WORK-PC-03: other authorized workflows
YourHand's Windows agent is installed on the PC. The assistant gets access through the authorized account and the exposed capabilities-not by taking over an unrelated user's computer.
The first lesson: a click isn't one method
Different Windows tasks need different techniques.
- Browser pages: the Chrome DevTools Protocol (CDP) can be more precise than guessing coordinates on a screenshot.
- Desktop applications: Windows UI Automation can expose controls by role, label and state.
- OS and files: native APIs and guarded file/process actions can avoid unnecessary cursor movements.
- Visual or unusual interfaces: screenshots and native input can be fallbacks when semantic interaction isn't enough.
A useful control layer should select the appropriate path rather than treating every job as "look at image, guess where to click."
That is the idea behind YourHand's capability routing.
The second lesson: multiple PCs change the problem
A single-machine demo doesn't answer the questions that matter in everyday use:
- Which account owns each device?
- Which devices can this conversation actually target?
- What happens if a machine disconnects or the UI changes?
- Can one authorized user share access without installing competing agents?
- How does the assistant know whether a timed-out action already happened?
In YourHand, device pairing, account-scoped access, controlled sharing, operation journaling, and telemetry are important parts of the system-not decorative settings.
The third lesson: a timeout does not mean nothing happened
Desktop automation can time out after an action already changed the world.
Blindly repeating that action might create duplicate files, send something twice, or make the state worse.
So the execution layer needs to distinguish success, failure, and unknown outcome, and use fresh observations before retrying state-changing operations. This is especially important when the assistant works across remote machines.
Three workflows worth testing
1. Multi-PC status check
"List my connected Windows PCs. On each authorized machine, tell me the active windows and relevant system status."
2. Desktop + browser workflow
"On my test PC, inspect the open browser tabs, open the designated application, and tell me what you actually observed."
3. Project file organization
"On my authorized workstation, inspect this test folder, suggest an organization plan, then carry out only the changes I approve."
Start with disposable or non-sensitive tasks. Real Windows apps have permissions, dialogs, unsaved changes and other surprises that no polished demo can eliminate.
How to try it
The official hosted YourHand beta offers Google sign-in and a guided Windows installation/connection flow.
- Open YourHand.
- Sign in and pair a Windows PC you own or are authorized to operate.
- Connect a supported AI client using the website's instructions.
- Test a small read-only task, then gradually try other workflows.
The beta is currently free. Supported clients and the behavior of individual tools can vary; always check your device permissions and review sensitive operations.
Watch the product demo: https://youtu.be/r96J26DnNc4
A note on the open-source repository
The YourHand Community repository is an AGPL-3.0 source preview, with architecture, tests, security documentation and explicit limitations. It is not the ready-to-install hosted app or a production deployment recipe. I'm sharing it for technical feedback and review, while the official service is maintained separately.
What would you automate first?
If you could give a compatible AI chat authorized access to your actual Windows applications-not just a web page-what is the first repetitive task you'd delegate?
I'm particularly interested in feedback from Windows developers, IT operators, QA engineers and people whose work crosses several PCs.
Try the hosted beta: yourhand.wolvexai.com
Explore the documented source preview: GitHub
Top comments (0)