I built an open-source bridge that lets one AI chat work across multiple Windows PCs

رتAhmed Zaki — Wed, 07 Oct 2026 08:00:33 +0000

I wanted to solve a very specific problem: AI chats can reason, but they usually cannot actually operate the Windows computers where the work lives.

So I built YourHand - a Windows control layer that connects supported AI clients to one or more Windows PCs.

The goal is simple:

One AI chat should be able to work across the computers you already use.

That means opening applications, reading the screen, clicking, typing, managing files, running commands, and coordinating tasks across multiple machines - without forcing everything into a browser-only workflow.

Why I built it

My daily work is spread across several Windows machines. Different computers have different applications, files, sessions, and workloads. A normal AI assistant can tell me what to do, but I still have to jump between machines and execute every step manually.

I wanted the interaction to feel closer to this:

Me: Open Revit on PC-075 and check whether the project is already running.
AI: [checks the device, opens the app if needed, reports the result]

Me: Now open Chrome on BeDo-Tower and test the latest build.
AI: [works on the second machine without losing the first context]

The important part is not the mouse click itself. The important part is persistent, account-scoped access to multiple real Windows computers from the same conversation.

The architecture

The system is split into a few layers:

AI client
   
YourHand MCP / plugin
   
Control plane
   
Capability router
   
Windows agent
   
OS / browser / UI / files / commands

The control plane handles account identity, device ownership, sharing, routing, and request coordination.

Each Windows machine runs a lightweight agent. The agent exposes capabilities such as:

window and application control
keyboard and mouse input
screen observation
file access
command execution
browser control
device health and telemetry

The agent is designed to run quietly in the background. The user should not need to keep a terminal window open just to make the computer controllable.

Why a capability router matters

One of the biggest lessons from building this is that desktop automation should not depend on a single technique.

A click can be executed through several possible paths:

direct OS or application APIs
browser CDP when the target is a web page
semantic Windows UI Automation
pixel-based interaction
foreground input as a fallback
vision-assisted fallback when the UI cannot be addressed semantically

So instead of treating "click this button" as a single primitive, YourHand routes the task to the most appropriate capability.

That makes the system more robust than relying entirely on screenshots and blind cursor movement.

Multi-device was a first-class requirement

A lot of automation tools assume one session, one machine, one active task.

That was not enough for what I wanted.

YourHand treats devices as explicit resources tied to the user's account. A conversation can target different computers by name, and the control plane routes each request to the correct agent.

That also made sharing important.

The design goal is that one Windows agent can remain installed while access is granted to another authorized account. The device does not need a second competing agent just because another team member needs access.

Authentication without handing over passwords

I did not want users copying AI-provider passwords or long-lived manual tokens into the desktop agent.

The current flow uses Google sign-in and OAuth-based authorization for the YourHand account. Devices are then paired to that account and can be explicitly shared.

The separation is important:

the AI client gets authorization to call YourHand
YourHand decides which devices that account may use
the Windows agent only accepts work routed through the control plane

Observability matters more than I expected

Desktop automation fails in messy ways.

A command can time out even though the action happened. A window can disappear between discovery and input. A browser can be alive while the target tab is not. A device can reconnect with a stale socket.

Because of that, I added telemetry around calls: success/failure, execution method, latency, retries, and device health.

This has been one of the most valuable parts of the project. Without observability, every failure looks like "the AI clicked the wrong thing." With telemetry, you can distinguish routing failures, UI failures, transport failures, and application failures.

Open source, but not just a demo

I published a community repository because I want the Windows/AI-control layer to be inspectable and extensible.

The project is still in beta, but the target is not a toy demo. The architecture is being built around:

multiple Windows devices
multiple authorized users
account isolation
browser and native UI control
background execution
recovery after restarts
device telemetry
controlled sharing

The hard part is not making a mouse move once. The hard part is making the whole path reliable enough that a user can trust the system across real machines.

What I am testing now

The current beta is focused on three things:

Reliability - fewer fragile UI assumptions and better recovery.
Speed - choosing direct APIs or semantic automation before falling back to screenshots.
Multi-device workflows - keeping one conversation useful across several computers.

I am also testing the onboarding flow for people who have never configured an MCP server or a Windows automation agent before.

Try it

YourHand is currently free during beta.

Website: https://yourhand.wolvexai.com/

Open-source community repository: https://github.com/archahmedzaki/YourHand-Community

I would especially like feedback from people who build developer tools, IT automation, MCP integrations, or Windows desktop software.

The question I am trying to answer is:

What would make you trust an AI chat enough to let it work across your real computers?

DEV Community: رتAhmed Zaki