Building a Software KVM in Python with Clean Architecture and AI

What is mouse-share?

mouse-share is a software KVM — it lets you share a single keyboard and mouse across multiple computers on your LAN. Move your cursor off the edge of one screen and it appears on the next machine, exactly like a multi-monitor setup but across separate PCs.

If you've used Barrier or Synergy, you know the concept. I built mouse-share because the existing options are either abandoned, commercial, or carry decades of C++ complexity. I wanted something lightweight, strongly typed, and written in modern Python — with gaming-grade latency to boot.

The result is a ~3,500-line Python project with clean architecture, protocol-based dependency injection, platform-specific backends for Linux and Windows, a binary UDP event protocol, and a 310-line ops runbook. This article walks through the architecture and how AI accelerated the build.

Architecture Overview

The project follows a layered architecture that keeps platform-specific code isolated from business logic:

src/mouse_share/
├── event/                 # Domain — platform-agnostic events & codec
│   ├── types.py           #   Frozen InputEvent dataclasses
│   ├── codec.py           #   Binary UDP wire format
│   ├── held.py            #   Tracked key/button state
│   └── scancode.py        #   HID ↔ platform scancode maps
│
├── edge/                  # Domain — screen geometry & crossing
│   ├── detector.py        #   Cursor tracking state machine
│   └── layout.py          #   Multi-screen adjacency graph
│
├── capture/               # Infrastructure — OS input capture
│   ├── base.py            #   CaptureBackend protocol
│   ├── linux_x11.py       #   X11 XRecord
│   └── windows.py         #   Win32 low-level hooks
│
├── emulate/               # Infrastructure — OS input injection
│   ├── base.py            #   EmulateBackend protocol
│   ├── linux_x11.py       #   X11 XTest
│   └── windows.py         #   Win32 SendInput
│
├── net/                   # Infrastructure — networking
│   ├── protocol.py        #   Message framing & constants
│   ├── control.py         #   TCP control channel
│   ├── transport.py       #   UDP event transport + batching
│   ├── link_selector.py   #   Fastest-NIC latency probing
│   └── sockopt.py         #   Low-latency socket tuning
│
├── clipboard/             # Infrastructure — clipboard sync
│   ├── base.py            #   ClipboardBackend protocol
│   └── qt.py              #   Qt-based implementation
│
├── platform.py            # Factory — platform detection & DI
├── app.py                 # Application — orchestrator & state machine
├── config.py              # Application — config persistence
├── hotkey.py              # Application — escape combo detector
└── screen.py              # Application — multi-monitor detection

At the centre is a state machine that drives the entire application:

IDLE → CONNECTING → ACTIVE_LOCAL → ACTIVE_REMOTE
                                 ↘ CONTROLLED

When you're using your local machine, you're in ACTIVE_LOCAL. Move the cursor off-screen and focus switches to the remote peer (ACTIVE_REMOTE). The remote peer enters CONTROLLED, injecting your input. Push the cursor back across the edge and focus returns.

Domain Layer — Platform-Agnostic Events

Every input event in mouse-share is a frozen dataclass using USB HID scancodes as the universal keycode space. This means the domain layer has zero platform dependencies:

# src/mouse_share/event/types.py

@dataclass(frozen=True, slots=True)
class MouseMove:
    dx: int
    dy: int
    event_type: EventType = EventType.MOUSE_MOVE

@dataclass(frozen=True, slots=True)
class KeyPress:
    scancode: int
    event_type: EventType = EventType.KEY_PRESS

@dataclass(frozen=True, slots=True)
class KeyRelease:
    scancode: int
    event_type: EventType = EventType.KEY_RELEASE

InputEvent = MouseMove | MouseButton | MouseScroll | KeyPress | KeyRelease

The InputEvent union type gives exhaustive pattern matching everywhere events are consumed. Frozen dataclasses make events hashable and safe to pass between threads. Slots keep memory tight — these objects are created at up to 1,000 Hz on a busy mouse.

HID scancodes were the right choice for the canonical keycode space. Every platform captures in its native codes (Win32 virtual keys, X11 keysyms) and the capture backend translates to HID at the boundary. The emulation backend translates back on the receiving end. This keeps all the cross-platform mapping logic in exactly two places per platform.

Protocol-Based Dependency Injection

Instead of ABC inheritance, mouse-share uses Python's @runtime_checkable Protocols for all platform contracts. Here's the capture backend interface:

# src/mouse_share/capture/base.py

@runtime_checkable
class CaptureBackend(Protocol):
    def start(self, on_event: Callable[[InputEvent], None]) -> None: ...
    def stop(self) -> None: ...
    def set_suppressing(self, suppressing: bool) -> None: ...
    def set_escape_callback(self, callback: Callable[[], None]) -> None: ...
    def cursor_position(self) -> tuple[int, int] | None: ...

And the emulation backend:

# src/mouse_share/emulate/base.py

@runtime_checkable
class EmulateBackend(Protocol):
    def start(self) -> None: ...
    def stop(self) -> None: ...
    def inject(self, event: InputEvent) -> None: ...
    def warp(self, x: int, y: int) -> None: ...
    def cursor_position(self) -> tuple[int, int] | None: ...

Why Protocols over ABCs? Three reasons:

1. Structural typing. A class satisfies the protocol by having the right methods — no inheritance required. Platform backends don't need to import or subclass anything from the core.
2. Runtime checking. @runtime_checkable lets you do isinstance(obj, CaptureBackend) for defensive assertions without paying the ABC tax.
3. Test doubles for free. Any object with the right shape works, which makes mock creation trivial in the 20-module test suite.

Platform Factory — Targeting Linux and Windows

Platform-specific modules are lazily imported through factory functions. This avoids import errors on the wrong OS — you can develop the Windows backend on a Linux machine without import failures:

# src/mouse_share/platform.py

def create_capture_backend() -> CaptureBackend:
    if sys.platform == "win32":
        from mouse_share.capture.windows import WindowsCapture
        return WindowsCapture()

    if sys.platform == "linux":
        session_type = os.environ.get("XDG_SESSION_TYPE", "").lower()
        if session_type == "wayland":
            raise RuntimeError(
                "Wayland capture is not yet supported. "
                "Run under X11 or Xwayland (XDG_SESSION_TYPE=x11)."
            )
        from mouse_share.capture.linux_x11 import X11Capture
        return X11Capture()

    if sys.platform == "darwin":
        raise RuntimeError("macOS capture is not yet implemented.")

    raise RuntimeError(f"Unsupported platform: {sys.platform}")

The pyproject.toml uses environment markers to keep platform dependencies separate:

• Linux: evdev >= 1.7 + python-xlib >= 0.33 (X11 XRecord/XTest)
• Windows: pure ctypes — zero extra dependencies
• macOS: stubs ready for pyobjc-framework-Quartz + pyobjc-framework-Cocoa

Each platform implementation is radically different under the hood. The Windows capture backend runs a Win32 message pump with SetWindowsHookEx for WH_MOUSE_LL and WH_KEYBOARD_LL in a dedicated thread. The Linux backend uses XRecord to tap the X11 event stream. But from the orchestrator's perspective, they're identical — just a CaptureBackend that calls on_event.

Networking for Low Latency

A software KVM lives or dies by its latency. mouse-share uses a dual-channel architecture:

• TCP (port 24800): Control channel — length-prefixed JSON messages for handshake, screen config, clipboard sync, and focus switching.
• UDP (port 24801): Event transport — binary-encoded input events batched and flushed at 125 Hz.

The binary codec keeps events tiny:

Header (4 bytes): version:u8 | flags:u8 | count:u16 LE

MouseMove:    0x01 | dx:i16 | dy:i16        → 5 bytes
MouseButton:  0x03 | button:u8 | pressed:u8 → 3 bytes
KeyPress:     0x10 | scancode:u16           → 3 bytes
KeyRelease:   0x11 | scancode:u16           → 3 bytes

Pre-compiled struct.Struct objects eliminate per-call overhead in the codec. A typical batch of 8 mouse moves fits in a single 44-byte UDP datagram — well under the MTU.

Mouse move coalescing is critical. At 1,000 Hz polling, the mouse generates far more events than we need to send. Consecutive MouseMove events are collapsed into a single event with accumulated deltas before each flush. This cuts traffic dramatically without adding any perceptible latency.

On the socket side, every knob is tuned for LAN latency:

• DSCP/TOS marking (0xB8 — Expedited Forwarding) so managed switches prioritise our traffic
• TCP_NODELAY to disable Nagle's algorithm on the control channel
• SO_BUSY_POLL (Linux) — kernel spins for 50μs instead of sleeping, shaving ~45μs off receive latency
• SO_PRIORITY (Linux) — elevated kernel packet priority
• SO_BINDTODEVICE — NIC pinning when multiple interfaces are present

For multi-NIC setups, a link selector probes all local×remote IP pairs with timed UDP round-trips and picks the path with the lowest median RTT. This means if you have both Wi-Fi and a direct ethernet cable to your other machine, mouse-share automatically uses the cable.

Edge Detection & Focus Switching

The edge detector tracks the cursor position in layout coordinates. When the cursor reaches the edge of a screen and the next delta pushes it outward, a crossing event fires. The orchestrator then:

1. Records held keys and buttons via HeldInput
2. Sends synthetic release events for all held inputs on the source machine (preventing stuck keys)
3. Switches focus to the target peer via a SWITCH_FOCUS control message
4. Warps the cursor to the entry point on the target screen (proportionally mapped along the shared edge)

If things go wrong — say the remote machine crashes — pressing Ctrl+Alt+Escape immediately breaks out of remote mode and returns control to the local machine. The hotkey detector runs inside the capture backend itself, so it works even when events are being suppressed.

How AI Accelerated Development

I used Claude Code throughout this project as a pair-programming partner. Here's where AI made the biggest difference:

Scaffolding the architecture. The Protocol-based DI pattern emerged from an early conversation about how to keep platform code isolated. I described the constraints (multiple OS backends, no framework dependencies, easy to test) and the AI suggested @runtime_checkable Protocols with factory functions — a pattern I hadn't used at this scale before. It was the right call.

Platform-specific low-level code. Writing X11 XRecord capture code or Win32 SetWindowsHookEx message pumps is not something most Python developers do daily. The documentation for these APIs is sparse, scattered across man pages and decades-old MSDN articles. AI was invaluable here — it could synthesize working ctypes bindings and XRecord setup code that would have taken hours of documentation archaeology to piece together manually.

Comprehensive test suite. The project has 20 unit test modules covering everything from the binary codec to platform-specific capture and emulation backends. AI generated the initial test scaffolding and helped ensure edge cases were covered — things like truncated datagrams, scancode boundary values, and multi-monitor geometry.

The runbook. The 310-line operations guide covering installation, firewall rules, auto-start configuration (systemd, launchd, Task Scheduler), and troubleshooting was drafted collaboratively. AI knew the exact ufw commands, launchd plist format, and PowerShell Register-ScheduledTask invocations, saving significant research time.

What AI didn't do: replace understanding. Every suggestion was reviewed, tested, and often adapted. The architecture decisions were mine. The AI accelerated the implementation — especially in domains where documentation is sparse — but it didn't write the project for me. It's a research accelerator, not a code generator you trust blindly.

Lessons Learned

Clean architecture pays off for multi-platform projects. The upfront cost of defining protocols and factories is trivial compared to the cost of entangling platform code with business logic. When I added Windows support, I didn't touch a single line in the domain or application layers.

Protocols + factory functions = lightweight DI without frameworks. No dependency injection container, no decorators, no magic. Just interfaces defined as Protocols and a factory function that returns the right implementation. It's simple enough to fit in your head and powerful enough to support multiple platforms and a full test suite.

AI is best used as a research accelerator for obscure APIs. The places where AI saved the most time weren't the high-level architecture (that requires human judgement about trade-offs) but the low-level platform glue: ctypes struct definitions, X11 extension setup, Win32 message pump boilerplate. For these tasks, AI is dramatically faster than reading sparse documentation.

Try It Out

mouse-share is open source and available on GitHub: github.com/wright-benjamin-1701/mouse-share

The project ships with a comprehensive runbook covering installation, firewall setup, and auto-start configuration for all supported platforms.

Contributions welcome — especially macOS support (the protocol stubs are ready) and Wayland capture via the input-capture portal.