How a simple idea to control a Mac by tapping a desk inspired me to build a Windows application with hand gestures, automation, and hardware sensor support.
What if you could control your computer just by tapping your desk?
No extra hardware. No special touchpad. Just a microphone, an ordinary desk, and a few taps.
It sounds a little strange at first, but that’s exactly what caught my attention when I came across Holo, an open-source project for macOS created by JustinGamer191. I use both macOS and Windows regularly, so I tend to notice when an interesting application exists on one platform but not the other.
Holo was one of those projects. The idea was surprisingly clever. It used a MacBook’s microphone to recognize taps in different areas of a desk and turn them into actions.I liked the concept, but I also wanted something similar for Windows.
And once I started thinking about what a Windows version could look like, I had another thought. Why stop at desk taps? That’s how HoloWin started.
The Project That Started It All https://github.com/JustinGamer191/Holo
The original Holo is an experimental macOS application that turns the surface around a MacBook into an input device. Instead of requiring a dedicated touch-sensitive surface, it listens to the acoustic characteristics of taps through the built-in microphone. You calibrate four areas around your laptop, assign actions to them, and Holo tries to determine which area you tapped. For example, tapping one part of the desk could open an application, while tapping another could execute a keyboard shortcut.What interested me was how little hardware the concept required. Your desk isn’t connected to your computer. There are no sensors attached to it. The software is simply trying to understand where a sound came from. Of course, acoustic classification isn’t perfect. Different desk materials, microphone positions, and background noises affect the results. But the idea itself was enough to get me thinking. Since I also use Windows, I wanted to explore whether I could bring that same interaction to Windows 11 while making it useful beyond the original four-zone concept. I didn’t want to build a completely unrelated project and call it something similar. HoloWin is directly inspired by Holo, and I wanted to preserve that connection. At the same time, Windows gave me an opportunity to experiment with additional input methods and automation features. So, I built HoloWin
HoloWin is my open-source Windows 11 application that lets you control your PC using desk taps, hand gestures, keyboard shortcuts, and supported hardware sensors. I built it using WinUI 3 and .NET 10, with the goal of creating something that feels like a native Windows application rather than a quick cross-platform wrapper. The original desk-tap concept is still at the center of the project. You configure four zones around your workspace, calibrate them, and assign actions to each one.
For example, you could set up your desk like this:
- Front-left tap: Play or pause music
- Front-right tap: Open an application
- Rear-left tap: Switch browser tabs
- Rear-right tap: Run a custom macro
These are just examples. The actions are customizable. I also added a dashboard where you can trigger zones manually, which makes testing your configurations easier before enabling microphone detection. But I didn’t want HoloWin to be limited to a single way of interacting with a computer. That’s where the additional features come in.I Wanted More Than Just Desk Taps Hand gestures using your webcam. One of the biggest additions I worked on was camera-based gesture recognition.The idea is simple: if a microphone can recognize a desk tap, why not use a webcam to recognize a hand movement? HoloWin includes a Camera Studio that uses OpenCV and ONNX-based hand tracking to detect hand landmarks and recognize supported gestures.You can configure gestures to trigger actions, just like desk taps. For example, you could use a swipe to switch browser tabs or assign a hand pose to control media playback. There are also options for custom finger symbols. I like this feature because it offers another way to interact with Windows without constantly reaching for a mouse.
It isn’t perfect, though.
Camera-based recognition depends on lighting, hand visibility, and how consistently a gesture is performed. I’m treating it as an experimental feature rather than pretending it will work flawlessly in every setup.
Custom macros and automation
I also wanted the actions to be more flexible than simply opening an application. That’s why HoloWin includes a macro system. Instead of assigning only one command to a desk zone, you can combine multiple actions into a sequence.For example, a single tap could open a website, launch an application, and execute another configured command. HoloWin supports actions involving applications, files, websites, media controls, text, keyboard shortcuts, screenshots, and virtual desktops. The macro system also supports delays between actions. I included approval requirements for potentially sensitive operations, such as launching executables, running scripts, making HTTP requests, and taking screenshots. When you’re building an application that reacts to sounds or gestures, accidental triggers are something you have to take seriously. I don’t want a random noise on someone’s desk to execute a sensitive command without appropriate safeguards.
Hardware sensors and device-based triggers
Another area I wanted to explore was the hardware already available on Windows devices. Depending on the computer, Windows can expose information from ambient-light sensors, Bluetooth devices, presence sensors, and motion sensors. HoloWin can discover and monitor supported hardware, and some of those inputs can be connected to automation rules. For example, an ambient-light reading crossing a configured threshold could trigger an action. Bluetooth connection changes and supported presence events can also be used as triggers. Not all discovered sensors currently support automation. Some are available only for monitoring or inventory, and device compatibility varies considerably. I think that’s an important limitation to be clear about. The point is to experiment with what the hardware supports, not to claim that every Windows PC suddenly has capabilities it doesn’t actually possess.
Global hotkeys
Not everything needs a fancy gesture. Sometimes pressing a keyboard shortcut is still the easiest option. I added global hotkeys so users can trigger their configured zone actions from anywhere in Windows. This is also useful when microphone detection isn’t practical, or when you want to test an action without tapping your desk. HoloWin includes tray controls as well, so the application can remain accessible without keeping its main window open.
The Hard Part: Figuring Out Where Someone Tapped
The interface and automation features are interesting, but the acoustic detection is one of the more challenging parts of the project. Think about what the software actually receives. When you tap your desk, your microphone captures a short sound. It doesn’t receive a message saying, “That was the front-left corner.” HoloWin has to analyze the audio and make an educated classification based on calibration data. The process looks roughly like this:
Microphone → Tap detection → Feature extraction → Zone classification → Action execution
I’m using Windows Audio Session API (WASAPI) for audio capture. The application listens for short impact-like sounds and extracts features from the captured audio. During calibration, it collects examples from each desk zone. When a new tap is detected, HoloWin compares its acoustic features against the calibrated examples to estimate which zone produced it. If the result passes the configured checks, the application triggers the assigned action. The problem is that desks are noisy environments. Typing, moving objects, speaking, or accidentally hitting the desk can produce sounds that resemble taps. So the application needs to decide not only which zone a sound came from, but whether it should be treated as a valid tap at all. I’ve included confidence thresholds, rejection checks, cooldowns, and other controls to help reduce unwanted triggers. Still, I don’t want to oversell the accuracy. The behavior depends heavily on the microphone, desk material, room acoustics, and calibration quality. That’s why I consider HoloWin an experimental project. Calibration Matters More Than You Might Think. One thing that becomes obvious when working with acoustic sensing is that not every desk behaves the same way. A wooden desk, a glass table, and a desk covered by a large mat can produce very different sounds. Even moving your laptop or microphone can affect the results. So HoloWin includes a guided calibration workflow. You select your microphone, create a profile, and collect ten accepted taps for each of the four zones. That’s 40 accepted taps in total. You can also collect negative examples, such as typing or speech, to help the system distinguish actual desk taps from unwanted sounds. Once calibration is complete, HoloWin saves the profile locally. There’s also a separate evaluation workflow that uses fifteen new taps per zone. I wanted that distinction because good results on calibration data don’t necessarily mean the system will perform equally well during normal use. If you’re experimenting with HoloWin, I’d recommend evaluating it with fresh taps instead of relying only on the initial calibration. And if you change your desk setup significantly, recalibrating is usually a sensible idea.
Why I chose WinUI 3 and .NET 10
Since HoloWin is specifically for Windows 11, I wanted to use a native Windows technology stack. I built the interface with WinUI 3 and used .NET 10 for the application and supporting services. For the hardware-facing parts, I combined several technologies:
- OpenCV for webcam capture.
- ONNX models for hand detection and landmark tracking.
- Windows APIs for hotkeys, hardware discovery, and system integration.
- WASAPI for microphone capture.
I also separated the user interface, signal-processing components, and action-handling logic. This makes it easier to test individual parts of the application without having to run the entire interaction pipeline every time. For example, acoustic processing can be tested using generated signals, while the action system can be tested independently of physical desk taps. As someone who uses both macOS and Windows, I appreciate applications that feel appropriate for the operating system they’re built for. With HoloWin, I wanted to take advantage of the Windows ecosystem rather than simply reproduce the macOS interface.
Privacy Was Something I Had to Think About
An application that uses a microphone, webcam, and system automation needs to be clear about what it’s doing. HoloWin processes desk-tap audio locally and doesn’t require cloud processing or telemetry. Microphone listening and camera tracking are explicitly activated.By default, the application stores calibration features rather than raw audio recordings.Optional debug WAV recording is available for troubleshooting, but it’s disabled by default.Profiles and settings are also stored locally. Some configured actions, such as HTTP requests or opening websites, can access the network, so not every possible workflow is offline. I also included controls for sensitive actions because I think users should know when an automation can launch a program, run a script, or perform another operation with wider effects. It’s particularly important when the trigger might be a sound or gesture rather than a deliberate button click.
How you can try HoloWin
If you want to experiment with it yourself, the project is available on GitHub.
GitHub: https://github.com/rotsl/HoloWin
Website: https://rotsl.github.io/HoloWin/
HoloWin targets Windows 11 build 22621 or later. You can use the packaged Windows executable or portable ZIP, following the instructions in the repository. If you prefer building from source, you’ll need the .NET 10 SDK and the documented dependencies.
Clone the repository:
git clone https://github.com/rotsl/HoloWin.git
cd HoloWin
Restore dependencies and download the hand-tracking model:
dotnet restore HoloWin.sln
.\scripts\download-hand-model.ps1
Build the application:
dotnet build HoloWin.sln -c Debug – no-restore
Then run the development build:
.\run.ps1
Once HoloWin is running, I’d suggest starting with the desk-tap functionality. Select your microphone, create a calibration profile, and assign a simple action to one of the zones. Try something harmless first, like controlling music. After testing manual actions, enable microphone listening and see how well the system recognizes taps on your desk. You can explore Camera Studio separately if you’re interested in hand gestures.
What still needs work
I want to be clear that HoloWin isn’t a finished alternative to traditional input devices. It’s an experimental application, and there are limitations. Acoustic recognition can struggle with background noise or inconsistent taps. Hand tracking can be affected by poor lighting. Some hardware features only work on machines with the necessary sensors and drivers. There are also areas where broader compatibility testing is still needed. I’m not trying to suggest that tapping a desk is always faster or better than pressing a keyboard shortcut. In many cases, the keyboard will still win. What interests me is whether these interactions can be useful in specific situations. Maybe it’s pausing music while you’re doing something else. Maybe it’s triggering a repetitive workflow. Or maybe it’s simply having fun with a different way of controlling your PC. For me, the experimentation is a big part of the project.
What’s next for HoloWin?
There are plenty of directions I could explore further.I’d like to keep improving recognition reliability, reducing false triggers, and making the overall experience more consistent across different hardware setups. The camera and hardware-sensor features also leave room for more experimentation. But I’d rather focus on making existing interactions dependable than keep adding features that look impressive but aren’t practical. If you’ve worked with audio classification, computer vision, WinUI 3, or Windows hardware APIs, I’d be interested in feedback and contributions. The project is open source, and I think it can benefit from testing on more desks, microphones, webcams, and Windows devices than I can cover myself.
Thanks to JustinGamer191 for creating Holo and sharing the original idea as an open-source project.
Top comments (0)