Magnus AI: a real-time voice assistant that controls my Windows PC

Riddhi Das — Mon, 05 Oct 2026 15:12:24 +0000

What I Built

Magnus AI is a voice-first desktop assistant for Windows. You talk to it, and it can:

Open apps and bring windows to the front
Set volume and brightness
Run one-phrase routines like "Work Mode", "Study Mode" and "Night Mode"
Search the web, play YouTube, and set reminders
Read your clipboard
Look at your screen or webcam and help debug code

Demo

https://drive.google.com/file/d/137xF8eviOjMR5DFXnfyZKzUN_IJyS8Br/view?usp=sharing

Code

https://github.com/RIDDHIDEV-OPS/Magnus-AI

How It Works

Voice loop:microphone audio is resampled to 16 kHz mono and streamed over a WebSocket to the Gemini Live API. Replies return as 24 kHz audio, with about 250 ms turn detection and barge-in so you can interrupt.
Self-describing tools:each file in actions/ exposes one typed function, and its docstrings become function declarations at startup. A new skill is one new file.
Windows control:Win32 APIs, psutil and PyAutoGUI.
Interface:a PyQt6 HUD with an animated orb, waveforms and live CPU/RAM/GPU telemetry.
Memory:preferences are stored in a local JSON file, git-ignored.
Resilience:a fallback ladder across Gemini models when quota is hit.

What's Next

Magnus currently relies on Google's hosted Gemini model. My next step is an offline mode built on open-weight models (Gemma via Ollama plus faster-whisper), so voice and screen data never leave the machine.

I built Magnus AI, a voice assistant that helps my friend Vashkar plan meals around his allergies

Riddhi Das — Sat, 03 Oct 2026 11:25:35 +0000

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend

What I Built

My friend Vashkar Ghosh has to plan his meals around his allergies. That means reading labels, double-checking ingredients and searching for safe options, again and again.

So I built him Magnus AI, a voice-first assistant for Windows. Instead of typing and searching, he can just say what he needs. He tells Magnus what he can't eat, and Magnus suggests meals that avoid those ingredients and keeps his preferences saved for next time.

Magnus does more than meals, because a helper that lives on his PC should handle the small daily things too. He can:

Open and switch between apps by voice
Set volume and brightness
Run one-phrase routines like "Work Mode", "Study Mode" and "Night Mode"
Search the web and play YouTube
Set reminders and read his clipboard
Show Magnus his screen or webcam and ask for help

Demo

https://drive.google.com/file/d/137xF8eviOjMR5DFXnfyZKzUN_IJyS8Br/view?usp=sharing

Code

https://github.com/RIDDHIDEV-OPS/Magnus-AI

How I Built It

Real-time voice loop. Microphone audio is resampled to 16 kHz mono and streamed over a WebSocket to the Gemini Live API. Replies come back as 24 kHz audio. There's about 250 ms of turn detection, and barge-in lets Vashkar interrupt mid-sentence the way he would with a person.

Self-describing tools. Every file in actions/ exposes one typed function. At startup, Magnus turns the docstrings into function declarations, so adding a new skill means dropping in a single file.

Windows control. Win32 APIs (SetForegroundWindow, AttachThreadInput), psutil and PyAutoGUI let Magnus find, launch and bring windows to the front.

Interface. A PyQt6 HUD with an animated orb, live audio waveforms, and CPU/RAM/GPU telemetry served from a small local FastAPI dashboard.

Local memory. Preferences and learned context are stored in a local JSON file with thread-safe atomic writes, and it's git-ignored so personal data never reaches the repo.

Resilience. If an API quota error hits, Magnus steps down a fallback ladder of Gemini models instead of going silent.

Why Does Open Innovation Matter?

I want to be upfront: the model at the heart of Magnus is Google's Gemini Live API, which is not open-weight. What is open is the code (MIT-licensed), the Python ecosystem it's built on (PyQt6, psutil, PyAutoGUI, FastAPI), and the Open-Meteo data behind its weather tool.

Building this for a friend taught me what that trade-off costs. Meal and allergy information is personal, and with a hosted model, his voice and screen go to a server he doesn't control. Open-weight models would fix that by running on his own laptop, working offline, costing nothing per use, and letting me tune the assistant to him. My next step is an offline mode built on an open-weight model so his data never leaves his machine.

DEV Community: Riddhi Das

Magnus AI: a real-time voice assistant that controls my Windows PC

What I Built

Demo

Code

How It Works

What's Next

I built Magnus AI, a voice assistant that helps my friend Vashkar plan meals around his allergies

What I Built

Demo

Code

How I Built It

Why Does Open Innovation Matter?