DEV Community

Cover image for Great Sage (Raphael AI): The Universal, 100% Private Desktop Voice AI Companion for Everyone!
Adpirs
Adpirs

Posted on Edited on

Great Sage (Raphael AI): The Universal, 100% Private Desktop Voice AI Companion for Everyone!

#devchallenge #hf26challenge #ai #opensource

Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle Submission 💸

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle

What I Built

Great Sage (Raphael AI) is an open-source, autonomous desktop voice AI companion and personal operating copilot built for anyone and everyone who uses a computer—whether you are a student, gamer, content creator, developer, or daily PC user.

Inspired by the omniscient Great Sage / Raphael (Lord of Wisdom) from That Time I Got Reincarnated as a Slime and the engineering brilliance of JARVIS, Great Sage turns your personal computer into an intuitive, voice-controlled command center that runs 100% locally on your machine with zero subscription fees.

Who Is This For & Who Would Pay For It?

Most modern AI platforms force users into perpetual $20–$100+/month subscriptions and upload personal screen activity to corporate cloud servers. Great Sage flips this paradigm on its head by delivering an Everyday Desktop Companion that anyone would pay for as a one-time product

  1. Everyday Multitaskers & PC Users: Anyone tired of juggling dozens of windows and endlessly clicking around. Just say "Play some chill music on YouTube" or "Turn the volume down", and Great Sage handles it hands-free without touching the mouse or keyboard.
  2. Gamers & Streamers: Players in full-screen games who want to switch tracks, mute audio, launch Discord, or look up a tip without Alt-Tabbing or crashing their game.
  3. Students & Learners: Anyone studying who wants a private, offline tutor and note-keeper that doesn't track their search history or distract them with web ads.
  4. Developers & Creators: Anyone handling confidential work, proprietary code, or personal documents who demands 100% offline privacy and zero monthly API bills.

For a one-time purchase of $15–$29 (or free for the open-source community), everyday users get their own loyal, offline AI companion for life on their laptop.

🌟 Complete System Capabilities:

1. Instant Holographic Magic-Circle HUD

  • Global Summon Hotkey: Press Ctrl + Shift + S (or Ctrl + Alt + S) anywhere on Windows, and the HUD instantly snaps to the center of your screen in milliseconds (<5ms) without flashing any Command Prompt windows.
  • Audio-Reactive Anime Interface: Translucent, floating borderless window featuring authentic rotating concentric magic-circle rune rings and real-time audio waveform visualizers.
  • Hands-Free Continuous Listening: Wake-word activated ("Daikenja", "Great Sage", "Raphael"), complete with instant voice barge-in interruption (speak while it talks to interrupt immediately).
  • Bilingual Japanese/English Output: Synthesizes spoken Japanese in Great Sage’s signature cadence (『告。』 / 『解。』) via high-fidelity neural audio, while rendering English translations on the display card.

2. Hands-Free Desktop & Application Control

  • Launch Any App: Seamlessly opens apps by voice (Spotify, Discord, VS Code, Chrome, Edge, Steam, VirtualBox, Terminal, Calculator, Notepad, OBS Studio, Blender, MS Word, Excel, PowerPoint, Paint).
  • Multi-Command Chaining: Understands compound directives in a single sentence (e.g. "Open VS Code and launch Spotify" or "Open Notepad and type my shopping list").
  • App Termination: Gracefully closes or terminates any background process or application on command.
  • Voice Dictation / Auto-Typing: Dictates text directly into target apps like Notepad or chat windows hands-free.

3. Media & Audio Entertainment Hub

  • Direct YouTube Video Autoplay: Scrapes YouTube video IDs in real time and automatically launches the exact top video with autoplay enabled.
  • Deep Spotify Control: Searches and plays songs, albums, artists, or playlists directly through the native Spotify protocol.
  • Hardware Volume & Media Keys: Adjusts master system volume ("Volume up", "Volume down", "Set volume to 40%", "Mute"), and controls playback (Play, Pause, Next Track, Previous Track).

4. Active Workspace & Context Awareness

  • "Help Me With What I'm Doing": When invoked, Great Sage inspects your active focused application (e.g. IDE, browser tab, or document) and reads your current clipboard context to give instant, tailored advice.
  • Instant Full-Screen Capture: Captures desktop screenshots on voice command and saves them directly to your storage.

5. Self-Improving Long-Term Memory (Knowledge Bank)

  • Automatic Preference Learning: Learns your daily habits, preferred programming languages, favorite music, study routines, and corrections after every conversation.
  • Persistent SQLite Brain: All memories are stored locally in data/raphael_memory.db and injected into future cognitive reasoning.
  • Knowledge Bank UI: Built-in visual memory inspection drawer where you can view, edit, search, and delete learned rules.

6. Power-User, Virtualization & Developer Tools

  • VirtualBox VM Automation: Starts virtual machines in GUI or headless mode ("Start Kali Linux VM"), halts VMs, and takes or restores snapshots by name.
  • Docker Management: Checks container status, starts, stops, or restarts Docker containers hands-free.
  • Git Voice Actions: Checks repository status, stages files, commits with voice messages, and pushes to remote branches.
  • Silent Shell Commands: Safely executes background terminal and PowerShell commands without pop-up terminal clutter.

7. Tensura "Appraisal" (Kantei / Kaiseki) & Error Diagnosis

  • Deep Code Appraisal: Analyzes files, functions, or architectures and reports security vulnerabilities, performance bottlenecks, and refactoring tips.
  • Stack Trace Diagnosis: Paste or dictate error traces, and Great Sage analyzes the exact failure point and provides working fixes.

8. Timers, Alarms & Tactical Presets

  • Live Countdown Timers: Set timers with custom labels ("Set a 15-minute study timer"); renders visual countdown pills directly inside the HUD.
  • Tactical Workspace Presets: Switch desktop modes in one command ("Focus Mode", "Gaming Mode", "Work Mode"), minimizing clutter and preparing your apps.

9. Hardware Telemetry & Autonomous Guardian

  • Live System Telemetry: Voice query CPU usage, RAM utilization, GPU temperature, disk space, network activity, and laptop battery health.
  • Autonomous Guardian Sentinel: An active background sentinel thread that auto-verifies local neural engines (Ollama) and heals services automatically if interrupted.

10. 100% Offline, Zero-Subscription Intelligence

  • Local Gemma 3 (4B) via Ollama: Runs entirely on your GPU VRAM with zero API costs, zero data sent to external servers, and unlimited daily usage without internet.
  • Optional Frontier Cloud Fallback: Integrated Google Gemini Flash-Lite router for when deep cloud reasoning is preferred.

Demo

Youtube: https://youtu.be/F6hp9alkW6U

Example Commands Anyone Can Use:

  • "Great Sage, open Spotify and turn the volume up."
  • "Play relaxing lofi on YouTube."
  • "Help me with what I'm doing." (Reads foreground window context and clipboard to assist immediately).
  • "Remember that I prefer dark mode and study late at night." (Persisted to memory).
  • "Mute audio and lock workstation."

Code

GitHub repo: https://github.com/SripadaError101/Great_Sage

How I Built It

  1. Local Neural Inference: I integrated Ollama running Google's open-weight Gemma 3 (4B). The model is pinned directly in GPU VRAM with a keep_alive daemon for instant <100ms response times.
  2. Hybrid Cloud Fallback: Integrated an optional router for Google's Gemini Flash-Lite when users want cloud frontier capabilities while traveling.
  3. Desktop Automation Harness: Engineered a Python FastAPI backend connected to native Win32 APIs via ctypes and subprocess, enabling instant desktop control without external bloat.
  4. Zero-Infobar Edge App HUD: Designed a custom lightweight Chromium app profile that sets permanent native microphone permissions and screen geometry, opening a clean, borderless floating window in milliseconds.

Why Does Open Innovation Matter?

Open innovation is the single reason a project like Great Sage can belong to everyone, rather than being locked behind big-tech paywalls:

  1. AI as an Asset, Not a Landlord: Cloud AI forces users into perpetual rent-seeking subscriptions ($240+/year for Copilot Pro or ChatGPT Plus). Open-weight models running on consumer hardware prove that powerful AI can be owned outright by everyday users.
  2. Uncompromising Personal Privacy: Everyone has personal data on their PC—family photos, banking tabs, personal journals, and private chats. Running open-weight models locally guarantees that personal life never leaves the user's hard drive.
  3. Inclusive & Offline: Big-tech AI stops working the second your Wi-Fi drops or your mobile hotspot runs out. Great Sage operates on airplanes, trains, during internet blackouts, or in remote regions.
  4. Zero Censorship or Vendor Depreciation: Users can swap between Gemma, Llama, Mistral, or specialized models with a single line in config.py without worrying about an API provider changing terms of service.

Top comments (0)