Hey everyone.
I don't know if you feel the same way when coding, but while current AI coding assistants are incredibly useful, they sometimes feel a bit clunky to use.
You either have to constantly shift your focus back and forth between your code and the IDE sidebar, or context-switch to a browser to ask ChatGPT or Claude when you hit a general problem outside your specific project.
As a minimalist (read: lazy) developer, I kept thinking: How cool would it be if this AI could materialize on my screen like those old-school desktop pets (like Shimeji or BonziBuddy), wandering around and always on call?
Driven by this random thought over the holidays, I built an open-source desktop pet app β CodeWalkers.
πΆ What exactly is this?
Simply put, it's a cross-platform transparent desktop app built with Tauri V2.
It spawns a few pixel-art characters on your screen. These little guys don't just aimlessly wander over your code editor or browserβmore importantly, they come with an "AI brain".
When you run into a coding issue, just click on a character, and a terminal panel pops up right there in place.
Currently, I've wired them up to real backend brains: GitHub Copilot CLI and Gemini CLI. You can directly ask them how to write a regex, configure Nginx, or even explain a piece of code right in the popup panel.
Even better, while it's "thinking" (calling the LLM API), a classic RPG-style thinking bubble pops up over its head. This turns the normally boring CLI loading spinners into a visual, interactive pet experience.
π οΈ How does it work? (Let's talk tech)
To bring this slightly goofy but cool idea to life, I used the Tauri v2 + React + TypeScript stack.
Even though it just looks like a little guy walking around the screen, I actually hit quite a few massive roadblocks during development. If you're interested in building desktop apps with Tauri, here are some pitfalls to avoid:
1. Phantom Clicks: The Transparent Window Passthrough Problem
To build a desktop pet, the first step is to make the app window globally transparent (transparent: true) and remove the OS window shadows and borders.
But this introduces a paradox: How do you make the mouse click the character, without blocking clicks to the code editor behind it?
Tauri provides the set_ignore_cursor_events API to control mouse passthrough. However, on macOS, if you have a transparent window, clicking a fully transparent pixel (alpha=0) automatically passes the click through to the desktop app underneath.
My workaround: I combined React's requestAnimationFrame with pixel-level hit-testing, and applied an extremely faint background color (rgba(255, 255, 255, 0.01)) to the transparent wrapper. This essentially tricks the macOS hit-testing system, achieving perfect precision without messing up the UI.
2. Stuffing a Real CLI into a Pet's Brain: PTY & Environment Variables
I wanted the pet to actually do work, not just be a dumb chatbox. So, I hijacked std::process::Command in the Rust backend.
To get a highly interactive CLI tool like Copilot running, I had to handle complex separation of standard output (stdout) and standard error (stderr). Through stream parsing, I intercepted all the ANSI escape codes and progress spinner characters (like β ) spat out by the CLI, converting them in real-time into the thinking bubble UI above the character.
Additionally, to prevent environment variable pollution, I built a dynamic injection mechanism using a local .codewalkers.env file, allowing you to securely feed it API tokens locally.
3. High FPS Animation Without Lag
Many overlay apps built with Electron or Tauri tend to hog the CPU the moment animations are involved.
In this project, I avoided putting the character's coordinate positions (x, y) into React State (which would trigger insane amounts of re-rendering). Instead, I manipulated the DOM's transform property directly via ref, coupled with requestAnimationFrame. This brute-forced a web animation to run with native 60 FPS smoothness.
π¨ Customization and Skins
Knowing everyone has different tastes in virtual pets, CodeWalkers fully supports custom skins.
You just need to drop your favorite GIF into the designated directory, tweak a few Sprite configuration parameters, and you can raise Hatsune Miku, Mario, or your own custom pixel art right on your desktop.
The project has already achieved a complete loop: from drag-and-drop movement and click interactions to terminal conversations.
π Open Source & Contributing
CodeWalkers is now completely open-source. The codebase is clean, free of bloated dependencies, and I've already set up CI/CD and decent test coverage.
GitHub Repo: https://github.com/you-want/CodeWalkers
If you:
- Think this is a fun idea
- Want your own personal AI desktop assistant
- Want to learn real-world Tauri V2 development (especially transparent windows, multi-process IPC, and PTY interactions)
Feel free to drop a Star to support the project! π
I also highly welcome PRs! Whether it's adding new character skins, or integrating more LLM CLIs (like Claude, Ollama, or even a locally running DeepSeek)βI'm all for it.
Let's say goodbye to boring terminal boxes and add a little cyber-vitality to our screens!
Top comments (1)
It's fascinating how the most successful AI projects often incorporate elements of fun or engagement, like your "Desktop Pet" Copilot. In our experience with enterprise teams, AI agents that blend utility with entertainment not only boost productivity but also enhance user adoption. This approach can transform mundane tasks into interactive experiences, making the AI feel more like a partner than a tool. - Ali Muwwakkil (ali-muwwakkil on LinkedIn)