What I Built
Shadow Puppet Theater: Haunted Edition is a hand-gesture puzzle game where your shadow is the only way to talk to a ghost.
A ghost haunts a candlelit wall and speaks in riddles: “Something heavy sleeps beneath the floorboards. Show me its shadow.” You answer with your hand. A fist casts a bear, two fingers cast a rabbit, and an open palm casts a bird. Get it right and the paper-cut shadow appears and the ghost sets that spirit free. Get it wrong and the candle shivers, the wall shakes, and the ghost whispers a hint. Free all three and the story ends with a nod to Día de los Muertos.
I read the theme two ways. Ghost in the machine is literal: the ghost is a small language model living on your laptop. The Día de los Muertos side is the idea that the old spirits only want to be remembered, which is why the ending is a candle and not a jump scare. I also grew up with shadow puppets on a wall, and I wanted to see whether an AI could play along with that.
Demo
Playable preview (hand shapes are picked with buttons or keys 1, 2, 3): https://claude.ai/artifact/1sfw6yCzW63FmctK6c9Ehf
[Add a short video of the local build with the webcam and live ghost narration]
Code
[GitHub repo link, e.g. https://github.com/ITZMElk/shadow-puppet-theater]
How I Built It
Hand tracking: MediaPipe Hands reads 21 landmarks from the webcam. A few simple rules on finger extension classify the pose as fist, two fingers, or open hand. [confirm after testing]
The ghost’s voice: a local model through Ollama ([qwen2.5:3b]) writes each line. The game sends it the current riddle, the shadow you just cast, and whether you were right or wrong, and the ghost replies in a few short, eerie sentences. I used the same model in my earlier Sloth Study Digest entry, so I knew it fits on a student laptop.
The stage: the wall, candle flicker, and paper-cut animals are plain SVG and CSS, with the candle light driven by a CSS variable that flickers on wrong answers.
Game logic: three riddles in a fixed order. The model only narrates, so the rules never depend on what it says, and the puzzles can’t break.
[Add one real detail here, such as a model you tried and swapped, a prompt you had to tighten, or your measured response time]
Why Does Open Innovation Matter?
It works with the lights off. The ghost runs on Ollama and the hands are tracked by MediaPipe, so there are no API calls and no internet is needed. A haunted house with no Wi-Fi is on theme.
Your camera never leaves the room. The game watches your hands through a webcam, and with a closed API that would mean streaming video to someone else’s server. Here the frames are processed locally and nothing is uploaded.
I can change the ghost. Because I own the model, I can swap it for a bigger or spookier one, or rewrite the system prompt to make the ghost an old programming language or a grandmother’s spirit, without asking anyone’s permission.
It costs nothing to run, and it can’t be shut down or repriced later. A game that talks to a hosted API stops working when the service does.
Top comments (0)