This is a submission for the Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine
What I Built
For this year's "Ghost in the Machine" theme, I wanted to explore the intersection of memory, digital permanence, and Día de los Muertos. But as I started designing a digital ofrenda (memorial altar) to house family histories, letters, and spoken anecdotes, I hit a moral wall: I refuse to upload my family’s grief, intimate memories, and ancestral stories to a corporate cloud API to be monetized or used as training data.
So, I built Ánimas: The Sovereign 3D Ofrenda.
Ánimas is a 100% offline, privacy-first interactive memorial. It transforms fragmented family archives into a highly atmospheric, 16-bit retro-pixel 3D sanctuary. Using a locally running Gemma model and an offline RAG (Retrieval-Augmented Generation) pipeline, users can ask questions about their family history. The "ghost in the machine" channels these archives, answering with strict, grounded evidence, while the 3D WebGL environment dynamically reacts to the emotional sentiment of the memory—shifting lighting, candle flicker, and swirling marigold petals.
Demo
Code
shrguhaneogi-pixel
/
-NIMAS
Ghost in the Machine
(Note: Because Ánimas is designed to run 100% locally on your machine for absolute privacy, this video demonstrates the local execution of the Next.js/FastAPI monorepo)
How I Built It
Building a performant 3D application that simultaneously runs an LLM and vector database on consumer hardware required ruthless optimization and strict agentic scaffolding.
The Intelligence Layer (Gemma + SQLite-vec):
I used Google AI Studio to meticulously engineer strict Chain-of-Thought (CoT) prompts. Small models (like Gemma 2B/9B) need a reasoning_scratchpad to prevent hallucinations. The backend is a Python FastAPI service utilizing sqlite-vec for local vector embeddings. When a user asks, "What was Abuela's recipe for pan de muerto?", the local Gemma model retrieves the exact archival chunk and outputs a flat JSON contract driving both the text answer and the 3D scene's emotional state.
The 16-Bit 3D Sanctuary (React Three Fiber):
To keep the GPU load minimal while running local AI inference, I applied a massive pixel-art WebGL overhaul. I set the WebGL renderer's pixel ratio to 0.32 with hardware nearest-neighbor upscaling (image-rendering: pixelated), cutting GPU fragment shading by 88%. The environment uses a curated 16-color Día de los Muertos palette with 4x4 Bayer dithering for procedural linocut ancestor portraits.
The ReactBits WebGL Dither Cursor:
Instead of DOM-heavy SVG filters, I wrote a custom Three.js GLSL Post-Processing pass inspired by ReactBits. As you move the mouse, a top-down pixelated candle cursor leaves a dissipating, dithered 4x4 Bayer noise trail in the framebuffer, interacting dynamically with the 3D altar lighting.
Multi-Agent Scaffolding (Google Antigravity & Copilot):
To transition from AI Studio prototypes to a production monorepo, I used Google Antigravity. I exported a strict agent_skills.yaml file defining 6 specialized agents (AI/RAG, 3D/Creative, Security, etc.) to scaffold the Next.js and FastAPI environments, while GitHub Copilot assisted in writing the complex Three.js InstancedMesh math for the swirling petal vortexes.
Why Does Open Innovation Matter?
Because grief is not training data.
If you build a digital memorial reliant on a closed, venture-backed API, you are putting a ticking clock on your family’s history. When that service pivots, raises prices, or shuts down, your digital ofrenda goes dark.
Open innovation—specifically open-weight models like Gemma—changes the paradigm. It allows Ánimas to operate under the "One Machine, One Archive" principle. The entire RAG pipeline, embedding model, and vector database run on an air-gapped laptop. Open innovation means this software will still boot up and function exactly as intended thirty years from now, honoring the true permanence and reverence of Día de los Muertos.
My Agent Session
I utilized Google Antigravity and DevRelay to enforce strict privacy protocols and zero-cloud boundaries across the codebase. You can view the agent reasoning trace where we locked down the local-only FastAPI loopback and built the CoT hallucination defenses
The Spirit of It
During testing, I loaded a synthetic archive containing a fictional, fragmented story about a family's migration. I asked the local terminal, "Why did Tío Héctor leave the old house?"
There was a brief pause as the local GPU hummed. The 3D UI didn't just spit out text. The scene's memory_tone JSON triggered a shift: the vibrant amber lighting dimmed to a soft, somber glow, the swirling petals settled onto the chevron stone floor, and the Dither Cursor left a slow, fading trail. The model pulled the exact quote from a digital diary entry and politely abstained from guessing the parts of the story that were missing. It felt incredibly intimate, respectful, and slightly eerie—a true ghost in the machine, recalling a memory exactly as it was written, safe inside my own hardware.
Prize Categories
Gemma: For powering the local-first, zero-hallucination RAG and Sovereign Sentiment Engine using Gemma 2B/9B.
GitHub Copilot: For assisting in the procedural Three.js GLSL shader development and WebGL optimization.
Entire: (If applicable for the agent session trace embedding).
Top comments (0)