DEAD AIR: I Built a 3D Mystery Experience Powered by Local Gemma 4 AI 👻📻
What if you discovered a radio transmission that nobody should be able to hear?
What if the only way to understand it was to enter an abandoned observatory, discover its secrets and communicate with the mysterious signal itself?
For Hacktoberfest Week 4, I built DEAD AIR — The Last Frequency, a first-person 3D mystery experience powered by Gemma 4 running locally in the browser.
This is my interpretation of the theme Ghost in the Machine.
1. What I Built
DEAD AIR takes place inside an abandoned radio observatory.
At 03:17 AM, an impossible transmission begins broadcasting.
The player must explore the observatory, investigate an incident log, discover hidden signal fragments, tune a radio transmitter and reconstruct the final transmission.
But there is something unusual about this experience.
The mysterious entity communicating through the radio is powered by a real open-weight language model running locally in the browser.
No backend. No API key. No remote inference service.
Key Features
- First-person 3D exploration with WASD and mouse controls.
- Procedurally generated observatory environment.
- Interactive incident log, signal monitor and radio transmitter.
- Frequency-tuning puzzle with dynamic audio and visual feedback.
- Evidence collection and a six-step investigation progression.
- Local Gemma 4 AI assistant called THE SIGNAL.
- Evidence-grounded AI responses with citations.
- Deterministic mystery reconstruction and validation.
- Scripted preview when WebGPU or model loading is unavailable.
2. Demo
The project has been built and tested locally.
A video walkthrough and hosted browser demo are not included in this initial submission. The repository includes screenshots, including evidence of the successful real Gemma inference run, and instructions for running the experience locally.
Real Gemma 4 inference:
More screenshots and setup instructions are available in the repository.
3. Code
GitHub Repository:
https://github.com/TechGenDM/Hacktoberfest-Week-4-2026
Branch: main
Commit: 19aa676
4. How I Built It
The application uses:
| Technology | Purpose |
|---|---|
| Vite 8 | Development and production builds |
| TypeScript 7 | Application logic |
| Three.js 0.187 | 3D environment |
| Transformers.js 4.3.2 | Browser-based model inference |
| Gemma 4 E2B | Local language model |
| WebGPU | Hardware-accelerated inference |
| Web Workers | Background model processing |
| Web Audio API | Procedural ambient audio |
The environment is constructed using procedural geometry and textures rather than relying on downloaded 3D assets.
The interesting engineering decision
I deliberately separated the mystery-solving system from the language model.
The game itself controls the evidence, puzzle progression and final answer.
Gemma does not decide what happened. It investigates the evidence available to the player and helps interpret it.
Every response passes through a deterministic validation layer.
The validator checks:
- Whether sentences include evidence citations.
- Whether the cited evidence has actually been discovered.
- Whether numerical claims appear in their cited evidence.
- Whether named entities appear in the discovered evidence.
Responses receive one of four verdicts:
VERIFIED · PARTIAL · REJECTED · NO SIGNAL
Unsupported sentences can be struck out with an explanation.
This is not a guarantee of semantic correctness, but it provides a deterministic first layer of protection against unsupported AI responses.
Real Gemma 4 inference results
I tested actual inference in headless Chrome on macOS using WebGPU and the q4f16 model path.
| Measurement | Result |
|---|---|
| Initial load, including download and shader warm-up | 279 seconds |
| Model download | Approximately 3.38 GB |
| Generation speed | 28–33 tokens/second |
| Response time | 1–2.5 seconds |
I tested several questions.
Question: Where is the carrier coming from?
Result: Three evidence-cited sentences. All were marked VERIFIED.
Question: Who sent the signal from Mars?
Result: NO SIGNAL ON THAT BAND.
Question: What frequency should I tune TX-2 to?
Result: The response repeated a cited sentence. It was accurate but not sufficiently helpful. This is a known limitation.
Testing
The following checks passed:
- TypeScript compilation.
- Production build.
- Complete automated game playthrough.
- All eight evidence items discovered.
- Incorrect reconstruction order rejected.
- Correct reconstruction accepted.
- Ending sequence completed.
- Scripted fallback verified.
- No runtime errors during the automated playthrough.
Browser compatibility beyond Chrome on macOS and low-memory GPU configurations remains untested.
5. Why Does Open Innovation Matter?
I wanted to explore what happens when an AI-powered interactive experience does not depend on a centralized AI service.
DEAD AIR uses an open-weight Gemma model with browser-based inference.
The model runs on the user's hardware rather than sending every conversation to a remote inference API.
This offers several advantages:
- Conversations can remain on the user's device.
- No application backend or AI API key is required.
- The model can be downloaded and used locally.
- The system does not depend on a proprietary inference endpoint.
- The application combines an open-weight model with transparent, application-level validation.
One important distinction: the initial model download requires an internet connection. After the weights are cached, inference can run locally without a remote AI service.
For me, open innovation is not just about accessing a model. It is about having the freedom to run, inspect, adapt and build around the technology.
6. What I Learned
The most interesting part of this project was not getting a model to generate dialogue.
It was learning how to build an experience where the model has boundaries.
A language model can produce convincing answers even when it does not have enough information. In a mystery game, that can completely destroy the experience.
Separating deterministic game logic from AI interpretation made the system more predictable.
I also learned how important browser GPU support, model quantization, Web Workers and initial model-loading performance are when building local AI applications.
7. Prize Categories
Gemma — Best Use of Gemma
The project uses Gemma 4 E2B as its central conversational AI model, running locally through Transformers.js and WebGPU.
Final Thoughts
DEAD AIR started as an idea: what if we could build an entire interactive mystery world where the intelligence behind the experience lives inside the browser?
It became an experiment in combining 3D environments, open-weight AI, local inference and deterministic engineering.
There is still plenty to improve. This is an initial prototype, not a finished commercial game.
But it works, the real Gemma inference has been verified, and the mystery can be completed.
Sometimes the most interesting thing is not the signal itself.
It's what happens when somebody finally answers it.
Built for the Hacktoberfest 2026 DEV Challenge — Week 4: Ghost in the Machine.
Top comments (0)