This is a submission for the Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine
What I Built
Ofrenda is an offline oral-history keeper. You record one family story — a voice memo is enough — and it becomes a printed keepsake: a transcript you can correct, the one line worth keeping, a short remembrance, and a card you can print and put on the wall.
Two open-weight models do the listening and the writing, both inside the browser tab:
- Whisper tiny.en turns the recording into text.
- Gemma 3 1B writes a title and a three-sentence remembrance.
Everything else is code: choosing the quote, enforcing the word limits, rejecting clichés and invented dialogue, laying out the card, painting it on a canvas, exporting the PNG, keeping a shelf in localStorage.
I read "Ghost in the Machine" as the voice inside a recording, and DÃa de los Muertos as the night you are supposed to remember out loud. The two fit together too well to ignore. I did not want to build another chatbot with a skull emoji; I wanted to build the card you put on the altar.
Demo
Live: https://4-thkind.github.io/ofrenda/
Code: https://github.com/4-thkind/ofrenda
The Use the sample story button loads a 33-second recording, so you can try the whole flow without a microphone: transcribe → correct a word → pick the line → write the card → download the PNG.
The first run downloads the models (about 40 MB for Whisper, 570 MB for Gemma) and the browser caches them. After that the app works offline. Chrome or Edge with WebGPU is fastest; without it, the app falls back to WASM on the CPU.
4-thkind
/
ofrenda
An offline oral-history keeper: record a family story, transcribe and remember it locally with open-weight models.
Ofrenda
A family story, kept offline.
Ofrenda is an offline oral-history keeper. Record one family story — a voice memo is enough — and it becomes a printed keepsake: a transcript you can correct, a line worth keeping, a short remembrance, and a card you can put on the wall.
Two open-weight models do the listening and the writing. Everything else — the quote selection, the word limits, the clichés, the layout, the fallbacks — is ordinary code with tests.
Nothing is uploaded. Not the audio, not the words, not the name.
Why
DÃa de los Muertos is a night for remembering out loud. "Ghost in the Machine" is the same idea from the other side: the ghost is the voice inside the recording.
The obvious way to build this is to send the audio to a transcription API and the transcript to a chat API. That means a grandmother's…
How I Built It
No framework, no bundler, no build step. The app is ES modules, one canvas, and a stylesheet. The models run through Transformers.js and ONNX Runtime Web.
The design rule that shaped everything: the model writes; the code decides what is true.
A 1B model is good at prose and bad at being trusted. So the parts a family would care about are not generated:
| Decision | Made by |
|---|---|
| Which sentence becomes the quote | scoring in text.js — first person, distinct sensory details, comfortable length, penalty for funeral language |
| Whether the remembrance is acceptable | guards in remembrance.js
|
| What the card says when the model fails | a deterministic fallback built from the facts |
| Where every line sits |
card-layout.js, tested without a canvas |
Gemma's output is accepted only if it is 18–95 words, 2–4 sentences, ends with punctuation, contains no markdown, no greeting-card language ("loving memory", "rest in peace"), no machine voice ("circuits", "as an AI"), and no quotation marks — because the model does not know what anyone actually said, and putting words in a grandmother's mouth is the one thing a memory keeper cannot do.
What broke, and what fixed it
Every one of these was found by running the app, not by reading it:
-
English-only Whisper rejects the language flag.
onnx-community/whisper-tiny.enthrows if you passlanguageortask. Stop passing them. - The 1B echoed the prompt. Given a raw string, Gemma continued the instructions and fell into markdown fences. Given a chat template — system and user messages — it wrote three clean sentences. Same model, same words, different shape.
-
The 1B invented dialogue. One remembrance included
She'd simply say, "Listen closely"— a line that appears nowhere in the story. The quote guard exists because of that sentence. - The 1B wrote about its own circuits. "The scent of warm bread fills my circuits with memories." Now rejected by the same bouncer.
-
q4f16reloaded the tab on first load. Twice. The same model atq4loaded in 89 seconds and generated normally. - Two models, one GPU. Holding Whisper and Gemma at once was more memory than the tab was given. The listener is now disposed before the writer wakes; the weights stay cached, so it wakes again in seconds.
- The card was hollow. The first layout left the bottom third of the paper blank. The quote and the remembrance are now one composition, centred between the divider and a drawn marigold.
Measured on the demo machine: 33.2 s of audio → transcript in 42 s on WebGPU; one title plus one remembrance in about 37 s; the card renders in under 10 ms. 44 assertions in four test files, no test framework.
Why Does Open Innovation Matter?
A grandmother's voice is the most personal data a family owns. The ordinary way to build this app is to send the audio to a transcription API and the transcript to a chat API. That means the voice leaves the laptop, lands in someone's logs, and stays there — for a memory you intend to keep for decades.
Open weights change the bargain. Download the model once and the machine in front of you can do the listening. The audio is decoded in the tab with the Web Audio API, the models run on the device, and the shelf is localStorage. The only network requests are the model files and the library itself.
The second thing open weights gave me is the right to refuse. Because the model is small, local, and mine, I can wrap it in hard rules — no quotes, no clichés, no machine voice — and fall back to prose built from facts when it fails. With a closed API you get a text blob and a policy page. With open weights you get a component you can constrain, test, and replace.
And the keepsake is a PNG, not a subscription. The card survives the app. That is the whole point of an ofrenda: the thing you keep outlives the thing that made it.
My Agent Session
I built this with an AI coding agent (OpenCode) in one session, and I want to be specific about the split, because "AI built it" is not a useful sentence.
I set the constraints and the taste: no paid APIs, no uploads, a keepsake rather than a demo page, the theme, and the rule that the model may not invent dialogue. The agent wrote the modules and the tests, ran them, and drove the browser to verify the real flow — including finding the seven bugs above by running the app. I made the calls that could not be automated: which model, which guard, what the card should feel like.
The build is 25 commits, grouped by concern, so the history reads like the work: pure rules and their tests first, then the model wrappers, then the painter, then the controller.
The Spirit of It
The first time the whole flow ran, the code picked this line from the transcript:
But every time I smell bread, I hear her humming in the kitchen.
and Gemma put this above it:
Sunday's Warmth
Amma · 1938 – 2019 · grandmother · Kochi
The story is synthetic — I wrote it and generated the audio offline — but the shape was right. The line the scoring chose was the line a family would choose. The card looked like something you would actually print and put next to a candle.
Then, a few runs later, the model wrote "fills my circuits with memories," and the guard threw it out. That contrast is the project in miniature: the ghost can talk, but it does not get to lie.
Prize Categories
Best Use of Gemma — Gemma 3 1B (q4, ONNX) runs locally in the tab through Transformers.js and writes both pieces of prose on the card. Every output passes through the guards before it is allowed onto the paper, and the deterministic fallback keeps the card honest when the model is not.
Whisper is MIT; Gemma 3 is open-weight under the Gemma Terms of Use; Transformers.js is Apache-2.0. The demo audio is synthetic speech generated offline. No real family recording is in the repo.
Top comments (0)