Séance: ask your grandmother's stories a question, and hear the answer in her voice (Whisper + Gemma 4, local)

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine

What I Built

Séance is a private archive for the stories your family tells, and a way to ask those stories questions.

My grandmother sends me voice notes in Malayalam. Séance takes those recordings, transcribes and translates them with an open Whisper model on my laptop, and has Gemma 4, also running on my laptop, turn each one into a memory card: the people and places she mentions, the most vivid moments in her own words, and questions I could ask her next time.

Then I can ask the archive a question, like "Who was there when they came to Kerala?", and get an answer with her exact words, the second she said them, and her voice playing that moment back.

I read the theme two ways. "Ghost in the Machine" made me think of the voices of the people we love, sitting in old voice notes on our phones. Día de los Muertos is about keeping people present by remembering them out loud. So Séance also builds an ofrenda: one offline web page with each person's words, places and stories, where every quote has a ▶ button that plays their voice.

The one thing it will never do is put words in someone's mouth. There is no "chat with Grandma" bot. Every quote is checked against the transcript, and anything that can't be found there is thrown away.

Demo

Code

clyv / seance

🕯️ Séance

Ask your family's stories a question. Hear the answer in their own voice.

Séance is a private, offline archive for the stories your elders tell. Record your grandmother talking about the house she grew up in. An open Whisper model transcribes it Gemma (running on your own computer through Ollama) writes a memory card, and later you can ask the archive things like "What did Grandma say about the monsoon?" The answer comes back with her exact words, the minute she said them, and a button that plays that moment.

It also builds an ofrenda, a single web page in the spirit of Día de los Muertos, that gathers each person's words, places, years and the questions you still want to ask them.

🎬 Demo: add your demo video here

Built for the DEV Hacktoberfest 2026 Open-Source AI Challenge, Week 4: Ghost in the Machine.

---

The

…

View on GitHub

How I Built It

The flow is: voice note → Whisper (timestamped transcript + English translation) → Gemma 4 (memory card) → quote check against the transcript → archive, search and the ofrenda page. Questions go through search → Gemma → the same quote check.

The open stack, all running locally:

• faster-whisper with OpenAI's open Whisper weights. It keeps a timestamp for every segment, which is what makes "play the moment she said it" possible. For Malayalam, Whisper's translate mode adds an English version, and the original words stay next to each quote.
• Gemma 4 (gemma4:e4b) through Ollama writes the memory cards and answers questions. Ollama's structured outputs force every reply into a JSON schema, so parsing never depends on the model behaving.
• Keyword search (BM25), plus EmbeddingGemma for semantic search when it's installed, combined with rank fusion.
• Plain Python for everything else, with a small standard-library Ollama client.

It reads the recordings families actually have: WhatsApp voice notes (.ogg/.opus), call recordings (.m4a, .amr, .3gp), and family videos. Formats a browser can't play get a small WAV copy so the ofrenda's ▶ buttons always work.

Three decisions shaped it.

1. Gemma chooses; code quotes. When Gemma picks a quote for a memory card, Séance finds those words in the transcript with an in-order fuzzy match and shows the transcript's wording with its timestamp. If Gemma tidied the grammar, the original wording wins. If the words aren't there at all, the quote is discarded. Answers to questions go through the same check: Gemma has to cite a passage by quoting it, and an answer with no verifiable quote is flagged instead of shown as fact.

2. Search in two vocabularies. People don't ask questions in the words the speaker used. So each passage is also tagged with the memory card's headings for the moments inside it, which are in Gemma's words. A question about "the flood" can then find a passage where the speaker only said "the water came right up to the steps".

3. Long recordings in parts. A real conversation runs past a small model's comfortable context, so long transcripts are carded in parts, merged in code, and given one title and summary at the end.

What it actually took on my laptop (CPU only). My grandmother's first voice note was 1 min 19s of Malayalam. Whisper transcribed it in 123 seconds, translated it in 17 seconds, and Gemma wrote the memory card in 26 seconds.

What a real Windows laptop taught me. I built my Week 3 project on the same stack, and its first real run failed in two ways the tests didn't catch: the newest PyAV dropped an argument faster-whisper uses, and Whisper's device = "auto" tried to load CUDA libraries that weren't installed. Séance pins PyAV, defaults to CPU, and seance doctor checks for both and prints the one-line fix. The repo has 56 tests that replace Whisper and Gemma with fakes, including a fake Ollama server, and they pass on Python 3.10 and 3.13.

Why Does Open Innovation Matter?

These are some of the most private recordings a family has: an elderly relative's memories, sometimes their health, sometimes the only recordings of their voice. I don't want to upload them to a service to get a summary.

With open models running locally:

• Recordings and transcripts stay on my machine. The only network use is the one-time download of the model weights.
• It will still work in twenty years. A family archive has to outlive the tools that made it. If a hosted API shuts down or changes its terms, an archive built on it breaks. Here the archive is plain JSON, Markdown and audio, and the models are files on disk.
• It can be improved for my family's language. Standard Whisper small handled my grandmother's voice, but the English translation was rough: it heard "Madras" as "Madrasa" and turned a few lines into nonsense. Because the stack is open, the fix is a one-line config change to a Malayalam model fine-tuned by the open-source community, not waiting for a vendor to care about Malayalam.
• I can inspect every rule. The prompts, the quote-matching threshold and the search weights are all in the repo, so if it misquotes someone, I can see why and fix it.

Where closed models are better: a frontier API would summarise a long, chaotic conversation more reliably than a small model on a laptop, and would probably translate Malayalam better than Whisper small. For a family archive, privacy and permanence matter more to me, and the quote check means the weaker model can't put words in anyone's mouth.

Prize Categories

• Best Use of Gemma. Gemma 4 writes every memory card and answers every question, running locally through Ollama with structured outputs. EmbeddingGemma adds semantic search. Gemma is the core of the project, not an add-on.

Thanks for reading. Ask your elders about the house they grew up in. 🕯️