This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend
What I Built
I built Storyhouse for my grandmother.
She's in her eighties, and she's living with early Alzheimer's. The hardest part isn't only the memories she may lose — it's realizing how many stories she's carrying right now. The Sunday peach pies her mother baked. The dance where she met my grandfather. The old blue Beetle with the heater that never worked. Eighty years of stories, and almost none of them written down. And fewer people around to tell them to than there used to be.
I didn't want to build another chatbot.
I wanted to build somewhere her stories could live.
Storyhouse is a local-first, voice-first AI companion and family memory archive. It sits in the room, listens while she talks, responds warmly, and quietly turns the meaningful moments into a structured, family-owned Memory Vault — connecting people, places, recipes, and decades into something the whole family can keep forever.
It is deliberately not a few things: not an Alzheimer's treatment, not a medical device, not a replacement for family, and never a fake relative pretending to be her granddaughter. It's a companion and a memory-preservation system — and it always nudges her back toward real people.
The interface isn't a chat window. It's a house you walk into: a Living Room to talk, a Library of preserved stories in her own words, a Gallery, a Life Timeline, and a Vault she (and the family) fully own. There's a read-only Family View so relatives can wander the archive, and a printable storybook PDF so her words can live on paper too.
She spent 80 years collecting stories. We built somewhere for them to live.
Demo
The demo runs in Demo Mode with a fictional woman, "Margaret," so no real family memory is ever exposed. Her seeded life is built to show the system actually thinking: ask it about "Grandpa's car" and it retrieves the right memory; it links the blue Chevrolet to teaching their son to drive; it holds an unfinished story to gently return to later; and it keeps a deliberately conflicting date ("1957"… "actually, maybe 1958") as two recollections — because it is not allowed to quietly decide which one is true.
Code
🏠 Storyhouse
A home for the stories that make us who we are.
Storyhouse is a local-first, voice-first AI companion and family memory archive. It listens to an elderly loved one tell the stories of their life, remembers them, connects them across the decades, and preserves them in a structured, family-owned Memory Vault — all powered by an open-weight Gemma model running locally, so the most private memories a family has never leave the device.
She spent 80 years collecting stories. We built somewhere for them to live.
The person I built it for
I built Storyhouse for my grandmother, who is in her eighties and living with early Alzheimer's.
The hardest part isn't only the memories she may lose. It's realizing how many stories she is carrying right now — about her mother's Sunday peach pies, the dance where she met my grandfather, the old blue…
How I Built It
The intelligence is Google Gemma 3 (open-weight, 4B), running entirely locally via Ollama — no cloud, no API key, no bill. Gemma does the real work:
- the warm companion responses (streamed token-by-token, so it feels like talking, not waiting),
- memory extraction — turning a spoken passage into structured JSON (title, people, places, dates, emotions, themes),
- entity extraction and novelty reasoning,
- retrieval synthesis ("what did I tell you about Grandpa's car?").
Semantic memory search uses the open nomic-embed-text model, also local. Everything sits behind a provider abstraction, so the model is swappable and nothing is coupled to a vendor:
AIProvider
├── OllamaProvider (Gemma — the open-source core)
└── MockProvider (honest offline fallback: the app still opens, the Vault still works)
It's a real memory system, not a prompt stuffed with history. Her words → embedding → cosine search over the vault → only the relevant memories are assembled into Gemma's context. The whole archive is never dumped into the model.
And the honesty rails are in the data model, not just the vibes:
- Immutable originals — her exact words are stored once, never overwritten. A revised detail becomes a new recollection.
-
Provenance on everything — each fact is tagged
DIRECT(her words),APPROXIMATE,FAMILY_REPORTED,DERIVED(AI inference), orGENERATED(AI art). The UI shows it; the AI is told to say "you told me…", never "it is true that…". - Never invent — if nothing relevant is retrieved, it says "I don't have that one yet — I'd love to hear about it," instead of making something up. If she says "1944," it keeps the year; it won't fabricate a day.
- Consent — a story is kept only when offered and accepted. The microphone only listens when tapped.
For the whole family. A read-only Family View (passcode-unlocked) lets relatives browse the Library, Gallery, Timeline and Vault — but the Living Room, the microphone, and all editing stay private to her, and anything marked private is never shown. It's enforced on the server, not just hidden in the UI. It runs on the home device; for family far away, a Cloudflare Tunnel keeps the data on that device while making it privately reachable — no cloud database, ever. And the whole archive exports as a storybook PDF (her words, provenance, and the memory art embedded) or as JSON/Markdown — the family owns it, forever.
Memory art uses another open-weight model — Flux.1 schnell — hosted on Cloudflare Workers AI, which turns a scene she described ("an enormous orange tree, the light coming through all gold") into a labeled painting. It runs on Cloudflare's GPUs so it doesn't fight Gemma for the laptop's 6 GB. It's the one piece that calls out — disclosed in the app, and always labeled "AI-generated visual interpretation," never a real photograph.
Stack: React + Vite + Tailwind on the front, Node + Express on the back, and the Memory Vault in SQLite via Node's built-in node:sqlite (which, as a bonus, means zero native build step — a real gift on Windows without a C++ toolchain). A nice detail for small hardware: on a 6 GB laptop GPU, Gemma and the embedder were fighting for VRAM, so I pinned both resident with keep_alive and a boot warmup. Warm turns land in ~2 seconds.
Why Does Open Innovation Matter?
Because these are a dying woman's most intimate memories.
For this project, open-weight, local AI isn't a nice-to-have — it's the only ethically defensible architecture.
- 🔒 Private by construction. Her stories are processed on the device and stored in a local vault. Nothing is uploaded. A closed API would mean sending my grandmother's life to a company's servers.
- ✈️ Works offline. It runs on a laptop with no internet — and the roadmap puts it on her own phone, fully on-device, using Gemma 3n via Google AI Edge / MediaPipe.
- 💸 Free to run, forever. No per-token meter running while a grandchild keeps a grandparent's memory alive.
- 🔧 Ours to change. We own the weights. We can swap models or fine-tune on the family's own voice, because nothing is locked behind a vendor. Even the image model is open-weight (Flux.1) — the only thing we don't own is the GPU it briefly borrows.
- 🧭 Provable. Because I control the whole pipeline, I can show that her conversations never leave the device — not just promise it.
A closed API could have produced the conversation. It could not have produced the ownership. That difference is the entire point.
My Agent Session
I built this with Claude Code (and saved the session with DevRelay). Here's the real build — including the honest engineering saves: swapping a native SQLite dependency for Node's built-in node:sqlite when there was no C++ toolchain, pinning both models resident to stop VRAM thrashing on a 6 GB GPU, and catching the moment it fabricated a memory (a stale fallback embedding had broken retrieval) and hardening it so it can never invent her life.
For the Hacktoberfest 'Build for a Friend' challenge: build Storyhouse — a local-first, voice-first AI memory companion for my grandmother (80s, early Alzheimer's) to preserve her life stories. Open-source AI must be at the core. Can it run open AI locally, and ideally on-device on her phone?
Yes — and Gemma is the ideal pick. Gemma 3 runs locally via Ollama now, and Gemma 3n is purpose-built for on-device Android (text + audio) via MediaPipe, so the same open model family can eventually run fully offline on her phone. For a weekend we'll ship a local-first web MVP (installable as a PWA), with on-device Android documented as Phase 2. Let me check the hardware first.
6 GB VRAM — enough for gemma3:4b (quality sweet spot) plus a small embedder. Pulling both open models.
Open core confirmed. Architecture: React + Vite + Tailwind client; Node + Express API; a real Memory Vault in SQLite (not chat logs). Every model sits behind an AIProvider abstraction (OllamaProvider for Gemma + an honest MockProvider fallback) so the open model is the core and nothing is coupled to a vendor. Scaffolding the server, DB schema, provider layer, memory store, and the fictional 'Margaret' demo seed.
First real snag: better-sqlite3 tried to compile from source and failed — no Visual Studio C++ toolchain on this Windows machine.
Rather than make the user install a multi-GB compiler, I switched to Node's built-in node:sqlite (--experimental-sqlite) — same synchronous API, zero native build. A nice side effect: the project now installs clean on any machine with no toolchain.
Second snag, specific to small VRAM: the first conversation turn timed out and retrieval came back empty. Gemma (~5 GB) and the embedder were evicting each other between turns. Checked residency:
They fit together; the models were just unloading on the default keep_alive. Fix: pin both resident with keep_alive:'30m' on every call + a boot warmup. Warm turns now land in ~2–4 s.
Core loop working end-to-end. The companion is grounded ONLY in retrieved memories (embed the query → cosine search the vault → assemble just the relevant memories → Gemma). Tested:
Memory extraction: Gemma turns a spoken passage into a structured, provenance-tagged memory — never a raw transcript, never inventing facts.
Honesty rails are in the data model, not just prompts: immutable originals (a revised date becomes a SECOND recollection — both kept, never silently chosen), provenance tags on every fact (DIRECT / APPROXIMATE / DERIVED / GENERATED), never-invent-date-precision (I clamp year-only dates so '1944' never becomes a fabricated day), consent-to-save, mic-only-on-tap, and AI art always explicitly labeled as an interpretation.
Verified the UI live in the browser: the warm 'Memory House' renders with a breathing presence orb ('I'm here.'), a Gemma-online indicator, the Living Room conversation, and the Library showing memory cards with 'Her words' provenance badges and entity chips. Then shipped it: wrote the README + docs + MIT license, committed, and pushed a public repo.
Why does the open-source approach matter here, honestly?
Because these are a dying woman's most intimate memories. Local, open-weight AI is the only architecture that keeps them on the device (nothing uploaded), works offline, costs nothing to run, and lets the family own, swap, or fine-tune the model with nothing locked behind a vendor — and because I control the whole pipeline, I can PROVE nothing leaves the device rather than just promise it. A closed API could produce the conversation; it could not produce the ownership. That difference is the entire point of Storyhouse.
Prize Categories
- Gemma — the open-weight Gemma model is genuinely the core intelligence of the app (conversation, memory extraction, entity reasoning, retrieval), run locally. I've deliberately not claimed partner categories for tech I didn't actually use.
Built by Next Realm Interactive. For my grandmother — and for everyone's.
Top comments (0)