This is a submission for the Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine
TL;DR Shezhire turns a recorded family story into a Kazakh family tree (шежіре) with Gemma 4 running on open weights. The model reads, but it does not get the last word: every link has to point at the sentence and the second of the recording where it was said, and plain code checks that the words are really there. On 60 test families, Gemma 4 12B got 85.5% of its links right. Every one of the 118 wrong links was caught, so 100% of what reached the page was right.
What I Built
In Kazakhstan there is a question every kid hears sooner or later, usually from an old man at a wedding table: "Жеті атаңды білесің бе?" Do you know your seven fathers?
You are supposed to answer by heart. Your father, his father, his father, seven names back. Most of us can't. We get to three, maybe four, and then somebody's grandmother leans over and finishes it for us, with a village, a year and a story about why the family moved.
That grandmother is the whole archive. There is no database behind her. When she is gone, the tree is gone.
So I wanted the simplest possible thing: put a phone on the table, let her talk, get the tree out of the recording.
The first real run showed me the ghost. The story said:
Моего прадеда звали Сәкен… Его жену, мою прабабушку, звали Мария. У них было двое детей: Данияр и Ерлан.
My great-grandfather was Сәкен… They had two children, Данияр and Ерлан.
Gemma 4 12B read it and answered, confidently, with a quote: "Сәкен is the grandfather of Данияр." Of his own son. One line of JSON, and a whole generation that never lived appears between a father and his children. On a family tree that is the worst kind of mistake. It looks right. Somebody will print it.
Shezhire keeps the model but takes away its last word:
- Gemma reads the transcript and proposes links, each with a segment number and a quote.
- Code checks each one against the recording: is the quote really there, are both people named in that sentence, is there a kinship word of the right kind (Russian, Kazakh or English).
- What passes is drawn in ink and links to the exact seconds of audio. What fails stays on the page as a ghost: dashed, in red pencil, with the reason it was turned away.
Here the model decided that the narrator's younger brother has the same father as she does. Probably true. But nobody said it, and the code will not write down what nobody said.
Demo
Live page: Әлия's family tree, built by Gemma 4 12B. Click a name or a line. Ink lines show the quote and the time it was said; dashed lines show what the model claimed and why it was dropped.
The same story read by the small model, Gemma 4 E2B, for comparison.
The story is invented but written the way older relatives talk: Russian with Kazakh kinship words (атам, әжем, әкем), people called by role ("my grandfather") instead of name, a neighbour everyone calls апа who is not family.
Code
oleg-vdv
/
shezhire
Kazakh family tree (shezhire) from recorded stories: a local Gemma reads, code checks every link against the recording, invented relatives stay as ghosts.
Shezhire
A family tree from recorded stories, built by a local open-weight model and checked line by line against the recording.
Шежіре is the Kazakh word for a genealogy. Kazakh families keep track of seven generations of ancestors (жеті ата), mostly through stories told by the oldest relatives. When those relatives are gone, the stories go with them.
Shezhire takes a recording of a grandmother telling the family story, lets an open-weight model (Gemma 4 through Ollama) read the transcript, and builds the tree. The model is not trusted. Every link it proposes goes through code that has to find, in the recording itself:
- the quoted words, in the segment the model points to (or the one right next to it),
- both people, named in that sentence: by name, by a role already proven ("my grandfather" once we know who he is), or by "I/my" for the narrator…
How I Built It
audio ─faster-whisper─▶ timestamped segments ─Gemma 4 via Ollama─▶ claims (JSON)
claims ─verify.py─▶ verified links + ghosts ─tree.py─▶ generations, contradictions ─▶ one HTML page
Everything here is open: Gemma 4 weights, Ollama, faster-whisper, and about 800 lines of Python.
The check
For every link the model proposes, shezhire/verify.py asks four questions, in this order:
- Is this person in the recording at all? If a name is never spoken, it is a ghost.
- Is the quote really there? Fuzzy match against the segment the model pointed to. A paraphrase is not evidence.
- Are both people in that sentence? By name, by "I/my" for the narrator, or by a role that is already proven. At least one of them must be named literally.
- Is there a kinship word of the right kind? отец, әкем, жена, үйленді, бауыр, ата, әже, сын, дочь, родились… matched by prefix, so Kazakh suffixes (әкесінің) and Russian case endings (отца) still count.
Plus two rules I added after reading the small model's mistakes:
- "My mother" is the narrator's mother. "Мою маму зовут Ақмарал, она вышла замуж за Берік" does not make Ақмарал the mother of Берік, though a 2B model will happily say so.
- Two children listed together are not parent and child. "двое детей: Данияр и Ерлан" makes them siblings.
And two things that make it work on how people actually talk:
- Roles become names once proven. After the code verifies "Данияр is my grandfather", the word дед later in the story counts as Данияр. Only roles relative to the narrator, only after verification.
- One sentence of memory. "Моего прадеда звали Сәкен. Его жену звали Мария." The husband is one sentence back. The check allows exactly one, and the evidence on the page covers both sentences.
How I tested it
I don't have a GPU at home, so the eval ran on a free Kaggle T4 with Ollama inside the notebook. The runner is in the repo (kaggle/build.py).
eval/make_synthetic.py generates families with a known true tree and the traps real recordings have: a named neighbour who is "like family", a child whose name nobody remembers, "he married in 1942" with the husband in the previous sentence, Kazakh words inside Russian speech. I made 30 families, tuned the check on them, then generated 30 new families with a different seed that the rules had never seen.
Final setup, Gemma 4 12B:
| 30 families I tuned on | 30 new families | |
|---|---|---|
| Links the model proposed | 407 | 407 |
| Correct before the check | 85.5% | 85.5% |
| Wrong links caught as ghosts | 59 of 59 | 59 of 59 |
| Correct after the check | 100% (303 of 303) | 100% (290 of 290) |
| True links the check threw away | 46 of 350 | 59 of 351 |
| Time per story on a T4 | 45 s | 44 s |
"True links thrown away" is the price. Almost all of them fail rule 3: one of the two people is only named further back in the story. I chose that trade on purpose: a missing branch is a gap the family can fill, an invented one is a lie on the page.
What went wrong along the way
-
Thinking mode. Gemma 4 thinks by default. My first run on Kaggle took almost five hours, and with thinking on, the 12B model returned empty JSON for nearly every story. One flag (
"think": false) and it went down to 45 seconds a story. - A better prompt made a worse small model. I added rules to the prompt ("always give the narrator's own links", "a great-grandfather is not a grandfather"). The 12B model started finding more real links: 303 verified, up from 246. The small E2B model got confused by the longer prompt and mostly stopped copying quotes. The check rejected almost everything it said: 44 links survived out of 378. Its tree came out nearly empty, but not wrong. That is the failure mode I want: when the model gets worse, the page gets emptier, not more fictional.
- E2B with the short prompt, on the 30 new families: 71% correct before the check, 93% after, 77 of 87 wrong links caught. Small models can be used this way too, but 7% wrong is too much for a family tree, so the default is 12B.
What the check does not catch
- Direction. "А is the son of B" and "B is the son of A" have the same names and the same word. The tree's contradiction check (two fathers, a person who is their own ancestor) catches some of these, not all.
- It caught the Сәкен ghost for a modest reason. The quote did not name Сәкен, so rule 3 rejected it. The check does not understand generations; it just refuses to accept a link the words do not support.
-
Synthetic is easier than real. Real recordings have crosstalk, dialect and Whisper mistakes on Kazakh names. The audio path (
shezhire transcribe) is in the repo; the numbers above are on typed transcripts.
Why Does Open Innovation Matter?
- The recordings never leave the house. These are the voices of people who may already be gone, with names, villages, illnesses and old family arguments. With open weights, Whisper and Gemma run on the family's own machine. No account, no upload, no terms of service on your grandmother.
- It will still open in twenty years. A tree that lives in a subscription service disappears with the service. Shezhire writes one HTML file with every quote inside it, and the model weights sit on the same disk.
- I could see the mistakes. Because the weights are open, I could run the same 60 families through a 2B and a 12B model, with thinking on and off, two prompts, and compare their ghosts line by line. That is how I found the "my mother" rule and the thinking-mode trap.
- Kazakh. I could test exactly how the model handles әкем inside a Russian sentence and patch the gap in code, instead of waiting for a vendor to care.
A closed model would probably write nicer JSON. But a family tree does not need nicer JSON. It needs every line to point at the recording.
The Spirit of It
Día de los Muertos builds an ofrenda so the dead are remembered correctly: their names, their food, their photos. A шежіре makes the same promise in a different form.
The scariest ghost for that promise is not a spirit. It is a person who never lived, written into the family by a machine in a calm, confident voice. So in Shezhire the ghosts stay on the page, on purpose, in red pencil, each with a note: the recording does not say this. The tree is honest about what the machine wanted to add.
Prize Categories
- Best Use of Gemma. Gemma 4 (E2B and 12B) through Ollama is the reader; the eval compares both, with thinking on and off, on the same 60 families.
Top comments (1)
A question for anyone who has tried to record family stories: what broke first for you, the transcription or the people? I only tested on typed transcripts so far. If you have a short recording in Russian or Kazakh that you’re willing to share, I’d love to see how many ghosts Gemma finds in it.