This is a submission for the Hacktoberfest Open-Source AI Challenge Week 4: Ghost in the Machine
What I Built
Open a drawer in almost any family home and you will find them: letters, diaries, a ledger in someone's careful hand. The ink is still dark. The people who could read it quickly are getting older, and the people who inherit the drawer often cannot read it at all. A language they never learned. A cursive nobody taught them.
Heirloom is the tool I wanted for that drawer. You photograph a page. A model running on your own laptop reads it and shows you its reading beside the original photo, because a machine that reads handwriting will sometimes be wrong, and the honest design is one where a person can see that and fix it. Click any word, type what the page really says, press Enter. Heirloom then translates the page, pulls out the people, places, dates and objects in it, and files it in an archive you can search.
How wrong? In our demo the model read a full page of cursive and missed exactly one thing: the last letter of the writer's name. Fixing it took one click, and that click is the whole project. More on it at the end.
Heirloom runs on one computer with open weights. There is no account and no upload. After a one-time download of the model and the dependencies, running it needs no internet. In the demo video the Wi-Fi is off.
*The ghost. * The theme asked for ghosts. Ours is a voice. Handwriting keeps how someone paused, pressed hard, crossed a word out and went on. A page nobody can read is a voice nobody can hear. Día de los Muertos is the night of the year built on the idea that remembering is something you do, with marigolds, food and a name said out loud. Heirloom is the unglamorous version: make the page readable again, and keep it somewhere the family controls.
What it does:
- Reads handwriting from a JPG or PNG with Gemma 4, locally.
- Keeps the original beside the transcript, with zoom and pan, so every word can be checked against the ink.
- Lets you correct any word in one click. Corrections are saved with their date and always win.
- Translates into English, Hindi, Marathi or Spanish. After corrections, Refresh translates again from the fixed text.
- Pulls out facts (people, places, dates, objects). Click one to jump to the exact spot in the transcript.
- Searches the whole archive across transcripts, translations and facts.
- Deletes a page completely: photo, text and every correction.
Demo
Notes on the video:
- Recorded with Wi-Fi off. The model, the server and the database all run on the laptop.
- The page is a handwriting sample, not a family document. This cursive page took about 35 seconds from photo to transcript, translation and facts.
- The narration is AI generated (Clipchamp text to speech).
- There is no hosted version, on purpose. Running it on your own computer is the point.
Code
Heirloom
Read your family's handwritten letters, privately, on your own computer.
Heirloom turns photos of old handwritten family documents into text you can read, correct, translate and search. It runs entirely on your laptop with an open-weight model (Gemma 4 through Ollama). No account, no cloud, no internet needed.
Built for the DEV x MLH Hacktoberfest Open-Source AI Challenge, Week 4: "Ghost in the Machine" Category: Best Use of Gemma. Theme: honoring the past, in the spirit of Día de los Muertos.
| DEV post | TODO_POST_URL |
| Demo video | TODO_VIDEO_URL |
| Tags |
#devchallenge #hf26challenge
|
The screenshots show sample pages that ship with the project. They are generated images, not real family scans.
Why we built it
Many families hold letters, diaries and ledgers written by hand, often in a language or script that younger relatives cannot read. The people who could read them are getting older. The pages sit…
To run it on Windows you need Python 3.11 and Ollama:
git clone https://github.com/VinayakGawade009/Heirloom
cd Heirloom
ollama pull gemma4:e2b
pip install -r backend\requirements.txt
.\start.bat real
Open http://localhost:8000. The built web app is in the repo, so you do not need Node. MIT licensed.
How I Built It
Everything that does the work is open: Gemma 4 (open weights) served by Ollama, SQLite with FTS5 search, a small Python server and a React front end. The server talks only to Ollama on the same computer.
We were three people with one week. Before anyone wrote code we wrote down the API in one file (docs/api-contract.md), split the project along the seams where each part could be tested alone, and merged at the end.
The engine (Arushi)
The engine turns one photo into a transcript, a translation and a list of facts:
transcribe(image_path, model, language_hint=None) -> {"text": str, "uncertain": [{"start": int, "end": int}]}
translate(text, target_language, model) -> str
extract(text, model) -> [{"kind": str, "text": str, "start": int, "end": int}]
The rule that made the app trustworthy is this: the model never gets to say where. Small models invent character offsets. So for facts, the model only lists values (a name, a place, a date). Our code finds each value in the transcript, throws away anything that is not actually in the text, and keeps at most five occurrences. Every highlight in the app lands on text that really is in the transcript. Whether the transcript matches the ink is a different question, and that is what the correction loop is for.
Everything else is pinned to fit a small laptop. Each request uses num_ctx=4096 and temperature=0, so it fits a 6 GB GPU and varies as little as possible between runs. Photos are rotated by their EXIF tag and shrunk to 1280 px before the model sees them. Gemma 4 e2b (4.6 GB) is the default, and e4b is one click away in Settings.
The archive (Aniruddha)
The server keeps each original photo untouched and works through pages one at a time in a background worker, so a modest laptop is never overloaded. A page moves from queued to transcribing, translating, extracting and done, and if the server stops mid-page it picks that page up again at the next start. SQLite holds everything, and FTS5 powers search over the corrected text: every word must match, words match as prefixes, accents are ignored.
The face (Vinayak)
The web app is React, Vite, TypeScript and Tailwind, with fonts and icons bundled. I wrote a script, npm run check:offline, that fails the build if any source file contains a web address other than localhost. An offline promise you cannot test is a slogan. While the server was still being built the app ran against a mock API behind a one-line switch, so the screens were done before the backend was. We drafted the interface in Google Stitch, and much of the front end was built with the help of AI coding agents (Google Antigravity).
How fast it is
Measured on a laptop with an RTX 4050 (6 GB), model already loaded, all three steps together:
| Page | Time per page |
|---|---|
| Neat English | about 20 to 22 s |
| Catalan | about 32 to 41 s |
| Marathi | about 36 to 72 s |
The first page after Ollama starts takes about 20 seconds longer while the model loads. gemma4:e2b uses about 3.2 GB of the 6 GB GPU. gemma4:e4b took 45 to 168 seconds per page on the same laptop, so it is not the default.
Where it broke
- The small model almost never marks words it cannot read. We planned to highlight "unsure" words for a person to check. In practice that list is nearly always empty, so we made every word clickable instead.
- Handwriting quality varies. Neat English reads well. Catalan and Marathi pages came out partly wrong, and one Marathi page gave different text on different runs, even at temperature 0.
- One misread word carries into the translation. "Roja" translated as "redhead"; "rosa" gave "rose". That is why Refresh exists: correct the transcript, then translate again from the fixed text.
- Ollama's model runner crashed a few times during testing. If a page fails, quit Ollama completely, start it again, wait about ten seconds and retry.
- Windows is the only platform we tested.
Why Does Open Innovation Matter?
A ghost should live in a machine you own.
- Privacy. Family papers hold names, addresses and land records, some about people who are still alive. They should not have to leave the house to be read.
- It works with the lights off. The archive and its keeper are often somewhere with weak internet. Once the model and dependencies are installed, Heirloom needs none.
- No meter running. There is no per-page fee, so nobody has to decide which pages are worth the money.
- It will not disappear. There is no service to shut down or change its terms. The model, the data and the code are files on your computer, and a backup is a copy of one folder.
- The model is a setting. Settings lists the models you have installed in Ollama and uses whichever you pick. We picked a size to fit a 6 GB laptop GPU, and running locally let us make that choice.
There is a real trade-off, and we took it knowingly. A large hosted model would probably read difficult handwriting better than a 4.6 GB local one. We chose ownership over accuracy, then built the correction loop so a person can close the gap one word at a time.
My Agent Session
The Spirit of It
On the demo page, the model read a whole page of cursive correctly except for one thing: it dropped the last letter of the signature. That is the writer's name.
I clicked the word, typed the missing letter and pressed Enter. The correction was saved with its date. After Refresh, the corrected name showed up in the facts too.
It is a one-letter fix, and it is the whole design in one click. The machine does the reading. A person decides what a name is.
Prize Categories
Best Use of Gemma. Gemma 4 (gemma4:e2b by default, gemma4:e4b selectable in Settings) runs locally through Ollama and does all three model steps: transcription, translation and fact extraction. It is the only model in the project.
Thank you for reading, and thank you to my teammates @arushigulbhile and @aniruddhashirsat .
Top comments (0)