DEV Community

ninefyi
ninefyi

Posted on

Img2text for Thai billing using Gemma.

Hacktoberfest Weekend Challenge: Build for a Friend Submission 🤝

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend

What I Built

On the 25th of every month, the housekeeper at a rental building I'll call apartment_a walks past 16 rooms with a clipboard, reads the electricity and water meters, and writes the numbers by hand on a printed table: last month's reading, this month's reading, and then, in blue ink, "units = baht".

That sheet reaches someone close to me as a photo, and she has to turn it into what each room owes for electricity and water. Until now, that meant reading 2 rows of handwriting and doing the sums herself, every month.

So I built a small app for her. You drop in the photo, a model reads the handwriting, and you get a table to check against the photo. Then you download a CSV with one row per room: electric usage and cost (units × 7 baht), water usage and cost (units × 28 baht), a totals row, and a note on any row that needs a second look.

Everything runs on one laptop. The model is Gemma 3 12B through Ollama, and the photo never leaves the machine.

Demo

A 49-second walk-through: drop in the photo, let Gemma read it (the real wait is about three and a half minutes, sped up here), check the table, fix a wrong digit on purpose and watch the warning appear and clear, then download the CSV.

Demo Click Here
The demo uses a fabricated sheet in the same template. The real sheet is not in the repository.

Code

https://github.com/ninefyi/hacktoberfest2026

The core is three small files: billsplit/extract.py (reading the photo), billsplit/calc.py (usage, cost, and checks) and billsplit/app.py (the table and download).

How I Built It

The first attempt failed. I gave Gemma the whole photo and asked for the table as JSON. It mixed up the columns, invented digits, and dropped the water half completely.

What worked was cropping. The printed template is the same every month, so I describe it once in a small layout file (where the columns and rows sit). The app then cuts the photo into pieces of five rows. Each piece is the room-number column stitched next to either the electric block or the water block. On a piece like that, the model reads cleanly. On my fabricated demo sheet it got all 64 readings right.

A tilted photo almost put numbers on the wrong tenant. My first version assumed row n of the crop was room n. In the photo, the table drifts, and from room 207 down every row held its neighbour's numbers. A wrong number on the wrong row is the worst thing this app could do, so now the model has to read the printed room label on each row, and values are matched by label, never by position. If a label cannot be found, the row is left empty and flagged.

The sheet checks itself. The housekeeper already writes "units = baht" next to each reading. The app recomputes both from the readings and flags a row when they disagree, when a reading goes down, or when usage looks too high.

**The honest results on the real sheet. Handwriting is hard. On the real apartment_a sheet:

  • Water: 15 of 16 rooms read correctly. The model didn't find the last room and flagged it.

  • Electric: 9 of 16 rooms were exactly right. Of the 7 wrong, 6 were flagged by the checks above. One was not: the model read 2309 → 2320 as 9309 → 9320, a plausible usage, so nothing looked off.
    That last one is why the table is not optional. The model does the typing, she checks against the photo, and the flags tell her where to look first.

Things I tried that did not help. I added a second reading pass with a different crop and zoom, hoping to catch the errors where the model disagreed with itself. It caught nothing new, doubled the time, and added false alarms. I left it out. A more promising idea, which I have not built: next month, flag any "previous reading" that does not match last month's "current" reading.

Why Does Open Innovation Matter?

This sheet has tenants' room numbers and how much power and water each household uses. I did not want to upload a monthly stream of that to a cloud service just to avoid typing. An open-weights model that runs on a laptop made that choice unnecessary. It also costs nothing per photo, so it still makes sense for a building with 16 rooms.

It also let me look inside: when the model failed, I could change the crops, the prompt, and the checks until I understood why, instead of hoping a closed API would improve.

My Agent Session

I built this in one day with Claude Code running on my laptop, and it was less "write me an app" than a back-and-forth.

  • It started with questions, not code. Before any code, Claude interviewed me about the plan: who the friend was, what the real problem was, which prize to aim for, and how much time I had. The first idea we settled on (explain official paperwork) was wrong, and the questions are what surfaced the real one: the meter sheet.

  • My samples broke its assumption. It had assumed I would give it photos of meters. When I dropped in three handwritten sheets, it said so and changed the design instead of forcing the old plan.

  • Failing in the open. The first model attempts were bad (the 4B model invented digits; the 12B mixed up columns on a full page). Claude tried crops, a layout file and room-label matching, and told me each time which results it could verify and which it could not. When its "fix" for the drifting rows did not work, it reported that plainly (the model had been copying the room list I gave it) instead of declaring success.

  • It scored itself against the photo. I got honest numbers (water 15 of 16 rooms, electric 9 of 16, one error unflagged) because Claude compared the output with my sheet row by row rather than saying "looks good".

  • It built the demo. It generated a fabricated sheet in the same template, scored the app against known answers (64 of 64 readings), and recorded the demo video by driving the real app in Chrome.

  • I made the decisions. I changed the product three times (payment forms, then Excel, then CSV), and each time the agent re-planned and cleaned out the old code.

Prize Categories

Gemma: the whole app runs locally on Gemma 3 12 B.

Top comments (0)