DEV Community: Sandip Ghorai

Sloth Receipts: I photograph a receipt and a local AI does my expense report

Sandip Ghorai — Tue, 20 Oct 2026 04:41:11 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady

What I Built

Sloth Receipts is a small Python tool that turns a folder of receipt photos into an expense spreadsheet, using an open-weight vision model running on my own laptop.

The lazy workflow:

Snap a photo of the receipt and drop it in a folder.
Go to sleep.
Wake up to expenses.csv with the date, merchant, category and total filled in, plus a monthly summary by category.

Expense reports are a chore everyone puts off. Nobody enjoys typing numbers off crumpled paper, so I let a sloth do it.

It is lazy and honest about its mistakes:

Two-pass self-check. Each receipt is read twice. If the two passes disagree on the total, the row is flagged needs_review instead of being silently trusted.
Sanity checks. Missing totals, impossible or future dates, and unknown categories are flagged too.
Never re-processes. Each photo is hashed, so a nightly run only handles new receipts.
I only review the flagged rows, not every receipt.

Demo

Try it in two minutes. Install Ollama, then:

ollama pull qwen2.5vl:3b
python3 sloth_receipts.py ~/Receipts

The output is expenses.csv with these columns:

file,date,merchant,category,currency,total,needs_review,reason

Rows the model is unsure about are marked YES in needs_review, with the reason (for example, passes disagree on total), so a human only checks those.

Code

GitHub repo: sloth-receipts by SandipGhorai-max

sloth_receipts.py (Python standard library only):

#!/usr/bin/env python3
"""Sloth Receipts: receipt photos -> expense CSV, using a local open-weight vision model (Ollama)."""
import argparse, base64, csv, hashlib, json, re, urllib.request
from collections import defaultdict
from datetime import date, datetime
from pathlib import Path

OLLAMA = "http://localhost:11434/api/chat"
MODEL = "qwen2.5vl:3b"
CATEGORIES = ["Food", "Groceries", "Transport", "Travel", "Office",
              "Software", "Health", "Utilities", "Entertainment", "Other"]
IMAGE_EXT = {".jpg", ".jpeg", ".png", ".webp"}
FIELDS = ["file", "date", "merchant", "category", "currency", "total", "needs_review", "reason"]
LEDGER = ".sloth-receipts.json"


def read_receipt(path: Path, model: str) -> dict:
    prompt = (
        "Read this receipt photo. Reply ONLY with JSON: "
        '{"merchant": str, "date": "YYYY-MM-DD", "currency": "ISO code like INR or USD", '
        f'"total": number (the final amount paid), "category": one of {CATEGORIES}}}. '
        "Use null for anything you cannot read. Do not guess."
    )
    img = base64.b64encode(path.read_bytes()).decode()
    body = json.dumps({
        "model": model, "stream": False, "format": "json",
        "options": {"temperature": 0.2},   # slight variation makes the 2nd pass a real check
        "messages": [{"role": "user", "content": prompt, "images": [img]}],
    }).encode()
    req = urllib.request.Request(OLLAMA, body, {"Content-Type": "application/json"})
    try:
        with urllib.request.urlopen(req, timeout=300) as r:
            return json.loads(json.loads(r.read())["message"]["content"])
    except Exception:
        return {}


def to_amount(value):
    if value is None:
        return None
    s = str(value).strip()
    if re.search(r",\d{2}$", s) and "." not in s:   # "12,50" -> "12.50"
        s = s.replace(",", ".")
    s = re.sub(r"[^0-9.\-]", "", s.replace(",", ""))
    try:
        return round(float(s), 2)
    except ValueError:
        return None


def to_date(value):
    try:
        d = datetime.strptime(str(value), "%Y-%m-%d").date()
        return d if d <= date.today() and d.year >= 2000 else None
    except ValueError:
        return None


def process(path: Path, model: str, passes: int) -> dict:
    reads = [read_receipt(path, model) for _ in range(passes)]
    first = reads[0] or {}
    totals = [to_amount(r.get("total")) for r in reads]
    reasons = []

    total = totals[0]
    if total is None or total <= 0:
        reasons.append("total unreadable")
    if len(set(totals)) > 1:
        reasons.append(f"passes disagree on total {totals}")

    d = to_date(first.get("date"))
    if d is None:
        reasons.append("date missing or invalid")

    cat = first.get("category")
    if cat not in CATEGORIES:
        reasons.append("unknown category")
        cat = "Other"

    return {
        "file": path.name,
        "date": d.isoformat() if d else "",
        "merchant": (first.get("merchant") or "").strip(),
        "category": cat,
        "currency": (first.get("currency") or "").upper(),
        "total": f"{total:.2f}" if total else "",
        "needs_review": "YES" if reasons else "",
        "reason": "; ".join(reasons),
    }


def summarize(csv_path: Path):
    sums = defaultdict(float)
    review = 0
    with csv_path.open(newline="") as f:
        for row in csv.DictReader(f):
            review += bool(row["needs_review"])
            if row["total"] and row["date"]:
                sums[(row["date"][:7], row["currency"] or "?", row["category"])] += float(row["total"])
    print("\nMonthly summary")
    for (month, cur, cat), amount in sorted(sums.items()):
        print(f"  {month}  {cat:<14} {amount:>10.2f} {cur}")
    print(f"\n{review} row(s) flagged needs_review in {csv_path.name}")


def main():
    p = argparse.ArgumentParser(description="Receipt photos -> expense CSV, locally.")
    p.add_argument("folder", type=Path)
    p.add_argument("--model", default=MODEL)
    p.add_argument("--passes", type=int, default=2, help="reads per receipt (2+ enables the self-check)")
    a = p.parse_args()

    ledger_path = a.folder / LEDGER
    ledger = json.loads(ledger_path.read_text()) if ledger_path.exists() else {}
    csv_path = a.folder / "expenses.csv"
    new_file = not csv_path.exists()

    with csv_path.open("a", newline="") as out:
        w = csv.DictWriter(out, fieldnames=FIELDS)
        if new_file:
            w.writeheader()
        for img in sorted(a.folder.iterdir()):
            if img.suffix.lower() not in IMAGE_EXT:
                continue
            digest = hashlib.sha256(img.read_bytes()).hexdigest()
            if digest in ledger:
                continue
            row = process(img, a.model, a.passes)
            w.writerow(row)
            out.flush()
            ledger[digest] = img.name
            ledger_path.write_text(json.dumps(ledger, indent=2))
            flag = "  <-- REVIEW" if row["needs_review"] else ""
            print(f"{img.name}: {row['merchant']} {row['total']} {row['currency']}{flag}")

    summarize(csv_path)


if __name__ == "__main__":
    main()

To run it nightly while I sleep (crontab -e, use absolute paths):

0 3 * * * /usr/bin/python3 /home/YOUR_USER/sloth-receipts/sloth_receipts.py /home/YOUR_USER/Receipts

How I Built It

Model: an open-weight vision-language model (Qwen2.5-VL) served locally by Ollama. It can be swapped with --model.
Inference: one HTTP call per read to Ollama's /api/chat, with the image sent as base64 and format: "json" to force parseable output.
Trust, but verify: a small model can misread a blurry total, so I don't trust a single read. Each receipt is read twice at a slightly non-zero temperature, and any disagreement flags the row for human review. Because inference is local and free, the second pass costs only a few seconds of compute.
Validation layer: the model only proposes values. My code parses amounts (including 12,50 style decimals), rejects future dates, whitelists categories and flags anything doubtful.
Idempotent: SHA-256 hashes of the photos mean the nightly job only touches new receipts.
Dependencies: just the Python standard library.

Why Does Open Innovation Matter?

A receipt is more personal than it looks. It shows where I shop, what I buy, when I travel, and often part of a card number. Sending a shoebox of those to a cloud vision API is a hard sell, and a business sending client receipts would face compliance questions on top of that.

With an open-weight model:

Privacy: every photo stays on my machine. There is no upload step to audit because there is no upload.
Self-consistency is free: reading each receipt twice would double the bill on a paid API. Locally it costs nothing extra, so the added reliability is effectively free.
Zero running cost: a month of receipts costs nothing in tokens, so I never have to ration it.
Offline: it works in airplane mode, which helps when receipts pile up while traveling.
Swappable and tunable: if a better small vision model ships, I change one flag. A business could fine-tune on its own receipt formats and categories.
Pinned behavior: the model sits on my disk, so a provider update can't quietly change how my books get categorized.

Limitations

A 3B vision model is not perfect. Crumpled, faded or very long receipts can trip it up, and handwritten receipts are a weak spot. That is exactly why the tool flags uncertain rows for review instead of pretending to be accurate. For better accuracy, run --model qwen2.5vl:7b if your hardware allows.

Udhaar Voice: a credit ledger you fill by talking, built for shopkeepers, running with no internet

Sandip Ghorai — Tue, 13 Oct 2026 09:15:41 +0000

This is my submission for the Hacktoberfest Open-Source AI Challenge: Week 2, theme: Side Hustle.

The Problem

Small neighbourhood shops (kirana stores, tailors, tiffin kitchens, repair stalls) sell on credit all the time. "Ravi anna took 2 kg rice and oil, he'll pay Saturday." That sentence goes into a paper notebook, if it gets written down at all.

Then three things go wrong:

The shopkeeper is busy with customers and forgets to write it
Handwriting gets messy, and the notebook gets lost or damp
At month end, nobody is sure who owes what

Existing apps need typing, English menus and a steady internet connection. The shopkeeper has wet hands, a queue, and a phone.

What I Built

Udhaar Voice lets the shopkeeper say the entry in their own language, in a short voice note, and the system does the rest:

Transcribes the audio using Whisper, running locally
Understands the sentence with a local Gemma model and pulls out: customer name, items, amount, and promised pay date
Updates a ledger (one SQLite file) and keeps each customer's running balance
Generates a reminder message for any customer overdue, ready to send in one tap

A voice note like "Ravi anna ku oru kilo arisi, 80 rupees, Saturday tharuvaar" becomes:

Customer	Items	Amount	Due
Ravi	1 kg rice	80	Saturday

The shopkeeper can speak in their normal mix of languages. They never touch a form.

Who Would Pay For This

Shopkeepers who already lose money to forgotten credit. If the tool helps them recover even a few unpaid amounts a month, a small monthly fee pays for itself. That is a much easier sell than "digitise your business".

Why Open Innovation Matters Here

This idea only works because the pieces are open.

It runs offline. Shop phones and cheap laptops often have poor connectivity. Whisper and Gemma run on the device, so the ledger works when the network doesn't.
Customer data stays private. A credit ledger lists who owes whom and how much. That is sensitive community information. Nothing is uploaded anywhere.
No per-minute bills. Hosted speech APIs charge by the minute of audio. A shop recording 50 notes a day would make a cheap product unprofitable. Local inference costs nothing per note.
Language flexibility. Whisper handles many Indian languages and mixed-language speech reasonably well, and I can fine-tune later on local shop vocabulary such as item names and nicknames, which general tools often get wrong.
No lock-in. If a model gets better or a provider changes its prices, I swap the model, not the product.

How It Works

Step 1: transcribe locally with faster-whisper.

from faster_whisper import WhisperModel

stt = WhisperModel("small", compute_type="int8")  # runs on a normal CPU

def transcribe(audio_path):
    segments, info = stt.transcribe(audio_path, task="transcribe")
    return " ".join(s.text for s in segments).strip()

Step 2: turn the sentence into structured data with a local Gemma model.

import json, ollama

PROMPT = """You help a shopkeeper keep a credit ledger.
From the sentence below, return ONLY JSON with keys:
customer, items (list of strings), amount (number or null),
due (text or null), type ("credit" if they took goods, "payment" if they paid).
Sentence: {text}"""

def parse(text, model="gemma3"):
    r = ollama.chat(model=model,
                    messages=[{"role": "user", "content": PROMPT.format(text=text)}],
                    format="json")
    return json.loads(r["message"]["content"])

Step 3: save to the ledger and keep a running balance.

import sqlite3

db = sqlite3.connect("ledger.db")
db.execute("""CREATE TABLE IF NOT EXISTS entries(
    id INTEGER PRIMARY KEY, customer TEXT, items TEXT,
    amount REAL, due TEXT, type TEXT, created TEXT DEFAULT CURRENT_TIMESTAMP)""")

def save(entry):
    if entry["amount"] is None or not entry["customer"]:
        return "needs_confirmation"      # never guess money
    db.execute("INSERT INTO entries(customer, items, amount, due, type) VALUES (?,?,?,?,?)",
               (entry["customer"], ", ".join(entry["items"]),
                entry["amount"], entry["due"], entry["type"]))
    db.commit()
    return "saved"

def balance(customer):
    cur = db.execute("""SELECT COALESCE(SUM(CASE WHEN type='credit' THEN amount ELSE -amount END),0)
                        FROM entries WHERE customer = ?""", (customer,))
    return cur.fetchone()[0]

Design decisions

Read it back before saving. The app shows "Ravi, 1 kg rice, 80, Saturday. Correct?" with one tap to confirm. Speech recognition makes mistakes, and money entries must be right.
Missing amount means ask, not guess. The code above refuses to save an entry without a customer or an amount.
One file, no server. The whole ledger is a single SQLite file the shopkeeper can back up by copying it.

Demo

Code: [GitHub repo link]
Demo (video or GIF showing a voice note turning into a ledger row): [link]

My First User

[Edit honestly. For example: who tried it, what language they spoke, what the app got wrong, whether they would pay. If nobody has tried it yet, write: "I haven't had a user yet. This week I'll ask one local shopkeeper to test it and I'll update this post with what they say."]

What I Learned

The hard part is not the AI, it is the trust. Reading the entry back before saving mattered more than any model choice.
Names and local item words are the weakest point of speech recognition. This is where fine-tuning on real shop speech would help most.
Designing for wet hands and a queue changed the whole product: one tap, no typing.

What's Next

A simple phone-friendly interface with one big record button
A weekly summary of "who owes what"
Reminder messages written in the customer's language
Fine-tuning on real shop voice notes (with permission)

Prize Category

Best Use of Gemma: a local Gemma model turns messy speech into structured ledger entries.

Thanks for reading! I'd love to hear from anyone who has built for small shops or offline-first users.

I Taught My Laptop to Whisper Bird Names So I Could Finally Look Up

Sandip Ghorai — Tue, 06 Oct 2026 14:03:05 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

What I Built

Every time I try to identify a bird with an app, the same thing happens. I hear a call, pull out my phone, wake the screen, tap record, and by the time the app answers, the bird is gone and I'm staring at a glowing rectangle instead of a tree.

So I built Trailbird, a bird call identifier with almost no screen. The laptop goes in my backpack, one earbud goes in my ear, and I just walk. When it hears a bird, a calm voice says one sentence: "Northern cardinal, probably in the shrubs to your left." If it's a species I've never logged, it tells me that too.

It's for anyone who wants to learn bird calls without turning a walk into screen time.

Demo

[Add your video or link here. Even a 30-second clip of you walking and the earbud audio is enough.]

Code

[Embed your repo: {% embed https://github.com/YOUR-USERNAME/trailbird %}]

The core is a single loop: listen for 3 seconds, detect, speak, log.

while True:
    clip = listen()
    for species, conf in analyze(clip, SAMPLE_RATE):
        if conf < MIN_CONFIDENCE:
            continue
        if time.time() - last_spoken.get(species, 0) < COOLDOWN_SECONDS:
            continue
        speak(field_note(species, conf, species not in seen))
        log_sighting(species, conf)
        last_spoken[species] = time.time()

How I Built It

Everything runs locally:

BirdNET-Analyzer detects species from raw audio. Its weights are CC BY-NC-SA, so this is a non-commercial project.
Gemma 3 (4B) via Ollama turns "Cardinalis cardinalis, 0.82" into one friendly sentence.
Piper speaks it offline.
SQLite keeps my personal life list.

The pipeline: mic -> BirdNET -> Gemma -> Piper -> SQLite.

Three small decisions made it usable outdoors:

A cooldown per species. Without one, a single robin narrated itself endlessly. Two minutes of silence per species fixed it.
One sentence, enforced in the prompt. Language models love paragraphs. Nobody wants a paragraph in their ear on a trail.
A small model. A 4B Gemma runs on a laptop without a GPU, so the whole thing works in a backpack.

Taking it outside

[Where you went, how long, the weather.]

[Your best moment. What did it catch, and did it change where you looked?]

[Something that went wrong, such as wind noise, a false positive, or battery drain, and how you handled it.]

Why Does Open Innovation Matter?

The places I most want to bird, like ridgelines and wetlands, have the worst signal. A cloud API fails exactly there. Because the models are open, Trailbird works with no connection at all.

It also keeps my data mine. A log of when and where I walk stays in a local file, with no account and no upload. And since each piece is swappable, I could change the language model with a one-line edit, with no vendor, key, or bill.

Open models didn't just make this cheaper. They made it possible to build the version that works in the woods.

My Agent Session

[Optional: link or embed your DevRelay session, or delete this section.]

Prize Categories

Best Use of Gemma: Gemma 3 (4B) runs locally through Ollama and writes every spoken field note.

Trailbird: A Bird Call Identifier That Gets Your Eyes Off the Screen

Sandip Ghorai — Tue, 06 Oct 2026 13:55:34 +0000

This is my submission for the Touch Grass challenge.

The Problem With Bird Apps

Most bird identification apps do the job and ruin the walk. You hear something in the trees, pull out your phone, open an app, hold it up, and stare at a spectrogram while the bird flies off.

I wanted the opposite: a tool that makes you look up.

What I Built

Trailbird is a bird call identifier that runs entirely on a laptop, with no internet and almost no screen time.

You start it, put the laptop in your backpack, and walk. Every few seconds it listens. When it recognizes a bird, it says the name in your earbud, in one calm sentence, something like "Northern cardinal, probably in the shrubs to your left." New species get flagged for your life list. The screen is only for the start and the end of the walk.

Why Open Matters Here

This project only works because every piece is open and local.

No signal needed. The best birding spots, like ridgelines, wetlands and forest edges, tend to have the worst reception. A cloud API would fail exactly where I need it.
My location history stays mine. A log of when and where I was in the woods is personal. Trailbird writes to a local SQLite file and nothing else. No account, no upload.
Every part is swappable. The detector, the language model and the voice are separate pieces. Changing the model is a one-line edit, which is much harder with a closed API.
It costs nothing to run. No API keys, no per-call fees, no subscription for a hobby.

How It Works

mic -> BirdNET (species detection) -> Gemma via Ollama (field note) -> Piper TTS (earbud) -> SQLite life list

Piece	Role	Open?
BirdNET-Analyzer	Identifies species from audio	Open source (the model weights are CC BY-NC-SA, so non-commercial use only)
Gemma, served by Ollama	Turns raw detections into one short spoken sentence	Open weights
Piper	Local text-to-speech for the earbud	Open source
SQLite	Personal life list	Open source

BirdNET does the hard acoustic work. Gemma's job is only the last step: turning "Cardinalis cardinalis, 0.82" into something a person wants to hear while walking.

The core loop

import sounddevice as sd, sqlite3, subprocess, time
from birdnet_analyzer import analyze   # adjust to your installed BirdNET version
import ollama

SAMPLE_RATE = 48000
CLIP_SECONDS = 3
MIN_CONFIDENCE = 0.6
COOLDOWN_SECONDS = 120

db = sqlite3.connect("lifelist.db")
db.execute("CREATE TABLE IF NOT EXISTS sightings (ts REAL, species TEXT, conf REAL)")

def listen():
    audio = sd.rec(int(CLIP_SECONDS * SAMPLE_RATE),
                   samplerate=SAMPLE_RATE, channels=1)
    sd.wait()
    return audio.flatten()

def field_note(species, conf, is_new):
    prompt = (
        f"You are a calm birding guide. In ONE short sentence, tell the "
        f"walker I just heard a {species} (confidence {conf:.0%}). "
        f"{'This is a NEW species for their life list!' if is_new else ''} "
        f"Suggest where to look. No emoji."
    )
    r = ollama.chat(model="gemma3:4b",
                    messages=[{"role": "user", "content": prompt}])
    return r["message"]["content"].strip()

def speak(text):
    subprocess.run(["piper", "--model", "en_US-lessac-medium", "--output-raw"],
                   input=text.encode(), check=False)

seen = {row[0] for row in db.execute("SELECT DISTINCT species FROM sightings")}
last_spoken = {}

while True:
    clip = listen()
    for species, conf in analyze(clip, SAMPLE_RATE):
        if conf < MIN_CONFIDENCE:
            continue
        if time.time() - last_spoken.get(species, 0) < COOLDOWN_SECONDS:
            continue
        is_new = species not in seen
        speak(field_note(species, conf, is_new))
        db.execute("INSERT INTO sightings VALUES (?,?,?)",
                   (time.time(), species, conf))
        db.commit()
        seen.add(species)
        last_spoken[species] = time.time()

Design decisions that mattered

A confidence threshold plus a cooldown. Without them, one persistent bird would announce itself over and over. A 120-second cooldown per species keeps the earbud quiet.
One sentence only. A language model's default is a paragraph, which is the last thing you want in your ear on a trail. Forcing a single sentence turns the tool from a lecturer into a quiet companion.
A small model on purpose. A 4B Gemma runs on a laptop without a GPU. Bigger would be smarter, but "works in a backpack with no internet" is the goal.
Speech over screen. If the tool needs you to look at it, it has failed the theme. Audio out is the whole interface.

Taking It Outside

I took Trailbird to for about on a , with the laptop in my backpack and one earbud in.

What I noticed most was that after the first few minutes I stopped thinking about the tool and just listened. The screen stayed shut for the entire walk, which was the point.

Limitations I expect to hit, and that anyone trying this should know about:

Wind and traffic noise cause false positives, so the confidence threshold matters.
Laptop battery drains faster with a mic and a model running continuously.
BirdNET is strongest on common species and weaker on rare or overlapping calls, so treat results as "probably", not gospel.

What I'd Do Next

Run it on a Raspberry Pi or similar board so no laptop is needed
Add a wind filter to cut false positives
Load a regional species list so it prioritizes what is plausible for the season and place
Export the life list to eBird when I'm back online, as an opt-in step

Closing Thought

A good outdoor tool should make itself forgettable. Open, local models are what make that possible here: no signal, no account, no server holding my walks. Just me, an earbud, and the birds.

*Built with open-source and open-weight tools: BirdNET, Gemma (via Ollama), and Piper.

My grandpa's recipes lived in WhatsApp voice memos, so I built him an offline AI cookbook

Sandip Ghorai — Fri, 02 Oct 2026 05:59:03 +0000

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend

The Person I Built This For

My thatha has been cooking sambar for about fifty years and has never once written a recipe down.

What he does instead is send voice memos. Every few weeks one lands in the family group chat, four or five minutes long, usually recorded while he's standing over a pot. "Add a little hing. Not that much. A bit less. You'll know." Then someone replies with a thumbs up, the chat moves on, and the recipe sinks under forty photos of a cousin's baby.

I've been scrolling back through that chat for a year looking for his sambar. This weekend I finally did something about it.

What I Built

Thatha's Kitchen takes one of his voice memos and turns it into a recipe card: title, ingredients, steps, and the little asides he's famous for ("don't rush the onions, they'll punish you").

The part I care about most is that it doesn't guess. When he says "a handful of dal," the card says "a handful of dal" and adds a flag: ask Thatha how big his hand is. Next time I'm at his place I can ask, and fix the card.

It also exports everything to a printable PDF, because he will never open an app but he will happily read a paper cookbook.

Code: https://github.com/your-username/thathas-kitchen

How It Works

Everything is open source and runs locally on my laptop:

faster-whisper (large-v3, int8 on CPU) for transcription. His memos switch between Tamil and English mid-sentence, so I let Whisper keep the original language instead of forcing English.
Qwen2.5 7B through Ollama to pull the recipe out of the transcript as JSON.
Streamlit for a small page on top, and WeasyPrint for the PDF.

The prompt does most of the work. I ask the model for ingredients, steps and tips, plus a fourth list called unclear for anything vague or missing. My first version didn't have that list, and the model cheerfully invented "2 teaspoons of cumin" out of nowhere. For a dish my family has eaten for decades, that's a small betrayal.

import json
import ollama
from faster_whisper import WhisperModel

SYSTEM = """You turn a transcript of an elderly cook talking into a recipe.
Return ONLY JSON with these keys:
title (string), ingredients (list of strings), steps (list of strings),
tips (list of strings), unclear (list of strings).
Rules:
- Use the speaker's own words for quantities ("a handful", "a little").
- NEVER invent a quantity, ingredient or step that was not said.
- If something is vague or missing, put a short question in `unclear`.
- Keep his personal asides in `tips`."""

whisper = WhisperModel("large-v3", device="cpu", compute_type="int8")

def transcribe(path):
    segments, _ = whisper.transcribe(
        path,
        initial_prompt="sambar, rasam, kuzhambu, tamarind, hing, curry leaves, toor dal",
    )
    return " ".join(s.text.strip() for s in segments)

def extract_recipe(transcript):
    r = ollama.chat(
        model="qwen2.5:7b",
        format="json",
        messages=[
            {"role": "system", "content": SYSTEM},
            {"role": "user", "content": transcript},
        ],
    )
    return json.loads(r["message"]["content"])

Things that went sideways:

Whisper turned "kuzhambu" into something I can't print here. Giving it a vocabulary hint through initial_prompt fixed most of it.
The model kept "tidying up" his quantities into exact measurements. Adding the line "use the speaker's own words" stopped that.
Long memos sometimes came back as broken JSON. Ollama's format="json" mode fixed it.

Why Open Source Mattered Here

I want to be specific instead of just saying "privacy."

These are recordings of my grandfather's voice. Some are the last ones where he's telling a story and cooking at the same time. I wasn't comfortable uploading them to a service I can't see inside of, so everything runs on my laptop and nothing leaves it.

It also costs me nothing. There's no per-minute transcription bill and no API key to babysit, which matters when the project is a gift and not a startup.

And I could swap models. I tried Llama 3.1 8B and Qwen2.5 7B on the same memo, and Qwen was better at keeping the unclear list honest instead of filling gaps. I don't think I could have compared models like that, or pinned one, with a hosted API.

To be fair, a big hosted model would probably have been more accurate on the rare Tamil words. I decided that wasn't worth giving up the rest.

What He Said

I printed the first three recipes and gave him the pages on Sunday.

He put his glasses on, held the pages at arm's length, and read the sambar one slowly. Then he said:

"Who wrote this? It's exactly how I talk."

He found one thing wrong, which was that I'd put the tamarind in too early, and he corrected it in pen on the page. Then he asked if I could do his rasam next.

What's Next

I want to add photos of the finished dishes, let my cousins record their own memos, and build a read-aloud mode so he can have a recipe played back to him.

Thanks for reading. If someone in your family keeps their knowledge only in their head, go record them this weekend, even if you never build anything with it.-->

Turns my thatha's rambling voice memos into a printable family recipe book, 100% local with Whisper and Qwen. No cloud needed.

Sandip Ghorai — Fri, 02 Oct 2026 05:57:01 +0000