DEV Community: Zhao Yang

Shoebox: a receipt and invoice sorter for freelancers that never uploads a thing (local vision model)

Zhao Yang — Wed, 14 Oct 2026 12:05:36 +0000

This is a submission for the Side Hustle challenge: build something with open-source AI that someone would actually pay for.

Shoebox is a desktop tool for freelancers. You drop in a messy folder of receipts, invoices, and screenshots: PDFs, phone photos, whatever you have. Shoebox reads each one with an open-weight vision model running on your own laptop, pulls out the vendor, date, amount, tax, and client, sorts everything into categories, and exports a clean spreadsheet for your accountant.

Your invoices contain your clients' names, your rates, and your income. None of that is uploaded anywhere.

And because the AI runs on the customer's machine, each document costs me nothing to process. That's what lets me sell it for a one-time price instead of yet another monthly subscription.

The problem: the shoebox

Every freelancer I know has a version of the same folder. It's called receipts, or tax stuff, or 2026 DO NOT DELETE. It contains:

PDF invoices sent to clients
Invoices from software subscriptions, in three currencies
Phone photos of paper receipts, some of them sideways
Screenshots of payment confirmations
At least one file named IMG_4821.jpg that nobody can explain

Once a quarter, or once a year if they're honest, they spend a whole weekend opening every file and typing numbers into a spreadsheet. It's tedious, error-prone, and exactly the kind of work that makes people put off their taxes until it's urgent.

There are cloud tools that solve this. Most of them work the same way: upload all your documents to their servers, then pay a monthly fee. For many freelancers, both parts are a problem:

Their invoices aren't only their data. They contain client names, project details, and rates. Some clients have contracts that limit where that information can go.
Another subscription is a hard sell for a task you do four times a year.

I wanted something that works like a cloud service but runs like a desktop app: buy it once, and it works on your own laptop.

What I built

Shoebox has one main screen and one job.

1. Drop a folder in

Point Shoebox at a folder. It finds every PDF and image inside it, including subfolders, and processes them one by one. You can go make coffee.

2. Review only what needs reviewing

Most documents come through cleanly and go straight into the ledger. Anything Shoebox isn't sure about lands in a short review queue, with the reason it was flagged:

⚠ NEEDS REVIEW (3 of 87)

IMG_4821.jpg
  Total doesn't match the line items (items add up to 412.50, total reads 421.50)
  → Check the total

hosting-invoice-march.pdf
  Possible duplicate of hosting-invoice-march (1).pdf
  → Keep one

photo_2026-08-03.jpg
  New vendor, category unclear (Office supplies? Equipment?)
  → Pick a category; Shoebox will remember it

You fix three things instead of typing in eighty-seven.

3. Export for your accountant

Shoebox exports a spreadsheet with one row per document and a summary tab: income by client, expenses by category, and tax totals per quarter. Every row links back to the original file, so your accountant can check anything.

How it works

┌──────────────┐
│ Your folder  │  PDFs, photos, screenshots
└──────┬───────┘
       ▼
┌─────────────────────────────┐
│ 1. Text PDF? → extract text │  fast path, no AI needed
│    Image/scan? → vision LLM │  local model via Ollama
└──────┬──────────────────────┘
       ▼
┌─────────────────────────────┐
│ 2. Structured output        │  JSON schema: vendor, date, total, tax…
└──────┬──────────────────────┘
       ▼
┌─────────────────────────────┐
│ 3. Checks                   │  arithmetic, dates, duplicates
└──────┬──────────────────────┘
       ▼
┌─────────────────────────────┐
│ 4. Categories               │  your rules first, model only for unknowns
└──────┬──────────────────────┘
       ▼
   Ledger ──or── Review queue
       ▼
   Spreadsheet export

The same principle runs through the whole design: the model reads, but code checks. A vision model is very good at finding the total on a crumpled receipt. It's not something I want to trust blindly with numbers that end up on a tax return.

Step 1: Don't use AI when you don't need to

Many invoices are PDFs that already contain real text. Running a vision model on them would be slow and wasteful. So Shoebox tries plain text extraction first:

import pdfplumber
from pathlib import Path

def extract_text(path: Path) -> str | None:
    if path.suffix.lower() != ".pdf":
        return None
    with pdfplumber.open(path) as pdf:
        text = "\n".join(page.extract_text() or "" for page in pdf.pages)
    return text if len(text.strip()) > 50 else None  # too little text = probably a scan

Text PDFs go to a small text-only model. Images and scanned PDFs go to the vision model. In a typical freelancer's folder, a good share of documents take the fast path.

Step 2: Read the document with a local vision model

For images, Shoebox uses an open-weight vision-language model through Ollama. The response must match a JSON schema, so there's no free text to parse:

import ollama
from pydantic import BaseModel
from typing import Literal

class LineItem(BaseModel):
    description: str
    amount: float

class Document(BaseModel):
    doc_type: Literal["invoice_sent", "invoice_received", "receipt", "other"]
    vendor_or_client: str
    date: str               # ISO format, YYYY-MM-DD
    currency: str           # ISO code, e.g. USD, EUR, VND
    line_items: list[LineItem]
    tax: float | None
    total: float
    invoice_number: str | None

PROMPT = """Read this financial document and extract its details.
Use only what is visible in the document. If a field is not present, use null.
Write dates as YYYY-MM-DD. Use the currency shown on the document."""

def read_image(path: Path) -> Document:
    resp = ollama.chat(
        model="qwen2.5vl:7b",
        messages=[{"role": "user", "content": PROMPT, "images": [str(path)]}],
        format=Document.model_json_schema(),
        options={"temperature": 0},
    )
    return Document.model_validate_json(resp.message.content)

Step 3: Check the numbers with plain code

This is where most of the reliability comes from. A model can misread a 7 as a 1. Arithmetic can catch that:

from datetime import date

def check(doc: Document) -> list[str]:
    problems = []

    if doc.line_items:
        items_sum = round(sum(i.amount for i in doc.line_items), 2)
        expected = round(items_sum + (doc.tax or 0), 2)
        if abs(expected - doc.total) > 0.05 and abs(items_sum - doc.total) > 0.05:
            problems.append(
                f"Total doesn't match the line items "
                f"(items add up to {items_sum}, total reads {doc.total})"
            )

    if doc.tax is not None and doc.tax > doc.total:
        problems.append("Tax is larger than the total")

    try:
        d = date.fromisoformat(doc.date)
        if d > date.today():
            problems.append("Date is in the future")
    except ValueError:
        problems.append(f"Date couldn't be read ({doc.date})")

    if doc.total <= 0:
        problems.append("Total is zero or negative")

    return problems

Duplicates are caught two ways: an exact file hash (the same file saved twice), and a match on vendor + date + total (the same invoice downloaded twice under different names).

import hashlib

def file_hash(path: Path) -> str:
    return hashlib.sha256(path.read_bytes()).hexdigest()

def duplicate_key(doc: Document) -> tuple:
    return (doc.vendor_or_client.strip().lower(), doc.date, round(doc.total, 2))

Any document with a problem goes to the review queue instead of the ledger. Shoebox would rather ask you than be wrong on a tax document.

Step 4: Categories that learn from you

Categorizing is personal. One designer's "equipment" is another's "office supplies." So Shoebox doesn't let the model decide categories on its own. It checks a local rules file first, and only asks the model about vendors it has never seen:

import json

RULES = Path("~/.shoebox/rules.json").expanduser()

def load_rules() -> dict[str, str]:
    return json.loads(RULES.read_text()) if RULES.exists() else {}

def categorize(doc: Document, categories: list[str]) -> tuple[str, bool]:
    rules = load_rules()
    key = doc.vendor_or_client.strip().lower()
    if key in rules:
        return rules[key], True                # known vendor: no AI needed
    suggestion = suggest_category(doc, categories)  # small local text model
    return suggestion, False                   # unknown vendor: goes to review

def remember(vendor: str, category: str) -> None:
    rules = load_rules()
    rules[vendor.strip().lower()] = category
    RULES.parent.mkdir(exist_ok=True)
    RULES.write_text(json.dumps(rules, indent=2))

Every time you confirm a category in the review queue, Shoebox remembers it. The first quarter needs a few corrections. After that, most vendors are already known, and the review queue gets shorter every time.

Those learned rules live in a plain JSON file on your laptop. You can read it, edit it, or back it up.

Step 5: Export

The ledger exports to .xlsx with pandas and openpyxl: one sheet with every document, one summary sheet by quarter, category, and client. Each row includes the original file path, so nothing is a black box.

The business: why someone would pay for this

The pitch to a freelancer is simple:

Saves a weekend every quarter. That's the whole value.
Your client data stays on your laptop. That makes it easy to use even under strict client contracts.
One-time purchase. No subscription for a task you do four times a year.

That last point only works because of how the costs are structured. If every document went through a paid API, each new customer would add an ongoing cost for me, and I'd have to charge a subscription to cover it. With an open-weight model running on the customer's machine, my cost per document is zero. Selling it once doesn't put me at risk if a customer processes thousands of documents.

My cost is development time and a small download server. That's a business a single person can run on the side.

Why open innovation matters for this project

1. Client data never leaves the laptop

This is the main selling point, and it only exists because the model is open. Invoices contain client names, project descriptions, rates, and income. With a local model, there is no server to trust, no data processing agreement to read, and nothing to explain to a cautious client. I can honestly say that Shoebox cannot leak your documents to me, because they never reach me.

2. Zero cost per document makes a one-time price possible

With a paid API, my costs would grow with every receipt every customer processes. That pushes the product toward a subscription, and toward cutting corners (smaller images, fewer checks) to save money. Running open weights on the customer's hardware takes that pressure away. I can process every page at full resolution and run every check, because none of it costs me anything.

3. No lock-in to someone else's pricing or policies

A side hustle can't survive a supplier who doubles prices or discontinues a model. With open weights, the model Shoebox ships with today will keep working forever. If a better small vision model appears, I can test it and switch. That's a one-line change, and the decision is mine.

4. I can tune it for the documents freelancers actually have

General-purpose models are trained mostly on clean, English-language documents. Freelancers have crumpled receipts, invoices in several languages, and payment screenshots from local apps. With open weights, the path to improving that is clear: collect anonymized examples from users who opt in, and fine-tune a small adapter for the document types the base model gets wrong. With a closed API, I'd just have to hope the provider improves on my niche.

5. It works offline

Tax season sometimes happens on a train, in a café with bad Wi-Fi, or on a long flight. Shoebox doesn't care: after the first model download, everything runs offline.

Where closed would have been easier (to be fair)

The largest hosted vision models are more accurate on very messy documents, and they don't need a capable laptop. A 7B vision model wants a reasonably modern machine with enough memory. Older laptops will be slow, and some won't run it at all.

I accept that trade-off, because the checks in Step 3 catch most reading errors, and the review queue handles the rest. For a freelancer, "slightly slower, and I review 3 out of 87 documents" is a much better deal than "upload all my client invoices to a server and pay monthly."

My first user

[Replace this section with what really happened when you put Shoebox in front of a real freelancer: who they are (e.g. "a freelance illustrator friend"), how many documents they ran through it, what broke, whether they paid or said they would, and what they asked for next. Real details here matter more than anything else in the post.]

Limitations

It needs a decent laptop. The vision model needs a fair amount of memory. On older machines, processing is slow.
Handwritten receipts are hit-or-miss. Printed text works well. Handwriting is unreliable and usually ends up in the review queue.
It doesn't do your taxes. Shoebox organizes documents and adds them up. It doesn't know your country's tax rules and it doesn't replace an accountant.
Currency conversion is basic. Amounts are stored in their original currency. Conversion uses an exchange-rate file you update yourself, so totals in a home currency are only as current as that file.

What's next

[ ] Watch a folder and process new documents automatically as they arrive
[ ] Export formats that common accounting software can import directly
[ ] An opt-in, anonymized way to share hard examples for fine-tuning
[ ] A smaller model option for older laptops

Try it

pip install ollama pydantic pdfplumber pandas openpyxl
ollama pull qwen2.5vl:7b
python shoebox.py scan ~/Documents/receipts --out ledger.xlsx

If you're a freelancer with a folder you've been avoiding, I'd love for you to try it and tell me what breaks.

Then close the laptop. Your weekend is free. 🧾

Built with: Ollama, Qwen2.5-VL (open weights), pdfplumber, Pydantic, pandas, openpyxl, Python.

I turned my grandmother's voice messages into a family cookbook (open Vietnamese speech model + local LLM)

Zhao Yang — Sun, 11 Oct 2026 18:31:45 +0000

This is a submission for the Build for a Friend challenge: ship something with open-source AI that solves a real problem for someone you love.

Grandma's Kitchen turns my grandmother's voice messages into a printed family cookbook.

She talks, the way she always has. An open speech model fine-tuned for Vietnamese transcribes her recordings on my laptop. A small local LLM turns each rambling, wonderful memo into a structured recipe: ingredients, steps, and her tips kept in her own words. Then it produces a printable PDF, plus a list of questions for us to call and ask her about whatever she left out.

Nothing is uploaded anywhere. Her voice stays in the family.

The person: my grandmother

My grandmother is the reason our family gathers. Every Lunar New Year, every death anniversary, every Sunday that anyone can make it, it's her food on the table.

She has never used a measuring spoon. Ask her how much fish sauce goes into the braised fish and she'll say "enough," then add "until it smells right." Ask how long to simmer and she'll tell you "until the color is like this," while pointing at the pot.

None of it is written down. Her recipes exist in exactly two places: her hands and her memory.

Our family is spread out now. Some of us are in Hanoi, some in the south, and a few cousins are abroad and read Vietnamese slowly. Everyone wants to cook her food. Nobody can.

The one thing she does do is send voice messages. Long ones. If you ask her a cooking question in the family chat, she'll answer with a three-minute recording that wanders from the recipe to which market sells the best shallots to a story about my grandfather. Those recordings were already the cookbook. They just weren't usable.

So I built something to make them usable, without asking her to change anything.

What I built

The rule I set for myself: Grandma doesn't learn any new technology. She keeps sending voice messages the way she already does. Everything else is my job.

The pipeline:

 Voice messages            Local, on my laptop                         Output
 (as she always sends)
┌──────────────┐   ┌──────────┐   ┌──────────────┐   ┌───────────┐   ┌──────────────┐
│ .m4a / .aac  │──▶│  ffmpeg  │──▶│  PhoWhisper  │──▶│ Local LLM │──▶│  Validator   │
│ from family  │   │ 16kHz wav│   │ (Vietnamese  │   │ (Ollama)  │   │ "did she     │
│ chat export  │   │          │   │  speech→text)│   │ structure │   │  really say  │
└──────────────┘   └──────────┘   └──────────────┘   └───────────┘   │  that?"      │
                                                                     └──────┬───────┘
                                                                            ▼
                                                         ┌────────────────────────────┐
                                                         │ 📖 Printable cookbook (PDF)│
                                                         │ ❓ "Ask Grandma" call list │
                                                         └────────────────────────────┘

Each recipe page in the book has:

The dish name and roughly how many people it feeds
Ingredients, with what she said ("a handful of shallots") next to a rough estimate ("≈ 5–6 shallots"), clearly marked as an estimate
Steps in order, cleaned up but faithful
"Grandma says": her best lines, quoted directly from the transcript, because "don't stir it, you'll make the fish sad" is not something a recipe card should lose
An **English

Monsoon: a garden planner for places that never see frost (local LLM + open weather data)

Zhao Yang — Sun, 11 Oct 2026 17:06:36 +0000

This is a submission for the Touch Grass challenge: build something with open-source AI that gets people off the screen and into the world.

Monsoon is a garden planner for tropical and subtropical gardeners. Once a week it looks at your location, the 7-day forecast, and a local crop calendar. Then it hands you one short card: what to sow, transplant, or protect this week, and why.

It runs on an open-weight model on your own machine. Your garden, your location, and your questions never go to someone else's server. The only network call is a weekly weather fetch from an open-source weather API.

The whole interaction is designed to take less time than watering a single pot.

The problem: every garden app assumes you have a winter

Search "what should I plant this week" and you'll find a lot of excellent tools. Almost all of them are built around one number: your last frost date. Count back six weeks, start your tomatoes indoors, transplant after the frost passes.

That works well in Ohio. It makes no sense in Hanoi, Bangkok, Manila, Lagos, or Recife.

I garden on a balcony in Hanoi. Our year isn't split by frost. It's split by heat, rain, and the cool dry window between them:

Summer (roughly May–September): hot, humid, and heavy downpours. This is the season for water spinach, amaranth, gourds, and okra. Most cool-season greens bolt or rot.
Autumn into winter (roughly September–December): the start of the winter crop season. Farmers in the Red River Delta plant kohlrabi, cabbage, mustard greens, tomatoes, and potatoes now. It's October as I write this, so this window is open right now.
Late winter (January–February): cool and drizzly, with occasional cold snaps below ~10 °C that can stunt heat-loving plants.
Spring (March–April): warming up again, and time to switch back to summer crops.

Then there's the far north of the country: up in the mountains around Sapa, frost really does happen in some winters. So a planner for Vietnam can't just drop frost. It has to treat frost as one risk among several: heavy rain, heat stress, cold snaps, and frost where it applies.

Farmers here have handled this for centuries with seasonal crop calendars, which agricultural extension services now publish by province. But those calendars are written for fields, in PDFs. They don't help someone with six pots on a balcony who needs to know: should I sow the mustard seeds today, or will Thursday's storm wash them out?

That gap is what I built Monsoon to fill.

What I built

Monsoon does three things:

1. The weekly card (the main feature)

Every Sunday evening, Monsoon generates one card:

🌱 MONSOON — Week of 12 Oct · Hanoi

DO THIS WEEK
→ Sow mustard greens — Mon or Tue
  Cool season has started and the soil is still warm.
  Get them in before Thursday: ~35 mm of rain is forecast,
  which can wash out fresh seed.

ALSO GOOD
→ Transplant tomato seedlings in the evening (highs still ~31 °C)

SKIP
→ Water spinach — its season is ending; new sowings will be weak.

Next check: Sunday 19 Oct

That's the whole interface. You read it in about 15 seconds, then you go outside and do it.

The card can go to a Telegram message, a printable page, or plain text you stick on the fridge. I deliberately didn't build a dashboard with charts and streaks. Engagement is not the goal. Dirt under your fingernails is the goal.

2. Ask a question (optional)

Sometimes you need more than the card: "My kohlrabi leaves have holes in them, what now?" or "Can I grow basil through December?" You can ask the local model directly. It answers using your location, the season, and the same crop knowledge base the card uses.

3. Your garden profile

A small local file describes what you've got: balcony or ground, sun hours, what's currently planted, and roughly when you planted it. It never leaves your machine.

location:
  name: Hanoi
  lat: 21.03
  lon: 105.85
space: balcony
sun_hours: 5
planted:
  - crop: tomato
    stage: seedling
    date: 2026-09-20
  - crop: basil
    stage: established
    date: 2026-06-01

How it works

The most important design decision: the LLM is not in charge of agronomy.

Small models are good at language and reasoning over facts you hand them. They are not reliable sources of planting windows for a specific province. If you ask a small model cold, "when do I plant potatoes in Hanoi?", you might get a confident answer copied from temperate-climate advice.

So Monsoon splits the work in two:

┌──────────────────┐   ┌──────────────────┐   ┌───────────────────┐
│  Crop calendar   │   │ Weather forecast │   │  Garden profile   │
│  (curated JSON)  │   │  (Open-Meteo,    │   │  (local YAML)     │
│                  │   │   cached weekly) │   │                   │
└────────┬─────────┘   └────────┬─────────┘   └─────────┬─────────┘
         │                      │                       │
         └──────────────┬───────┴───────────────────────┘
                        ▼
            ┌───────────────────────┐
            │  Rules engine (Python)│   ← deterministic: what's *allowed*
            │  candidate tasks +    │
            │  weather flags        │
            └───────────┬───────────┘
                        ▼
            ┌───────────────────────┐
            │  Local LLM (Ollama)   │   ← language: what *matters most*,
            │  rank + explain,      │     and how to say it
            │  JSON-schema output   │
            └───────────┬───────────┘
                        ▼
            ┌───────────────────────┐
            │  Validator            │   ← rejects anything not in the
            │                       │     candidate list
            └───────────┬───────────┘
                        ▼
                  🌱 Weekly card

The rules engine decides what is possible. The model decides what matters most this week and how to say it. A validator makes sure the model can't invent a crop or a task that the rules didn't allow.

Step 1: A crop calendar as data, not vibes

Each crop is a small JSON record. Planting windows are stored per climate zone, so northern lowlands, the northern mountains, and the south each get their own windows.

{
  "id": "mustard_greens",
  "name": "Mustard greens",
  "zones": {
    "north_lowland": {
      "sow": [{ "start": [9, 1], "end": [2, 28] }]
    },
    "north_mountain": {
      "sow": [{ "start": [8, 15], "end": [10, 31] }],
      "frost_sensitive": false
    },
    "south": {
      "sow": [{ "start": [11, 1], "end": [3, 31] }]
    }
  },
  "seed_washout_risk": true,
  "max_temp_for_sowing": 32,
  "days_to_harvest": [30, 45],
  "containers_ok": true
}

Windows like September → February wrap around New Year, so the window check has to handle that:

from datetime import date

def in_window(today: date, window: dict) -> bool:
    start = date(today.year, *window["start"])
    end = date(today.year, *window["end"])
    if end < start:  # window wraps over New Year, e.g. Sep -> Feb
        return today >= start or today <= end
    return start <= today <= end

I seeded the calendar from published seasonal crop guidance and general horticulture references, then simplified it for home and container growing. This is the part of the project I'd most like other people to contribute to. It's a plain JSON file, so a gardener in Chiang Mai or Kerala can add their crops without touching any code.

Step 2: Weather from an open API, fetched once a week

Open-Meteo is an open-source weather API. It needs no API key, and you can self-host it if you want. Monsoon fetches a 7-day forecast once a week and caches it to disk:

import json
import pathlib
import requests

CACHE = pathlib.Path("cache/forecast.json")

def fetch_forecast(lat: float, lon: float) -> dict:
    try:
        r = requests.get(
            "https://api.open-meteo.com/v1/forecast",
            params={
                "latitude": lat,
                "longitude": lon,
                "daily": "temperature_2m_max,temperature_2m_min,precipitation_sum",
                "timezone": "auto",
                "forecast_days": 7,
            },
            timeout=10,
        )
        r.raise_for_status()
        daily = r.json()["daily"]
        CACHE.parent.mkdir(exist_ok=True)
        CACHE.write_text(json.dumps(daily))
        return daily
    except requests.RequestException:
        # No signal? Use last week's forecast and say so on the card.
        if CACHE.exists():
            return json.loads(CACHE.read_text()) | {"stale": True}
        return {"stale": True, "time": []}

If there's no connection, Monsoon still works. It falls back to the cached forecast (or to the calendar alone) and tells you so on the card.

Step 3: The rules engine turns weather into flags

A few simple heuristics cover most of what matters in a tropical garden:

def weather_flags(daily: dict) -> list[dict]:
    flags = []
    for day, rain, tmax, tmin in zip(
        daily.get("time", []),
        daily.get("precipitation_sum", []),
        daily.get("temperature_2m_max", []),
        daily.get("temperature_2m_min", []),
    ):
        if rain is not None and rain >= 20:
            flags.append({"day": day, "type": "heavy_rain", "mm": rain})
        if tmax is not None and tmax >= 32:
            flags.append({"day": day, "type": "heat", "c": tmax})
        if tmin is not None and tmin <= 10:
            flags.append({"day": day, "type": "cold_snap", "c": tmin})
        if tmin is not None and tmin <= 2:
            flags.append({"day": day, "type": "frost_risk", "c": tmin})
    return flags

The thresholds are rules of thumb, not science. Heavy rain means don't sow small seeds right before it. Heat means transplant in the evening and shade seedlings. A cold snap means cover basil and peppers. Frost risk only really fires in the mountains, which is exactly the point.

The engine then crosses three things: crops that are in their window, crops already in your garden, and these flags. The result is a short list of candidate tasks, each with an ID and the facts behind it.

Step 4: The local model ranks and explains

The model gets the candidate list, the weather flags, and your garden profile. Its job is to pick the single most important task, add one or two extras, flag anything to skip, and explain each choice in plain language.

I run it with Ollama and use structured output: the response must match a JSON schema, so I never have to parse free text.

import ollama
from pydantic import BaseModel

class Task(BaseModel):
    task_id: str
    when: str
    why: str

class WeekCard(BaseModel):
    do_this: Task
    also_good: list[Task]
    skip: list[Task]

SYSTEM = """You are a calm, practical gardening helper for a home gardener.
You may ONLY choose tasks from the provided candidate list, using their exact task_id.
Use the weather flags to decide timing. Keep every 'why' under 30 words.
Never invent crops, dates, or numbers that are not in the input."""

def make_card(candidates: list[dict], flags: list[dict], profile: dict) -> WeekCard:
    resp = ollama.chat(
        model="qwen3:4b",
        messages=[
            {"role": "system", "content": SYSTEM},
            {"role": "user", "content": json.dumps({
                "candidates": candidates,
                "weather_flags": flags,
                "garden": profile,
            })},
        ],
        format=WeekCard.model_json_schema(),
        options={"temperature": 0.2},
    )
    return WeekCard.model_validate_json(resp.message.content)

Step 5: Trust, but verify

Even with a schema, the model could return a task_id that isn't in the candidate list. So a validator checks every task. Anything unknown gets dropped, and if the main task is invalid, Monsoon falls back to the top-ranked candidate from the rules engine:

def validate(card: WeekCard, candidates: list[dict]) -> WeekCard:
    allowed = {c["task_id"] for c in candidates}
    card.also_good = [t for t in card.also_good if t.task_id in allowed]
    card.skip = [t for t in card.skip if t.task_id in allowed]
    if card.do_this.task_id not in allowed:
        top = candidates[0]
        card.do_this = Task(task_id=top["task_id"], when="this week", why=top["reason"])
    return card

This is the piece that made me comfortable letting a small model write advice. It can phrase things badly, but it can't tell you to plant something that's out of season.

Choosing the model

I wanted three things: it runs on an ordinary laptop, it handles multiple languages well, and it follows a JSON schema reliably. A small Qwen3 model fits all three, and its open weights mean I can fine-tune it later.

A card is generated once a week, so even a slow machine is fine. Speed simply doesn't matter much for this use case.

Because the model is behind a single function call, swapping it is a one-line change. If a better small multilingual model comes out next month, I change the model name and keep going.

Why open innovation matters for this project

The challenge asks where the open approach worked better than a closed one. For Monsoon, the honest answer is that almost every key property of the project depends on it being open.

1. My garden is my data

A garden profile looks harmless, but it contains your precise location, when you're home, what you grow, and over time, a log of your habits. A planner that's useful every week for years builds up exactly the kind of long-term personal data that I don't want sitting on a server I don't control.

With a local model, the only thing that ever leaves my machine is a latitude/longitude pair sent to a weather API. And because Open-Meteo is open source, even that can be self-hosted.

2. It costs nothing to run, forever

A weekly card for one gardener is tiny. But a free tool for a whole community garden, a school, or a farmers' cooperative shouldn't depend on someone paying for API calls every month. Run on a spare laptop or a small single-board computer, Monsoon's running cost is effectively the electricity. Nobody has to keep paying for it to keep working.

3. It works when the internet doesn't

Gardens are often outside the strongest Wi-Fi, and rural connectivity can be patchy. Monsoon needs the network for exactly one request a week, and degrades gracefully without it. The model, the calendar, and your history are all local.

4. I can make it local in the cultural sense too

This is the part I care about most. Being open doesn't just mean "runs locally." It means I can adapt the system to a place:

The knowledge base is a plain, editable file. A gardener in Chiang Mai can add Thai crops and zones. A grower in Kerala can add monsoon timings for their region. None of that needs anyone's permission.
The model can be changed or fine-tuned. The next step I'm exploring is a small LoRA adapter trained on regional agricultural extension material, so the model uses the same terms and crop variety names local farmers use instead of generic, temperate-climate gardening language.
The prompts and rules are visible. If the advice is wrong, you can see why. Was it the calendar, the weather rule, or the model's phrasing? Then you can fix the right layer.

A closed, hosted model treats "Hanoi" as an edge case of a system built mostly around temperate assumptions. An open system lets the people who actually live in a place define what normal looks like.

Where closed would have been easier (to be fair)

Closed frontier models are better at answering open-ended plant questions from general knowledge. For the "ask a question" feature, a bigger hosted model would give richer answers.

But Monsoon's core loop doesn't need a genius. It needs a model that follows a schema, reads a short list of facts, and writes three clear sentences. Small open models are well suited to that. And splitting the system into rules + model + validator means the model's weaknesses are contained.

Making the screen the shortest part

The theme asks for builds where the screen is the shortest part of the experience. Here's how I approached that:

Design choice	Why
One card per week, not a feed	Nothing to scroll, nothing to check daily
One main task on each card	Easy to remember once you're outside
Timing built in ("Mon or Tue, before the rain")	You don't need to open a weather app separately
Printable / message-based output	The card can live on the fridge, not in an app
No streaks, badges, or notifications beyond the weekly card	No pressure to come back to the screen

About 15 seconds of reading per week. Everything else happens in the garden.

Limitations

The crop calendar is the bottleneck. The output is only as good as the windows in the JSON, and coverage is still limited to common vegetables and herbs.
Microclimates are hard. A balcony on the 12th floor facing west is not the same as a ground-level garden in the same district. The sun_hours and space fields help, but only a bit.
Weather thresholds are generic. 20 mm of rain is a rough cutoff. Different crops and soils react differently.
Small models still drift. Without the validator, a small model can suggest tasks that aren't in the list or get timing wrong. The validator catches those mistakes, but the explanations can still be bland.
This is not expert agronomic advice. It's a nudge for home gardeners. For pests and diseases, the card points you to local extension services or experienced growers.

What's next

[ ] More crops and zones, especially Southeast Asia and South Asia
[ ] A LoRA fine-tune on regional agricultural extension material
[ ] A simple "what I actually did" log, so next year's cards learn from this year
[ ] A community-garden mode: one card for a shared plot, sent to everyone in a group chat
[ ] Running it on a cheap single-board computer that prints the card on a small receipt printer every Sunday

Try it

pip install requests ollama pydantic pyyaml
ollama pull qwen3:4b      # or any model you prefer
cp garden.example.yaml garden.yaml   # edit with your location and plants
python monsoon.py card

If you garden somewhere without frost, I'd love your help with the crop calendar. Share your region's crops and planting windows in the comments.

Now close the laptop and go check on your seedlings. 🌱

Built with: Ollama, Qwen3 (open weights), Open-Meteo, Python, Pydantic.