This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady
What I Built
Sloth Fridge turns one photo of your fridge into a grocery list and a dinner plan, so you never have to stand there with the door open again.
You take a photo. It lists what's inside, asks you to fix anything it got wrong, then suggests what to buy and what to cook, including one "sloth pick": the laziest meal you can make right now.
I built it for hostel life. It knows my kitchen is a kettle and an induction plate, that I'm vegetarian, and that I have a budget. [Replace this with your own real moment, for example: "Last week I stood at the mess fridge at 11 pm for five minutes and still didn't know what to cook."]
Everything runs on my laptop. No cloud, no API bill, no photos leaving my machine.
Demo
In the video: [one line on what it shows, for example "a real photo of my hostel fridge goes in, I fix one wrong item in the table, and the grocery list and sloth pick come out. Everything runs offline on my laptop."]
Code
🦥 Sloth Fridge
Take one photo of your fridge. Get back what you have, what to buy, and what to cook, without standing there with the door open.
Everything runs locally on open models through Ollama. No cloud, no API bills, and your fridge photos never leave your laptop. Built for the DEV Hacktoberfest "Slow and Steady" challenge.
Why open models
- Private: photos of your room and food stay on your machine.
- Free to run: set it up once, never pay per scan.
- Works offline: no internet needed after the models are downloaded.
- Swappable: change the vision or text model with one environment variable and benchmark them against each other.
How it works
photo ─► [Pass 1: vision model] lists distinct items + freshness guess
─► [Pass 2: vision model] one focused "how many X?" question per item
─► [You] fix anything wrong in an editable table
─► [TextHow I Built It
Stack: Ollama running qwen2.5vl:3b (vision) and llama3.2:3b (text), Streamlit for the UI, SQLite for memory, Pydantic for strict JSON output. All of it runs on an RTX 4050 laptop GPU with 6 GB of VRAM.
The pipeline has four steps:
- Pass 1, describe. The vision model lists the distinct items it can see, as schema-constrained JSON.
- Pass 2, count. For each item, the vision model gets one focused question: "How many separate tomatoes are visible?"
- You correct. An editable table lets you delete wrong items, add missed ones, or fix counts.
- Plan. A separate text model takes the corrected list, your usual items, and your constraints, and returns a grocery list, a few meals, and the sloth pick.
Small models are decent at spotting things and weak at reasoning, so the vision model only describes and a second model plans.
What broke, and what I did about it:
- Empty answers. My first vision model returned nothing at all. It looked like a "thinking" variant that spent its whole budget reasoning. I switched to a non-thinking model and added a clear error message for empty responses.
- Runaway loops. The next model got stuck repeating items until it produced a 1,300-line JSON that was cut off mid-item. Fixes: a hard cap on output tokens, a small repeat penalty (pure greedy decoding made loops more likely), a prompt that says "mention each item once", and a salvage parser that recovers every complete item from broken output.
- Bad counts. Asking one model to list and count everything in a single JSON answer gave confident wrong numbers. Switching to coarse levels (one / few / many) was honest but too vague, so I added the second pass with one counting question per item. It's slower, but numbers are only filled in when the model gives a sensible answer. Otherwise the field stays blank rather than wrong.
- It learns from corrections. Every item you delete or add is logged in SQLite. Delete the same false item twice and it's filtered from future scans.
Why Does Open Innovation Matter?
Four things mattered for this project, and a hosted API would have changed each of them:
- Privacy. A photo of your fridge is a photo of your room and your life. Here it never leaves the laptop.
- Cost. Set it up once and scanning is free forever. For something I'd use daily, per-scan pricing would defeat the point of being lazy.
- Offline. After the models are downloaded, it needs no internet, which matters on patchy hostel Wi-Fi.
- Control. When models misbehaved, I could change the things that fixed them: swap models with one environment variable, constrain the output with a JSON schema, cap tokens, and tune sampling. Several of my fixes above came from that control.
Where closed models still win: [Fill in after your benchmark. For example, whether a closed model counted or recognized items better, and by how much.] For this project, the trade was still worth it, because [your honest reason].
| Model | Item recall | Precision | Avg seconds / photo | Offline | Cost per scan |
|---|---|---|---|---|---|
| qwen2.5vl:3b | [ ] | [ ] | [ ] | Yes | Free |
| [second open model] | [ ] | [ ] | [ ] | Yes | Free |
| [closed model] | [ ] | [ ] | [ ] | No | [ ] |
Measured on [N] real fridge photos on an RTX 4050 laptop GPU (6 GB).
My Agent Session
[Optional: embed or link your DevRelay session here. If you didn't record one, delete this section.]
The Lazy Payoff
[Only keep this if it's true. For example: "I used it on my hostel fridge for [N] days. Deciding what to cook went from about [X] minutes to about [Y]." If you haven't used it yet, delete this section.]
Prize Categories
[Only list a category if you actually used that partner's technology. If not, delete this section.]
Top comments (0)