This is a submission for the Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle
What I Built
MenuMate turns a photo of a restaurant menu into a translated menu with allergen and diet tags on every dish.
Small restaurants lose customers because tourists and newcomers can't read the menu. Professional translation is expensive, and Google Translate mangles dish names. With MenuMate, the owner snaps a photo, picks the languages, and gets a clean, printable menu card in a minute. Each dish gets tags like vegetarian, vegan, gluten, dairy, nuts and shellfish.
Who would pay: small restaurants and cafes, at a small monthly fee, or a one-time fee per menu update.
Demo
Code
MenuMate
Photo of a menu in, translated menu with allergen tags out. Powered by Gemma (open-weight).
Run locally with Ollama (no key needed)
- ollama pull gemma3:4b (any vision-capable gemma3 works)
- pip install -r requirements.txt
- python -m uvicorn app:app --reload
- Open http://127.0.0.1:8000
Use a hosted Gemma instead
Set GEMMA_API_KEY (free from aistudio.google.com) before step 3.
How I Built It
- Model: Gemma 3 (open-weight, vision-capable), running locally through Ollama. It reads the menu image directly, so there's no separate OCR step.
- Backend: a small FastAPI app that sends the photo and a strict prompt to Gemma, then parses the JSON it returns (dish name, price, tags, translations).
- Frontend: one HTML file that renders the result as a menu card with a language tab for each translation, plus print/PDF and copy buttons.
- Switchable: if no API key is set, the app finds the Gemma model on your machine automatically. If a key is set, it uses hosted Gemma instead.
I built this with AI assistance (Claude), and tested it by running it on my own laptop with real menu photos.
Why Does Open Innovation Matter?
- Privacy: menus, prices and customer details never leave the restaurant's own machine.
- Cost: there's no per-request fee, so a few dollars a month can actually be a profitable price for a tiny business.
- No lock-in: I can swap Gemma for another open model, or change hosts, without rewriting the app.
- Customizable: because the weights are open, the model can later be fine-tuned on a specific cuisine's dish names, which generic APIs handle badly.
My First User
I showed MenuMate to Rina, who runs a small family cafe near my house. She has a handwritten menu, about 20 items, and gets a lot of tourists on weekends who can't read it.
I photographed her menu and let her press Translate herself, choosing Spanish and French. It took about a minute on my laptop.
What she liked: She read the Spanish version and said it sounded like how the dishes are really described, not like a literal translation. She especially liked the allergen tags, because customers ask her about nuts and dairy all the time.
What went wrong: It misread one dish name written in cursive and tagged a plain chicken sandwich as containing dairy. She spotted both mistakes right away, which showed me the tags need a human check before printing.
Would she pay? She said she'd pay a small monthly fee, around $20, if she could update the menu herself when specials change.
What she asked for next: A way to edit a dish before printing, and a QR code tourists could scan to see the menu in their own language.
Prize Categories
- Best Use of Gemma
Top comments (0)