YardWizard: snap your empty corner, then go plant something real

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

What I Built

I photographed an empty lawn and my AI told me it could see tomato plants and basil.

There were no tomatoes. There was no basil. There was grass.

That moment shaped the whole design of YardWizard, an app that looks at a photo of your balcony, backyard or lawn and suggests flowers and decorative plants that are easy to buy locally and right to plant this month. It has plant lists for India, Australia, the USA and the UK.

The screen part takes about 30 seconds: upload a photo, read the suggestions. After that the app has done its job. The real work is walking to a nursery, picking up a pot of crossandra or a bag of daffodil bulbs, and getting your hands in the soil. Every suggestion ends in something you can physically go and buy this week, and each card tells you whether to plant it now or wait, and when.

It's for people who want a garden but don't know where to start, especially people with a bare balcony or an empty patch of lawn and no idea what suits it.

Demo

Code

jobs-code / yardWizard

🌱 YardWizard

Upload a snap of your balcony, backyard or lawn and get flowers and decorative plants that are easy to buy locally and right to plant this month.

Built for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass. The screen part takes about 30 seconds. After that, you go to a nursery and plant something.

How it works

1. You pick your region and upload a photo of your space.
2. Gemma 3 4B (vision), running locally through Ollama, reads the photo and returns a small JSON profile: sunlight (full / part / shade), space type (pots / ground), visible plants, and a one-line note.
3. A hand-written plant list for your region does the actual recommending, filtered by sun, space and the current month. Each card shows plant now or next: .
4. If the model misjudges the light, change the dropdowns and tap Suggest plants again.

The…

View on GitHub

How I Built It

The pieces are deliberately small:

• Gemma 3 4B (vision), running locally through Ollama. It looks at the photo and returns a tiny JSON profile: how sunny the space looks, whether it's pots or open ground, what plants are visible, and one friendly sentence about the space.
• A hand-written plant list per region, with sun needs, pots-or-ground suitability, and the months each plant is good to start. This list does the actual recommending, filtered by the current month.
• Flask and a single HTML page. No framework, no database, no API key.
• Dropdowns to correct the model. If Gemma misjudges the light, the user changes it with one tap and re-runs.

I split the work like this on purpose. A 4B model is good at "what kind of space is this?" and unreliable at "which plant should I tell someone to buy in Kerala in October?" So the model reads the scene, and a list I wrote makes the recommendation. The model can't suggest a plant that doesn't exist or isn't sold locally, because it never picks the plants.

I also tried the text-only qwen3 I already had installed, but it can't read images, so I pulled Gemma 3 for the photo step. This is my second project on the same local Gemma setup, after the workout planner I built last week.

What worked and what didn't

I tested eight photos across the regions:

Gemma got the overall scene right every time. But on lawns it twice invented crops that weren't there, once tomatoes and basil, once just "vegetables". Small vision models seem to like seeing what a garden "should" contain.

My first fix backfired. I told the model to list only plants that were clearly visible, and it copied my instruction into its answer. Two photos came back saying "I can see: only plants clearly visible." Small models follow examples more than rules. The fix was to remove the instruction from the JSON template, rephrase it, and add a small code-side filter that throws away echoed instructions. Now the model's mistakes stay inside the model.

That is the argument for the architecture. The balcony, the most common real use case, worked cleanly every time. The lawn, where the model hallucinates, can't do any damage, because Gemma doesn't choose the plants and the user can correct the light and space in one tap.

Limits I should be upfront about:

• Judging sunlight from a single photo is rough. The dropdowns exist because of this.
• The plant lists are a starting point. The USA is far too varied for one list, so mine covers temperate regions and the Deep South or the desert would need their own. Australia and India vary a lot too, and a local nursery knows your microclimate better than I do.
• Offline here means no internet at all on the laptop. I did not run the model on a phone, and that is the next step.

Why Does Open Innovation Matter?

• Photos of your home never leave your machine. A picture of someone's balcony or backyard says a lot about where they live. Nothing is uploaded anywhere.
• It works with no internet. I recorded the demo with Wi-Fi off. That matters for a gardening tool, since gardens are often far from good signal.
• I could see and fix the model's behavior. When Gemma hallucinated, I could change the prompt, filter the output, and rebuild the design around the model's weak spot.
• Adding a country is just adding a list. USA and UK took minutes to add, with no retraining and no API changes. Anyone can add their own district, or swap in a different model.

What's Next

Running the model directly on a phone, local language plant names, more regions, and a "what's flowering near me" feed that sends you outside to look at the real thing.

Prize Categories

Best Use of Gemma