What's That Green? — Plant identifier

🌿 What's that Green Thing — Instant Plant Identification

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

---

What I Built

"What's that Green Thing" is an AI botanical companion designed to do one simple, joyful thing: get you outside, exploring the living world, and touching actual grass.

Modern life traps us behind monitors and glowing rectangles. We walk past hundreds of fascinating plant species every day in parks, sidewalk cracks, hiking trails, and backyards—often without knowing what they are, how they survive, or how to nurture them.

What's that Green Thing bridges the gap between technology and the physical world. Users can step outside with their phone or laptop, point their camera at any mysterious leaf, flower, or succulent, and instantly receive:

• Common & Botanical Latin Names: Spot-on taxonomy to bridge everyday curiosity with real botanical science.
• Estimated Confidence Score: Transparent certainty ratings on every specimen analyzed.
• Engaging 2-Sentence Field Summary: An approachable look at the plant’s origins, adaptations, and natural habitat.
• Essential Care & Habitat Guidelines: Quick, actionable insights for Water 💧, Sunlight ☀️, and Soil 🌱.

Whether you are an urban forager, weekend hiker, budding gardener, or just someone looking for an excuse to take a walk during a coding break, this tool turns every stroll into an interactive nature walk.

---

Demo

• Local Preview: Run locally in seconds with python app.py at http://localhost:5001.

Highlights

• 📷 Live Viewfinder: Stream direct camera feed with an interactive specimen reticle and one-tap shutter capture.
• 📁 Drag-and-Drop: Instant upload supporting JPEG, PNG, and WebP images up to 16 MB.
• ⏳ Botanical Loading Ticker: Cycles through playful nature-themed status updates ("Consulting the forest spirits...", "Analyzing leaf venation patterns...").
• 📋 Botanical Field Card: Displays a clean presentation card with a one-click "Copy Botanical Card" button.

---

Code

The complete source code is open source and available on GitHub:

👉 https://github.com/ssindher11/whats-that-green

Architecture Overview

• app.py: Lightweight Python/Flask server utilizing the official google-genai SDK with Pydantic structured schemas, image pre-processing (Pillow), and automated model resilience.
• templates/index.html: Zero-build frontend combining Tailwind CSS (via CDN) and vanilla JavaScript for camera streams, canvas capture, and responsive botanical UI cards.
• requirements.txt: Minimal, clean dependencies (flask, google-genai, python-dotenv, pillow).

---

How I Built It

1. Multimodal AI & Structured Outputs

The intelligence engine is built on Google Gemini 3.8 Flash via the official google-genai Python SDK. We configure Gemini with a specialized botanical persona and enforce rigid JSON schemas via Pydantic:

class CareTips(BaseModel):
    water: str = Field(description="Watering requirements and schedule")
    light: str = Field(description="Sunlight conditions")
    soil: str = Field(description="Soil composition and potting medium")

class PlantIdentification(BaseModel):
    is_plant: bool = Field(description="True if image contains botanical specimen")
    plant_name: str = Field(description="Common name of the plant")
    scientific_name: str = Field(description="Botanical Latin binomial")
    confidence: str = Field(description="Estimated confidence percentage")
    description: str = Field(description="Engaging 2-sentence botanical summary")
    care_tips: CareTips

2. High-Effort Reasoning (thinking_config)

Plant morphology can be subtle—differentiating lookalike cultivars or venation patterns requires more than superficial classification. By passing:

thinking_config=types.ThinkingConfig(thinking_budget=1024)

the model reasons through leaf margins, stem nodes, and petiole structure prior to synthesizing the final diagnostic report.

3. Native Browser Camera Integration

Rather than requiring a native mobile app download, the app leverages the browser’s native navigator.mediaDevices.getUserMedia API. Users can frame specimens directly on their device screen, flip between front and rear cameras, and snapshot frames directly onto an HTML5 canvas for immediate analysis.

---

Why Does Open Innovation Matter?

Open innovation is essential for environmental literacy and real-world technology:

1. Democratizing Biodiversity Education: Nature identification tools shouldn't be locked behind aggressive paywalls or subscription pay-traps. Open innovation lets anyone build accessible tools for students, community gardens, and nature lovers.
2. Extensibility & Local Resilience: Open-weight models and transparent SDKs empower developers to customize schemas, support regional plant databases, or adapt tools to run offline on lightweight field devices.
3. Encouraging Real-World Curiosity: When developers have open access to cutting-edge multimodal vision models, we can craft creative applications that motivate people to unplug, step outdoors, and connect with the natural world around them.

---

My Agent Session

This application was developed pair-programming with Antigravity, leveraging agentic coding workflows to:

• Rapidly scaffold the Flask backend and implement the Google GenAI SDK integration with Pydantic structured schema outputs.
• Build and polish the responsive Tailwind CSS single-page interface with live webcam controls and loading animations.
• Perform end-to-end multimodal testing against real botanical photos and verify edge cases (e.g., distinguishing real foliage from illustrations and non-plant objects).

---

Prize Categories

• Primary Track: Touch Grass (Encouraging real-world outdoor engagement, nature exploration, and unplugged curiosity)
• Multimodal AI / Vision Track: Leveraging multimodal reasoning for practical, real-world visual identification.