DEV Community

SHAIK MUSTAFA ALI
SHAIK MUSTAFA ALI

Posted on Originally published at github.com

Soil & Compost Doctor: The Offline Open-Source AI That Gets Your Hands in the Dirt

#devchallenge #hf26challenge #opensource #python

Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass Submission 🌿

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

What I Built

Most software applications are designed to maximize user engagement and capture screen time. Soil & Compost Doctor was built with the exact opposite philosophy: keep screen time strictly under 10 seconds per interaction so people can put their phones away and literally touch grass.

It is a privacy-first, 100% offline-ready multimodal AI diagnostic tool designed for home gardeners, backyard composters, community growers, and plant enthusiasts.

What it does:

  1. Multimodal Visual Diagnosis: Identifies crop & plant foliar diseases (e.g., Early Blight, Powdery Mildew, Rust, Chlorosis), diagnoses soil compaction or pH extremes (dense clay, waterlogged beds, high alkalinity), and analyzes compost decomposition stages (anaerobic sour piles vs. cured black gold humus).
  2. Immediate Physical Action Items: Instead of dumping walls of theoretical plant biology, it delivers one tangible, real-world outdoor task (e.g., "Prune the bottom 4 inches of infected leaves immediately and mulch with clean straw" or "Add 3 buckets of dry brown autumn leaves and turn the compost pile with a pitchfork").
  3. Hands-Free Voice Summary: Gardeners have dirty gloves and mud on their fingers. Using the native Web Speech API, the app reads the diagnosis and physical task aloud so users never have to smudge their screen with wet soil.
  4. 10-Second Screen Stopwatch: A built-in timer measures the diagnostic cycle, reminding users to slip their device into their pocket and get their hands back in the soil.

Demo

🔗 GitHub Repository & Full Setup: https://github.com/Mustafa1765/Soil-Compost-Doctor

💻 Local Web App Interface: http://127.0.0.1:5000 (Runs 100% locally on localhost with zero cloud dependency)

The 10-Second Outdoor User Flow: 

 [ In The Garden / Dirt on Hands ]
                │
         1. Snap Specimen 📸
                ▼
  [ Soil & Compost Doctor Web UI ]
                │
  • ChromaDB Local RAG (Agricultural Extension Data)
  • Ollama Llama 3.2 Vision (100% Local Inference)
                │
                ▼ (⚡ 2.0s Latency)
   [ 🔊 Speaks Diagnosis Aloud ]
   "Your plant has Early Blight. Prune lower foliage now and mulch."
                │
         2. Put Phone in Pocket
                ▼
    👉 [ Touch Grass & Garden! ]
Enter fullscreen mode Exit fullscreen mode

Real-World Diagnostic Output in the Field:

  • Specimen: Diseased Tomato Leaf
  • Category: Leaf Health
  • Diagnosis: Early Blight (Alternaria solani)
  • 🚜 Immediate Outdoor Action Item: > "Prune off all affected lower leaves within 6 inches of the soil immediately and bury them away from your garden or discard in trash (do not compost)."
  • 🔊 Hands-Free Voice Summary: > "Your plant has Early Blight. Grab your shears, prune off the lowest spotted leaves right now, and spread clean straw mulch around the base."
  • Interactive Checklist: [✓] I did this physical task in the garden! 🎉 Hands back in the dirt. Phone put away.
  • ⚡ Screen Time Metric: 2.03s (Target Met! < 10s)

Code

GitHub logo Mustafa1765 / Soil-Compost-Doctor

🌿 Soil & Compost Doctor

Touch Grass Edition — Privacy-First, Offline-Ready AI for Gardeners

Hacktoberfest 2026 License: MIT Local Inference Screen Time

"Put your phone away and get your hands in the dirt."
An open-source, local-first multimodal AI assistant designed to identify plant leaf diseases, diagnose soil deficiencies, evaluate compost decomposition stages, and deliver immediate physical outdoor tasks in under 10 seconds.

🎯 The "Touch Grass" Philosophy

Technology usually keeps us glued to screens. Soil & Compost Doctor is engineered to do the exact opposite:

  1. Under 10 Seconds Screen Time: Snap a photo or tap a specimen preset.
  2. Hands-Free Audio Readout: The Web Speech API speaks the immediate diagnosis and physical instructions aloud so you don't need to tap a phone screen with muddy gardening gloves.
  3. Real-World Action Item: Every scan outputs a concrete physical task (e.g., "Prune bottom 4 inches of infected leaves," or "Add 3 buckets of dry brown leaves and turn…
View on GitHub

👉 Full Source Code: https://github.com/Mustafa1765/Soil-Compost-Doctor

Core Code Highlights:

1. Multimodal Local Inference with Ollama (app.py) 

# Pass garden photo & retrieved extension guidelines to local open-weight vision model
response = ollama.chat(
    model="llama3.2-vision",
    messages=[
        {"role": "system", "content": system_prompt_with_rag_context},
        {"role": "user", "content": "Analyze specimen for immediate outdoor physical action.", "images": [image_b64]}
    ]
)
Enter fullscreen mode Exit fullscreen mode

2. Local Agricultural Vector Search with ChromaDB (build_kb.py) 

# Query persistent local vector store for peer-reviewed extension guidelines
collection = chroma_client.get_collection(
    name="soil_compost_kb",
    embedding_function=get_embedding_function()
)
results = collection.query(query_texts=["dark concentric target spots on leaf"], n_results=1)
Enter fullscreen mode Exit fullscreen mode

3. Hands-Free Voice Readout via Web Speech API (index.html) 

// Automatically read diagnosis aloud so gardeners don't touch screen with dirty hands
const utterance = new SpeechSynthesisUtterance(data.spoken_summary);
utterance.rate = 1.0;
window.speechSynthesis.speak(utterance);
Enter fullscreen mode Exit fullscreen mode

How I Built It

The entire application is constructed around open-source AI and local on-device inference:

  1. Open-Weight Vision Model (Meta Llama 3.2 Vision via Ollama):
    We harness llama3.2-vision running on a local Ollama instance. The model inspects specimen images (leaf spot patterns, soil compaction fissures, compost moisture levels) directly on the local machine with zero external cloud calls.

  2. Open-Source Vector Database (ChromaDB):
    We use ChromaDB as a persistent local vector store (./chroma_db). It stores verified agricultural extension guidelines. When an inquiry is processed, ChromaDB performs vector similarity retrieval to ground the vision model's output in peer-reviewed horticultural practices, preventing hallucinations.

  3. Deterministic Offline Embeddings (embeddings.py):
    To ensure the app functions even when completely disconnected from the internet, we developed a deterministic domain embedding engine combining agricultural vocabulary projection and hashing tricks. It runs in milliseconds with zero dependencies on Hugging Face downloads.

  4. Web Speech Synthesis API:
    Built with native browser window.speechSynthesis, the app automatically speaks the diagnosis aloud the moment inference finishes, allowing gardeners to keep working without looking at the screen.

Why Does Open Innovation Matter?

Open innovation isn't just a technical preference for this project—it is the only reason the project can exist:

  1. Gardens and Allotments Don't Have Wi-Fi:
    Gardening, composting, and farming happen outdoors: in rural allotments, community plots, backyards, and greenhouses where cellular reception is weak or nonexistent. Closed cloud APIs (like OpenAI GPT-4o or Google Cloud Vision) immediately fail when there is no internet connection. By using open-weight models (llama3.2-vision) and local vector storage (ChromaDB), Soil & Compost Doctor works in airplane mode in the middle of a forest.

  2. Zero Operating Cost for Community Gardeners & Students:
    Commercial APIs charge per image and per token. For community garden clubs, students, or hobbyists, monthly API bills are a barrier to entry. Open-weight models running on Ollama cost $0.00.

  3. Total Privacy & Land Sovereignty:
    Gardeners should not have to upload photos of their homes, yards, private properties, or crops to big tech cloud servers. Local AI ensures not a single byte of image data leaves the user's machine.

  4. Hackability & Community Extension:
    Because the knowledge base is open code (build_kb.py), anyone in the open-source community can fork the repository, add their native flora or regional soil types, and customize the guidelines for their climate zone.

My Agent Session

During development, pair-programming with an advanced AI coding assistant enabled rapid architecture of:

  • A custom offline embedding function to bypass external ONNX download timeouts.
  • Clean integration of Flask, ChromaDB persistent storage, and Ollama multimodal chat.
  • A resilient fallback engine that allows instant testing even while vision models are loading.
  • A responsive, outdoor-friendly Web UI with Web Speech synthesis and under-10-second interaction tracking.

Prize Categories

  • Week 1 Theme: Touch Grass (Primary Category)
  • General Hacktoberfest Open-Source AI Challenge

Thank you for organizing the Hacktoberfest Open-Source AI Challenge! Now put your phone away, get outside, and touch grass! 🌿

Top comments (0)