This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

🌿 WildSense — Look Closer. Then Look Away.

What I Built

WildSense is a privacy-first, offline AI plant exploration companion designed to turn screen time into real-world nature time.

We carry incredibly powerful technology in our pockets, but we often use it to scroll through endless digital content instead of noticing the world around us. WildSense takes a different approach: use AI as a bridge to the outdoors, not another reason to stay online.

📱 Take a photo. Discover a plant. Explore the world around you.

WildSense combines local AI-powered plant recognition with a botanical encyclopedia and nature-exploration features to encourage people to step outside, observe their surroundings, and learn from the natural world.

🌱 Key Features

• 🔬 Offline AI Plant Recognition: Uses a locally running FastViT ONNX model with a 4,271-taxon taxonomy to suggest plant identities without sending photos to a cloud AI service.
• 🌍 Botanical Encyclopedia: Explore an offline collection of 114 botanical species, including regional-language names and toxicity notes.
• 📸 Multi-Photo Exploration: Analyze multiple plant photographs and select the plant part you're observing to support identification.
• 🧭 Nature Missions: Turn outdoor exploration into small, engaging challenges that encourage observation and discovery.
• 📓 Personal Field Journal: Keep track of exploration sessions and discoveries locally.
• 🌿 Grass Score: Make outdoor exploration more engaging through progress and nature-focused activities.
• 🔊 Audio Narration: Listen to botanical information using browser-based speech synthesis, including offline-capable system voices where available.
• 📵 Phone-Free Field Mode: Use the app to learn what you need, then put your phone away and experience nature directly.
• 🔐 Local-First Privacy: Plant recognition runs on-device, and journal data is stored in a local SQLite database.

🎯 Who is it for?

WildSense is for students, nature enthusiasts, beginner botanists, hikers, families, and anyone who wants to reconnect with the outdoors without depending on a constant internet connection.

The philosophy is simple:

Technology should help us notice more of the real world—not distract us from it.

Demo

🎥 Video Demo: [Watch the WildSense walkthrough on YouTube] https://youtu.be/_LjfTU0UkzQ

The project is designed to run locally. For the full local-inference experience, follow the setup instructions in the repository.

Code

💻 GitHub Repository: https://github.com/kusalar/WildSense

The repository includes the frontend, FastAPI backend, local ONNX inference pipeline, botanical database, specialist-classifier experimentation, evaluation tools, and documentation.

The project is built to be explored, tested, and improved by the open-source community.

How I Built It

WildSense is built around local AI inference rather than a cloud-dependent identification API.

🧠 Open-Source AI at the Core

• FastViT + ONNX Runtime: Runs the plant-image inference pipeline locally on the CPU.
• 4,271-Taxon Classification Space: Provides a broad taxonomy for candidate plant identification.
• Experimental Indian Flora Specialist: An optional specialist classifier uses FastViT image embeddings and a conservative arbitration layer to improve selected regional plant identifications.
• FastAPI + Python: Powers the backend, inference pipeline, API endpoints, and application logic.
• SQLite: Stores the offline botanical encyclopedia, field journal, and exploration data.
• Next.js + React + TypeScript + Tailwind CSS: Powers the interactive frontend.
• Browser Speech Synthesis: Provides audio narration without requiring a cloud speech service.

🏗️ Architecture

The basic workflow is:

1. The user photographs a plant.
2. The image is processed locally by the AI inference pipeline.
3. The model produces candidate matches using image embeddings and taxonomy labels.
4. When enabled, the specialist classifier passes through additional confidence, margin, and contradiction checks.
5. WildSense displays a prediction or falls back when the specialist result does not meet its acceptance criteria.
6. The user can explore botanical information, record discoveries, and continue their exploration offline.

The specialist is experimental and disabled by default. The global model and specialist arbitration have been evaluated on a small held-out dataset, so real-world identification accuracy still needs broader validation. Predictions should not be used to determine whether a plant is edible or safe for medicinal use.

I also designed WildSense to work on modest hardware. The local inference approach avoids requiring a dedicated GPU or a paid AI API for the core plant-recognition workflow.

Why Does Open Innovation Matter?

Open innovation made WildSense possible as a project that people can inspect, run locally, evaluate, and improve instead of treating AI as an inaccessible black box.

A closed cloud API might offer convenient image recognition, but it would not automatically give users the same combination of local processing, data control, offline access, and freedom to experiment with the model pipeline.

With open tools and locally runnable models, I can:

• Keep plant photos private: Images can be processed locally instead of being sent to a remote AI provider.
• Build for limited connectivity: The core recognition workflow does not depend on a constant internet connection.
• Experiment beyond a black-box response: I can evaluate embeddings, compare candidate matches, test specialist classifiers, and introduce conservative rejection rules.
• Make the project accessible: People can explore a local-first AI application without needing a paid inference API.
• Invite community improvements: Developers can contribute better regional datasets, improve recognition and abstention, expand the botanical encyclopedia, and add languages and nature missions.

Open innovation is especially important for regional biodiversity. Communities should be able to contribute local plant knowledge and improve tools for their environments instead of relying entirely on centralized systems.

The goal is not to make AI the center of the outdoor experience. It's to use open AI to help people look up from their screens.

Prize Categories

Used ' Eleven Labs ' for Speech to Text

🌿 Final Thought

WildSense is an experiment in building technology that encourages people to spend less time staring at technology.

Identify a plant. Learn something new. Notice the details. Put the phone away.

WildSense — Look closer. Then look away.

Hacktoberfest #OpenSourceAI #TouchGrass #LocalAI #ONNX #FastViT #NatureTech #AI