WildWhisper - Hands Free Trail Naturalist

This is a submission for the Hacktoberfest Open-Source AI Challenge — Week 1: Touch Grass.

What I Built

WildWhisper is a hands-free trail naturalist designed to help people explore nature instead of spending their time looking at a screen.

Imagine walking on a trail, noticing an unfamiliar plant, and wanting to learn more about it. WildWhisper lets you provide a trail photo and ask a question such as, “What kind of plant is this?”

The application uses a locally running vision model to analyze the image and generate a response, then converts that response into speech.

The goal is to make AI useful outdoors, where people should be able to focus on their surroundings rather than constantly interacting with their phones.

Key Features

• Local image analysis using Moondream through Ollama.
• Voice input using Vosk speech recognition.
• Spoken responses using ElevenLabs text-to-speech.
• A Flask web interface for interacting with the application.
• Sentry tracing to help investigate application behavior.
• A privacy-conscious approach that keeps image analysis and speech recognition local.

Demo

The project currently runs locally.

To try it, clone the repository, install the dependencies, configure the required environment variables, ensure Ollama and the Moondream model are available, and start the Flask application.

I haven't included a public demo link because a deployed demo or video isn't available yet.

Code

GitHub repository: https://github.com/inturi-vaishnavi/WildWhisper

The repository contains the Python application, Flask frontend, templates, static assets, and tests.

How I Built It

I built WildWhisper with Python and Flask, keeping the application centered around local AI inference.

Moondream + Ollama: The vision model runs locally through Ollama and analyzes the provided trail image.

Vosk: Speech recognition runs locally, allowing users to provide questions through voice input.

ElevenLabs: The generated answer is converted into speech so users can listen rather than read the entire response.

Sentry: Tracing helps investigate the inference pipeline and diagnose problems.

The project also includes a browser-based interface built around an existing Stitch design.

One of the challenges has been improving the reliability of speech recognition and making plant-identification responses more useful. I tested the microphone independently and confirmed that it captures audio, then tested Vosk directly and identified a transcription error that still needs improvement.

Why Does Open Innovation Matter?

Open innovation makes it possible to experiment with AI systems that people can run and inspect on their own machines.

Using an open-weight vision model locally means the core image-analysis step does not need to send every trail photo to a proprietary vision API. Local speech recognition also reduces the need to send microphone recordings to an external transcription service.

This architecture provides greater control over data handling and makes local AI experimentation accessible to developers who want to build practical applications beyond the browser.

WildWhisper still uses ElevenLabs for speech synthesis, so the application is not entirely offline. The project explores a hybrid approach that combines local inference with an external service where needed.

My Agent Session

I used VS Code and GitHub Copilot to develop and debug the application. I haven't published a shareable agent-session recording yet.

Prize Categories

This project is submitted for the Hacktoberfest Open-Source AI Challenge — Week 1: Touch Grass.

What's Next?

I want to improve voice transcription accuracy, make plant observations more specific and useful, and continue refining the hands-free experience.

AI-generated plant identification can be wrong, so WildWhisper should not be used to determine whether a plant is safe to touch or eat.

Thanks for checking out WildWhisper!