Wildlens

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

What I Built

WildLens — Your AI-Powered Nature Field Guide

So WildLens is basically a nature identification app, but the whole idea was to get people outside insted of just staring at there phone all day. You take a photo (or upload one) of a bird, bug, plant, mushroom, whatever you find, and the AI tells you what it thinks it is and why. It dosen't stop at the name tho, it tries to make you actually look closer at the thing.

What's inside:

• Nature identification: upload a photo and get a result, plus the visual clues behind it
• Lookalike Detective: when the AI isnt sure, it shows upto three possible candidates
• Observe Next: gives you ideas on what to look for next, like how it looks or behaves
• Ask WildLens: you can ask follow up questions about whatever you identified
• Voice field guide: talk to it using ElevenLabs, it feels more like chatting with a friend
• Camera support: take the picture right in the browser and send it for analysis

Basically i wanted AI to be a buddy for going outside and being curious, not one more reason to stay glued to the screen.

Demo

Live prototype:

wildlens-frontend.onrender.com

The backend is deployed seperately on Render and its what gives the frontend its API.

It works with photo identification and the conversational learning stuff. Heads up tho, the backend is on Render's free tier so the first request after a while might be slow, its just waking up.

Video Prototype

If you dont feel like waiting for the backend to wake up, here's a quick video of WildLens in action. It shows the photo identification, the lookalike stuff and the voice guide working together.

Code

GitHub repository:

TechyCoderzx / wildlens

WildLens 🌿 Discover the wild. Understand what you see. Then look closer WildLens is an AI-assisted nature field guide built to help people explore the living world around them. Upload or capture a photograph of an organism and WildLens returns a structured identification hypothesis, visible clues, possible lookalikes, and ideas for what to observe next. You can also ask follow-up questions or talk to the WildLens field guide using voice The idea is simple: use AI to make a moment outdoors more interesting, not to keep someone staring at a screen for longer than necessary Important: WildLens is an educational exploration tool. Its results are model-generated hypotheses, not verified scientific identifications. Do not use it to decide whether a plant, mushroom, animal, or other organism is safe to touch, eat, or handle.

Live project

• Website: https://wildlens-frontend.onrender.com
• Backend health: https://wildlens-k0ey.onrender.com/health
• Interactive API documentation: https://wildlens-k0ey.onrender.com/docs
• Source code: https://github.com/TechyCoderzX/wildlens The frontend and backend…

View on GitHub

The repo has the frontend, backend, API integrations and the project config.

How I Built It

WildLens uses a React + Vite frontend and a FastAPI backend. The backend connects the UI to the AI model and the voice services through different API endpoints.

Gemma 4 API — The Core Identification Engine

The main AI part uses Google's Gemma 4 through the Gemini API, the model id I configured is gemma-4-26b-a4b-it.

When someone submits a photo, the frontend sends it to the FastAPI backend. The backend processes the image and sends it to the model as a multimodal request, with instructions to identify the organism and explain the evidence for its answer.

Gemma 4 gives back structured results, things like candidate organisms, explanations, and info that helps you explore further. I also tried to build in uncertainty and lookalike reasoning, because treating every prediction like its 100% right is just wrong.

The backend validates what the model returns before it goes to the frontend. This matter because structured output from AI can sometimes come back incomplete or kinda messed up.

Using the API meant I could use a really capable multimodal model without hosting it on my own laptop, and nobody has to download model weights to try the app.

ElevenLabs — Voice-Based Learning

ElevenLabs is what powers the voice side of WildLens, to make the field guide more conversational and easier to access.

The integration has:

• Conversational voice interaction: connects the user to the WildLens Field Guide agent
• Text-to-speech summaries: reads the identification summary out loud
• Voice agent configuration: I setup an ElevenLabs agent to give short, friendly nature explanations, and to be careful about uncertainty and safety

The voice conversation uses a token flow that goes through the backend, meant for client-side WebRTC. The sensitive API keys stay on the backend and never get exposed in the browser.

Application Architecture

The main flow goes like this:

1. User captures or uploads a photo in the React frontend.
2. Frontend sends the image to the FastAPI backend.
3. Backend calls the Gemma 4 API and validates the identification that comes back.
4. Result shows up with explanations and possible lookalikes.
5. User keeps learning with follow up questions, observation tasks and voice.

Technology stack

• Frontend: React, Vite, JavaScript, HTML and CSS
• Backend: Python, FastAPI and Pydantic
• AI: Gemma 4 through the Gemini API
• Voice: ElevenLabs conversational AI and text-to-speech
• Deployment: Render for both frontend and backend

One of the harder parts was balancing detailed explanations with the response limits. Sometimes the model write more then the token budget I set, so getting reliable structured output and keeping answers short is definetely something I still need to improve.

Why Does Open Innovation Matter?

Open innovation makes it way easier for developers to try stuff with AI, mix different tools together, and turn an idea into a real working app without building every single piece from scratch.

WildLens puts multimodal AI, a normal web app and voice interaction together in one project. Because the model APIs and dev tools are so accessible, I could focus on the user experience, how the identification works, and whether people actually learn something from it.

It also shows something important I think, identifying an organism is only the begining. AI can help people become more observent by explaining what they can see, comparing similar species, and suggesting what to look into next.

I want WildLens to get people to step outside, notice the biodiversity around them, and use tech to feel more connected to nature.

My Agent Session

I used some dev tools and AI-assisted coding while building this, but I haven't included a DevRelay session link here. If I publish a saved session later i'll add the official embed or link in this section.

Render — Deploying WildLens to the Web

I used Render to deploy WildLens so people can open it with a public URL instead of having to run everything locally.

There are two seperate Render services:

Frontend — Render Static Site: hosts the React + Vite app, so the WildLens interface is available as a public website.

Backend — Render Web Service: hosts the FastAPI backend, which handles the image-analysis requests, talks to the Gemma 4 API, validates the model responses, and provides the voice endpoints for ElevenLabs.

The frontend talks to the backend using its public API URL, and the backend's CORS config allows requests from the deployed frontend origin.

Keeping them as two services helped me get the app online while keeping the API credentials on the backend and not in the frontend. Render also made it easy to deploy updates straight from GitHub whenever the project changed.

I used the free tier for the prototype, which was great since I didn't need to pay for hosting. The trade-off is free backend instances spin down after being inactive, so the first request can take a while to come back.

Prize Categories

• Best Use of Gemma 4
• Best Use of Render
• Best Use of ElevenLabs