Zoidex: An Open-Weight AI Field Guide for Touching Grass

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass.

What I Built

Zoidex is a camera-first biodiversity field guide designed to get people off their screens and more curious about the world around them.

You spot an unfamiliar plant on a hike. A bird lands in your garden. You find a mushroom on a walk and wonder whether it is safe to touch.

Instead of scrolling through search results, you open Zoidex, take a picture, and turn that moment into a field observation.

Zoidex uses AI vision to generate an observation containing:

• Species identification and scientific name
• Category and rarity information
• Safety information and ecological facts
• A saved image and field entry for later reference

The idea is simple: technology should make exploring nature more rewarding, not give us another reason to stay indoors.

Zoidex is designed for hikers, gardeners, birders, students, and anyone who wants to learn more about the biodiversity around them.

It is an early prototype, not a substitute for expert identification. In particular, AI-generated species and safety information should be independently verified before being used to make decisions about potentially toxic or dangerous organisms.

Demo

Live application: https://frontend-ociu.onrender.com

The application is deployed publicly and can be accessed through a mobile browser. Its camera-first interface is designed for use outdoors, where observations actually happen.

Code

GitHub repository: https://github.com/bl4m/zoidex

The project is built as a full-stack application, with a React frontend, a FastAPI backend, and persistent storage for observations.

How I Built It

The stack

• Frontend: React + Vite
• Backend: FastAPI
• Database layer: SQLModel
• Persistent database: Supabase PostgreSQL
• Image storage: Supabase Storage
• AI inference: Groq
• Vision model: An open-weight Qwen vision model
• Deployment: Render Static Site + Render Web Service

From a photograph to a field observation

The current pipeline looks like this:

Mobile camera
     |
     v
React frontend
     |
     v
FastAPI backend
     |
     v
Image validation
     |
     v
Qwen vision model via Groq
     |
     v
Structured output and safety checks
     |
     v
Supabase Storage + PostgreSQL
     |
     v
Saved field observation

The backend validates incoming images, sends them for model inference, evaluates the structured response, and persists approved observations and their associated images.

I also implemented the backend and deployment architecture, while using AI assistance during frontend iteration, debugging, and UI refinement.

The difficult part was everything around the model

Calling a vision model is relatively straightforward. Turning that call into a usable application involves a different set of problems.

Some of the challenges I worked through were:

• Mobile camera access: browser permissions and the HTTPS requirements for camera APIs.
• Deployment: hosting the frontend and backend separately while keeping them connected.
• CORS and proxy configuration: making cross-origin requests work correctly in production.
• Persistent images: ensuring captured images remain accessible after a request finishes.
• Storage permissions: configuring Supabase Storage for the application's image workflow.
• Database reliability: managing PostgreSQL connections and connection pooling.
• Structured AI output: making model responses usable by the application rather than relying on unrestricted prose.
• Prompt-injection resistance: treating text embedded in uploaded images as untrusted data, not as instructions.
• Content rejection: rejecting unsuitable inputs before they are uploaded to storage or written to the database.
• Graceful failure: handling unavailable services and rejected observations without leaving the frontend in a broken state.

For deployment, I chose Render because it made it straightforward to host both the frontend and backend from the same repository while keeping the demo publicly accessible. Supabase handles persistent data and image storage, leaving Render responsible for serving the application and API.

The result is not just a model call wrapped in a UI. It is a deployed system with an end-to-end path from camera input to a persistent field observation.

Why Does Open Innovation Matter?

Zoidex currently uses an open-weight Qwen vision model through Groq's hosted inference API.

There is an important distinction here: Groq provides the inference infrastructure, while Qwen is the model being used for image understanding. The current application relies on hosted inference; it does not run the model locally.

So why use an open-weight model?

Because I want the model to be a replaceable component of the system, rather than a permanent dependency on a single proprietary model API.

Open-weight models create room to experiment with:

• Model swapping: comparing different vision models without redesigning the entire application.
• Domain-specific fine-tuning: adapting models to biodiversity identification and structured field observations.
• Custom evaluation: testing model behavior against a curated dataset of plants, animals, and fungi.
• Self-hosted inference: exploring deployment options beyond a hosted inference provider.
• Cost and performance optimization: choosing a model and inference setup appropriate for real-world use.

For biodiversity applications, the distinction matters. A model that confidently produces a plausible species name is not necessarily a model that identifies a species correctly.

I want to move toward a system where identification quality, uncertainty, taxonomy, and safety can be evaluated independently of the underlying model.

Open innovation makes that experimentation more practical. It gives the project a path beyond its initial prototype and makes it possible to improve the intelligence behind the application without rebuilding the entire product.

What I Learned

Building Zoidex made me think about AI applications as systems rather than isolated model calls.

The model is only one part of the experience. Input validation, persistence, security, deployment, failure handling, and the interface all determine whether the application is useful outside a development environment.

I also learned to pay more attention to the boundary between what a model generates and what an application should trust.

A generated species name is not automatically a verified identification. A safety statement is not automatically authoritative. Structured output helps an application process responses, but structure alone does not guarantee correctness.

These are problems I want to explore further as I develop Zoidex.

What's Next?

The current version establishes the core camera-to-observation workflow. The next version will focus on making the identification process more reliable and the field experience more useful.

1. A more explicit agentic pipeline

I plan to evolve the current inference workflow into a sequence of specialized stages:

Photograph
    |
    v
Vision model
    |
    v
Safety and relevance check
    |
    v
Flora / fauna / fungi classification
    |
    v
Taxonomy verification
    |
    v
Authoritative biodiversity retrieval
    |
    v
Final field observation

The goal is not to add agents just for the sake of calling something agentic. Each stage should have a defined responsibility, and the system should be able to distinguish a model's initial hypothesis from information verified against external sources.

2. Better observations and uncertainty

• Confidence estimates and user corrections.
• Retrieval from authoritative biodiversity sources.
• Evaluation against a curated image dataset.
• Clearer uncertainty handling when the model cannot confidently identify a species.

3. A more complete field experience

• User authentication and private personal collections.
• Opt-in public collections.
• A GPS-based global discovery map.
• Regional biodiversity exploration and progress.
• Badges and meaningful challenges that encourage real-world observations.
• Streaks tied to actual field activity.
• Offline or self-hosted inference experiments.

Privacy will be a design constraint, not an afterthought. Location sharing will be opt-in, and private observations will be the default.

The longer-term goal is to make Zoidex a field companion that helps people build a better understanding of the biodiversity around them, rather than simply generating answers from photographs.

Prize Categories

I'm entering:

• Best Use of Render — Both the public frontend and the FastAPI backend are deployed on Render, with Supabase providing persistent database and image storage.

---

Zoidex started with a simple idea: make it easier to stop, look at something living, and learn what it is.

The interesting part is what comes after that first identification: verifying it, understanding its ecological context, recording the observation, and using technology to encourage more time spent exploring the real world.

That's the direction I want to take this project.

Thanks for checking out Zoidex!