Farhan Ahmad

Posted on Oct 12 AI-assisted

NaturalDex: Building an Offline-First Nature Collection Game During Hacktoberfest 2026

#devchallenge #hf26challenge #opensource #ai

Have you ever wanted to step outside, connect with nature, and explore beyond your comfort zone?

I did. And instead of just thinking about it, I decided to build something to make it happen.

This is the story of how I built NaturalDex — the problem I wanted to solve, how I approached the architecture, the technology stack I chose, the technical hurdles I faced along the way, and what I learned from the process.

This project is part of my journey through Hacktoberfest 2026, where I explored how to turn an outdoor exploration concept into a fully functional, privacy-first application.

The Problem

Modern screen addiction and constant doomscrolling diminish attention spans and keep people indoors. With the "Touch Grass" mindset in mind, I wanted to build an application that minimizes screen time while fostering genuine curiosity about the natural world.

The goal was to make outdoor exploration exciting by encouraging users to step out of their comfort zone, explore their local environment, identify real species, and gain field knowledge in a rewarding way.

That became the founding vision for NaturalDex.

Here It Is in Action 🎬

🎥 Click the preview above to watch the project demonstration on YouTube.

Explore the code:
KevinForsgren / NaturalDex

A field journal and outdoor collection game that encourages nature exploration through species identification and collection. Built with a privacy-first, offline-capable architecture.
NaturalDex

A field journal and outdoor collection game that encourages nature exploration through species identification and collection. Built with a privacy-first, offline-capable architecture.

Overview

NaturalDex turns outdoor observation into a collection game. Users capture three photos of a plant, animal, or fungus in the field, submit them for local AI identification via a paired home base (laptop running Ollama), and build a personal field journal with discovered species cards.

Architecture

Backend: FastAPI + SQLAlchemy 2.0 + Pydantic v2 (Python 3.14+)

Frontend: React 18 + Vite 6 + TailwindCSS v4 + React Router 6

Database: SQLite (local file)

AI: Local inference via Ollama on a paired home base (no cloud dependency)

Auth: JWT in HttpOnly cookies + Argon2 password hashing

Features

Daily field tasks — Three personalized tasks per day (flora/fauna/fungi categories)

Three-photo discovery flow — Capture multiple angles for accurate ID

Local AI identification —…
View on GitHub

Mobile UI

Core Features & Experience

The goal was simple: build a working prototype that demonstrates gamified nature discovery without relying on cloud privacy compromises or overly complicated setups.

Here is what the application currently delivers:

Daily Field Tasks: Receives three personalized tasks per day (covering flora, fauna, and fungi categories) that encourage users to step outside and observe local biodiversity.
3-Photo Discovery Flow: Users capture three angles of a plant, animal, or fungus in the field to feed into a local vision AI for accurate species identification.
Pokédex-Style Field Journal & XP: Discovered species unlock collectible cards with stable URLs, adding new entries to a personal Field Journal with leveling and XP progression.
Privacy-First Local AI: Operates offline without external cloud dependencies. Image classification runs directly on the user's home laptop using Ollama.

What I Built

At its core, NaturalDex is a mobile-first web application powered by local AI and self-hosted architecture, designed to turn outdoor walks into a personal natural history expedition.

1. Nature Card Collection

Users collect species cards by identifying surrounding flora, fauna, and fungi. Each card displays the common and scientific names, location recorded, species summary, and interesting biological facts.

2. Player XP & Progression System

To encourage long-term outdoor habits, NaturalDex incorporates player XP mechanics. Users earn XP by completing daily tasks and logging new species observations, turning nature walks into a rewarding progression loop.

3. Thoughtful User Experience

A great concept requires an intuitive interface. Key UX design choices include:

Responsive Mobile-First UI: Optimized for handheld camera capture on mobile while remaining accessible on desktop.
Seamless Home Base Pairing: QR code pairing connects mobile browser clients to a home base Ollama instance over LAN or ngrok tunnels.
Clear Feedback: Informative states during multi-photo upload, vision AI inference, and card generation.
Cohesive Aesthetic: A clean, natural visual language designed with TailwindCSS v4.

How I Built It

The Tech Stack

Technology	Purpose
React 18 & Vite 6	Frontend framework and fast build toolchain
TailwindCSS v4	Utility-first styling and responsive layouts
FastAPI & Pydantic v2	High-performance Python 3.14+ API backend
SQLAlchemy 2.0 & SQLite	Local database ORM and persistent storage
Ollama (Local LLM/Vision)	Privacy-first local vision AI inference (e.g., `llava-phi3`)
JWT & Argon2	Secure authentication stored in HttpOnly cookies

The Frontend Structure

The frontend is organized around key user flows—Discover, Collection, Journal, and Profile—ensuring clear separation of concerns:

frontend/src/
├── App.jsx
├── components/
│   ├── BottomNav.jsx
│   ├── Modal.jsx
│   ├── QRScanner.jsx
│   └── Toast.jsx
├── hooks/
│   └── useCamera.js
├── lib/
│   ├── api.js
│   ├── data.js
│   └── icons.jsx
├── pages/
│   ├── CardDetail.jsx
│   ├── Collection.jsx
│   ├── Discover.jsx
│   ├── Home.jsx
│   ├── Journal.jsx
│   ├── Login.jsx
│   ├── PairingPage.jsx
│   └── Profile.jsx
└── state/
└── AuthContext.jsx

Backend & Discovery Data Pipeline

The backend orchestrates photo submission, vision inference, rejection/policy validation, database updates, and XP awarding.

┌─────────────┐     ┌─────────────┐     ┌─────────────┐     ┌─────────────┐
│  Phone      │────▶│  Backend    │────▶│  Ollama     │────▶│  Backend    │
│  (3 photos) │     │  (queue)    │     │  (vision)   │     │  (persist)  │
└─────────────┘     └─────────────┘     └─────────────┘     └─────────────┘
       │                                        │
       │                                        ▼
┌──────┴──────┐                        ┌──────────────────┐
│  Validation │                        │  Database Ops    │
│  (policy)   │                        │  • species       │
└─────────────┘                        │  • collection    │
                                       │  • observations  │
                                       │  • XP events     │
                                       │  • journal       │
                                       │  • daily tasks   │
                                       └──────────────────┘

Dedicated API Client Pattern

To maintain a clean frontend architecture, backend communication is centralized in dedicated API utility modules:

const API_BASE_URL = import.meta.env.VITE_API_BASE_URL || "";

export async function getCollection() {
  const response = await fetch(`${API_BASE_URL}/api/collection`, {
    headers: { "Content-Type": "application/json" },
  });

  if (!response.ok) {
    throw new Error("Failed to fetch collection");
  }

  return response.json();
}

Challenges I Faced

1. Multi-Photo Vision AI Identification

Integrating Ollama vision models required structured JSON outputs to extract common names, scientific names, confidence levels, and quality checks. Handling non-vision fallback errors when users ran incompatible models (such as text-only models) required strict validation policies in services/vision.py.

2. Home Base QR Pairing Across Networks

Allowing a mobile phone browser to talk to an Ollama instance running on a paired laptop required a lightweight pairing protocol. I designed a challenge-response QR code pairing flow (/api/home-base/pairing-challenge) supported by ngrok tunneling so phones on cellular networks could pair with laptops on local Wi-Fi.

3. Idempotency & XP Integrity

To prevent duplicate collection entries when uploading photos, the backend uses unique observation linking (collection_card_observations) to ensure card counts, XP events, and daily task updates remain strictly idempotent.

4. Managing Scope

Rather than attempting to launch a full native mobile app immediately, I focused on building an offline-capable, responsive Web App (PWA architecture) to perfect the core identification and collection loop first.

Testing and Reliability

To ensure backend stability and reliability, testing was implemented using Pytest across key service layers:

API Endpoint Testing: Automated routes validation for /api/auth, /api/discoveries, /api/collection, /api/daily-tasks, and /api/home-base.
Vision Policy Checks: Verified backend rejection filters for low-quality or non-matching multi-photo submissions.
Database & ORM Verification: Model relationship testing between users, cards, species, and XP event tables with SQLite.
Manual Cross-Device UX Checks: Validated camera feeds, mobile viewports, and QR scan mechanics on handheld devices.

Running It Locally

Prerequisites

Python 3.14+
Node.js 18+
Ollama (with a vision model like llava-phi3)

1. Clone the Repository

Bash

git clone [https://github.com/KevinForsgren/NaturalDex.git](https://github.com/KevinForsgren/NaturalDex.git)
cd NaturalDex

2. Backend Setup

Bash

cd backend
pip install -e ".[dev]"
# or using uv:
uv sync --extra dev
uv run uvicorn src.main:app --reload

3. Frontend Setup

Bash

cd frontend
npm install
npm run dev

4. Configure Ollama Vision AI

Bash

# Install and serve Ollama
ollama pull llava-phi3
ollama serve

Copy .env.template to .env in the root directory:

Code snippet

DATABASE_URL=./Database/naturedex.db
VISION_PROVIDER=ollama
OLLAMA_BASE_URL=http://localhost:11434
OLLAMA_MODEL=llava-phi3
SECRET_KEY=your-secret-key-here

What I Would Improve Next

[ ] Convert the frontend into a native React Native / Expo application for improved mobile camera control.
[ ] Add offline queueing for photo submissions when cellular reception drops in deep outdoor areas.
[ ] Expand unit test coverage across frontend hooks and state contexts.
[ ] Add community card sharing features while preserving user location privacy.

What I Learned

Building NaturalDex highlighted that developing an AI application is far more than wrapping an API call. It requires orchestrating local hardware constraints, robust data validation, user privacy, and effective state management.

Working with AI coding agents also taught me the importance of detailed prompting—guiding models like an engineering manager with explicit contextual constraints yields vastly superior architecture and cleaner code.

DEV Community

NaturalDex: Building an Offline-First Nature Collection Game During Hacktoberfest 2026

The Problem

Here It Is in Action 🎬

KevinForsgren / NaturalDex

A field journal and outdoor collection game that encourages nature exploration through species identification and collection. Built with a privacy-first, offline-capable architecture.

NaturalDex

Overview

Architecture

Features

Mobile UI

Core Features & Experience

What I Built

1. Nature Card Collection

2. Player XP & Progression System

3. Thoughtful User Experience

How I Built It

The Tech Stack

The Frontend Structure

Backend & Discovery Data Pipeline

Dedicated API Client Pattern

Challenges I Faced

1. Multi-Photo Vision AI Identification

2. Home Base QR Pairing Across Networks

3. Idempotency & XP Integrity

4. Managing Scope

Testing and Reliability

Running It Locally

Prerequisites

1. Clone the Repository

2. Backend Setup

3. Frontend Setup

4. Configure Ollama Vision AI

What I Would Improve Next

What I Learned

Project Links

Prize Categories

Top comments (0)