DEV Community: RAKOTONDRAINIBE Rohiniaina Maria Grazia

Nook

RAKOTONDRAINIBE Rohiniaina Maria Grazia — Mon, 12 Oct 2026 06:58:45 +0000

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

Nook: I Built an AI Field Guide That Sends You Back Outside

What I Built

Nook is a nature fieldwork companion that turns your phone into a naturalist's notebook, and then invites you to keep exploring your environment.

You snap a photo of something you found outside: a leaf between two colors, a bark carved by insects, an odd shadow, a pebble that looks like it's been through something. Gemma 4, running as the core of the backend, looks at it with a field naturalist's eye.

The app is for anyone who's walked past interesting things a thousand times and never stopped to actually look. Kids, curious adults, people who want to spend more time outside but don't know where to start.

Demo

https://nook-i5bl.onrender.com

I recorded the demo with my phone to be able to take photos. And also, because it's the main use finality of this project:

Demo video

Welcome tab: the landing screen

on a smartphone: )

Discovery tab: The result of gemma's analysis

on a smartphone:
on a computer:

Poster tab: The journal of all of your discoveries

on a smartphone:
on a computer:

The full flow in three steps:

1. Observe — snap or upload a photo from your phone (mobile-first, works with capture="environment")

2. Investigate — Gemma 4 returns a structured Field Card:

What it sees (actual, specific observations from your photo — not generic filler)
Possible identification + alternatives
Two hypotheses on why it looks the way it does
What the photo can't tell (honestly stated)
An investigation question to sharpen your attention
A safe outdoor mission to go do immediately
An Atelier — a concrete creative task you can make from what you found (a leaf rubbing, a bark sketch, a pine cone decoration, a pressed flower)

3. Investigate + Preserve — talk to the ElevenLabs voice agent when you return, then save the session to your local field journal

Code

https://github/RoxyMaxy/Nook

Stack: Flask · Vanilla JS · Gemma 4 via Google AI Studio · ElevenLabs TTS + Agent · Render · Service Worker PWA

How I Built It

The AI layer is two services. ai_service.py sends the image to gemma-4-31b-it via the google-genai SDK with a strict JSON schema prompt — the model has to return eight structured fields, each capped at 25 words, in a concrete naturalist tone. No markdown, no filler, just what it actually sees. parser_helper.py sanitizes the response in case Gemma sneaks in backticks.

The narration layer (narration_service.py) takes the Field Card text and sends it to ElevenLabs TTS (eleven_turbo_v2_5, Rachel voice) as an audio stream. If there's no connection, it falls back to a procedurally generated E-major chime — an actual chord, not silence, so there's always audio feedback.

The voice agent is an ElevenLabs conversational widget embedded in the page. It loads after the Field Card appears, so the user reads what Gemma found, then taps to debrief out loud. The agent knows it's a field guide — it asks follow-up questions, sends you back outside if needed, never claims certainty.

The journal is pure localStorage. Base64 photo, full Field Card JSON, atelier completion status, your notes. It loads on startup so past entries are there when you return. Nothing goes to a server. Nothing is deleted unless you delete it.

The offline layer is a Service Worker that pre-caches the shell. If you analyze a photo while offline, it queues and syncs automatically when connection returns. The app works in a forest with no signal — the UI loads, past entries load, and the analyze call will retry when you're back.

The backend is Flask + Gunicorn, deployed on Render's free tier via render.yaml. Three environment variables: GOOGLE_API_KEY, ELEVENLABS_API_KEY, ELEVENLABS_AGENT_ID.

Why Does Open Innovation Matter?

I built this from Antananarivo, Madagascar.

Closed AI APIs aren't just expensive here — they're often inaccessible. Most require a credit card from a supported country. Payoneer as a transit layer, SWIFT, international transfer fees — all of that just to run POST /analyze. And if the API goes down or the pricing changes, your app breaks.

Gemma running through Google AI Studio changed the calculation. One Google account, one API key, no card required to get started. And because the model is open-weight, the honest path is clear: if Google AI Studio ever becomes a problem, you point the same google-genai client at a local Ollama instance and nothing else changes. The model is swappable. The app doesn't have to die with a vendor decision.

The offline design matters for the same reason. When I take this outside in a park with no signal — which is most of the time — the UI still loads, the journal still loads, the Service Worker still serves the shell. The analysis queues and fires when I'm back on Wi-Fi. The screen is the smallest part of the experience, which is exactly what the prompt asked for.

Open innovation didn't make this easier. It made it possible.

My Agent Session

Dut to some problem in my laptop, I couldn't use DevRelay. But here is my prompt: You are a senior full-stack developer.
Design and generate the complete code for a fieldwork web application called "Nook" (with the tagline: Observe. Investigate. Create.).
This project is an ecological application designed with the open-weight model Gemma 4 at its core framework.

However, the generated code must be complete, production-grade, and immediately functional, before tokens run out.

FRAMEWORK, RENDERING, AND PORT CONSTRAINTS:

The backend application must strictly use FLASK
The application must run locally on its native default port: PORT 5000 (localhost:5000).
The frontend must be served directly by Flask via its static and template folders, using only native HTML5 / CSS3 / Vanilla JavaScript (no separate Node.js frontend server).
Provide a configuration file for Render for deployment on their free tier.

AI ENGINE (GEMMA 4 ):

The application's core logic depends on API execution or inference of the "gemma-4-31b-it" or "gemma-4-26b-a4b-it" multimodal model.
You must use the official new google-genai Python SDK to handle all backend AI operations.
Configure the default model parameter to use "gemma-4-31b-it", butmake sure it reads from an environment variable GEMMA_MODEL_ID (so the user can easily swap it to "gemma-4-26b-a4b-it" via the .env file).
The backend must feature full validation and standard JSON parsing structures processing raw output string responses from Gemma safely.

USER STORY:
The goal of Nook is to encourage the user to leave their screen and explore nature.
This is the code pipeline:

OBSERVE: A paccessible mobile-first interface (capture="environment") with dark mode posibility). The user snaps or uploads a photo from their phone.
ANALYZE: The Flask receives the image, validates the type (jpeg, png, webp, gif), encodes it to base64 bytes, and directly queries the l Gemma 4 model using the google-genai client. The prompt sent to Gemma MUST demand a JSON object . The returned JSON must contain (written in a cautious, naturalist tone, max 25 words per string):
- "seen": 3 to 6 direct visual observations such as (color, shape, texture).
- "possible_identification": The possiblr identification might be still wrong, even if just rarely. It is better to state "possible", not confirming if not sure.
- "alternatives": 1 or 2 other plausible options of why something had became what it was when taken photos
- "hypotheses": 2 possible explanations regarding the state or origin of the found object
- "uncertain": What the photo cannot tell due to its limitations. The AI Asks further questions in the investigation_question
- "investigation_question": The AI asks further questions about the found objects for the AI to confirm hypothesis or suggestions, and helps people develop their observation and attention skills.
- "observation_mission": An action to do outside immediately without touching or disturbing nature, like following ants' paths to their anthill, or give a rabit a leaf, or caressing barks to feel the relief, etc...
- "atelier": A creative crafting task that should be a concrete, safe action to do , based with the found objects (those can be a herbier with a clover, a drawing of a bark's patterns, a flower crown, a petal coloration, squirrel's best ohotoshoot, a christmas decoration made from pine cones, leaves, etc...).
NARRATE: A Flask endpoint POST /narrate receives the card text and uses the ElevenLabs TTS API (Model: eleven_turbo_v2_5, default Voice ID) to return a binar audio/mpeg stream. The JavaScript interface displays a "Field Card " button to fetch and play this audio stream directly in the browser.
INVESTIGATE The interface integrates the official ElevenLabs conversational voice widget (<elevenlabs-convai> loaded via its script) by dynamically injecting the Agent ID. The user speaks to this voice guide in real-time to report their findings after returning from their outdoor mission.
PRESERVE: A field journal saves the session locally (base64 photo, Field Card JSON, atelier completion status, and user text notes) into the browser's localStorage. A history of past journal entries must be displayed upon loading the application, fot the hapiness to find memories back and remember what was learned through the little travel outside;.

SECURITY AND ROBUSTNESS:

Strict exception handling with clean Python logging
Robust parser helper to sanitize Gemma's JSON string in case it contains accidental markdown backticks ( json ... ).
Full isolation of secrets using an environment file (.env) containing: GOOGLE_API_KEY, GEMMA_MODEL_ID, ELEVENLABS_API_KEY, ELEVENLABS_AGENT_ID, and optional ElevenLabs voice variables.

STYLE & LAYOUT:
Make the web app present the theme as following:

Artisanal Minimalism: Clean design without clutter.
- Smart-Casual Tone: Low pressure, high intention. Cool attitude withpit forgettong the serious execution.
- Background: #0F0F11
- Primary Text & Headers: #F5F5F7
- Secondary Text & Metadata: #86868B
- Borders & Dividers: #2C2C2E
- Interactiveity and Hover State: #FFFFFF Those are by default. But people should be able to choose among lots of color themes (blue, orange, purple, green, etc...) and light or dark theme.

USER PROFILE:
Save everything (the user's color themes, options, and nature journal entries) directly inside the browser's memory, never on a distant database. This keeps their data safe and ensures the app works 100% offline.

ONLINE/OFFLINE FUNCTIONALITY:

The app must work even with zero internet (like deep in the forest).
Create a 'sw.js' file (Service Worker) so the app's pages and designs load offline.
If the user takes a photo or finishes a task while offline, save it inside the browser's local memory immediately.
As soon as the phone gets internet back, the app must automatically send those saved photos and notes to the Flask backend without making the user refresh the page.

RESULTING JOURNAL:
The result of PRESERVE user story: a personalized poster, or a little digital journal, according to the user's choices, contains all the result of the functionalities: the taken photo, the description, the possible causes of their state, cool artistic or useful things to craft from the shot objects.

Generate all the necessary files for this application now, ready to be executed on port 5000.

Prize Categories

Best Use of Gemma — Gemma 4 (gemma-4-31b-it) is the entire analysis engine. It receives the raw image bytes, returns a structured eight-field naturalist Field Card, and powers every section the user sees. Swappable to gemma-4-26b-a4b-it via one environment variable.
Best Use of ElevenLabs — Two integrations: TTS narration of the Field Card via eleven_turbo_v2_5 (so the user can listen and go back outside), and the conversational Agent widget for real-time voice debrief when they return.
Best Use of Render — Flask + Gunicorn deployed on Render's free tier. The render.yaml defines the full service with environment variables. Auto-deploys from GitHub on every push.

Gemma Atlas

RAKOTONDRAINIBE Rohiniaina Maria Grazia — Mon, 05 Oct 2026 06:58:47 +0000

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend

What I Built

Gemma Atlas

An adaptive, private study architect and self-paced curriculum engine powered by Google's open Gemma 9B and 2B models.

My younger siblings are homeschooled, and our mother is an English teacher. She designs their entire curriculum and teaches every class herself. But she also works at other institutions, which means my siblings are by themselves when she's not home to teach them.

Even when my mother give them topics and chapters to research and to learn, my siblings struggled to find contents because:

they don't know where to start,
they hesitate about what video or websites or document to trust and exploit for learning.

Having lived through the exact same struggle myself, I built Gemma Atlas to give them what I wish I'd had.

Gemma Atlas takes a topic name or any document — slides, lecture notes, PDFs, code files — and turns it into a complete, adaptive mastery roadmap powered by Gemma.

The system provides:

Autonomous curriculum deconstruction: Gemma analyzes the scope of the subject and generates a structured roadmap with chapters, sub-chapters, key notions, governing formulas, and exam traps.
Universal document treatment: Any file format (.pdf, .docx, .pptx, .txt, .md, code files) is parsed client-side and indexed into an in-memory TF-IDF vector store for grounded, source-anchored generation.
Student-profile-friendly content adaptation: Each session is personalized to the student's academic level, learning style (Visual, Auditory, Kinesthetic, Reading, Writing, Fast-Paced like ADHD), cognitive conditions, free time, and exam deadline.
5-Box Leitner spaced repetition: Auto-generated flashcard decks with review intervals of 1, 3, 7, 14, and 30 days. Cards advance or reset based on recall, countering the forgetting curve.
Crash Test exam arena: Scenario-based multiple-choice quizzes that explain not only the right answer, but also the traps.
Dual-engine inference: Runs with Cloud Gemma 9B endpoints via an Express backend, or fully offline in-browser with Gemma 2-2B loaded via WebGPU and WebLLM. Every information about the users are stored on their phone, so no one can access those information except themselves.

In short, Gemma Atlas resolves the uncertainty of "where to start to learn this topic?" my siblings

Demo

https://drive.google.com/file/d/1n3mnPZX5-dKbbW8FCN-4aqKNBlsQW7kv/view?usp=drive_link

Code

Here is the GitHub link of the Gemma Atlas Project
https://github.com/RoxyMaxy/Gemma-Atlas/

How I Built It

Stack: React 19, TypeScript, Vite, Tailwind CSS, Lucide Icons, Recharts — with an Express + tsx backend serving the /api/session and /api/chat routes.
AI inference runs through @google/genai, cycling through a prioritized list of Gemma/Gemini Flash endpoints until one responds.
The /api/session route generates the structured study session (roadmap, concepts, flashcards, and quiz) as a single validated JSON payload.
The /api/chat route handles follow-up Q&A in streaming SSE mode.
When the client requests local mode, the backend short-circuits and defers to @mlc-ai/web-llm, which loads gemma-2-2b-it-q4f16_1-MLC weights directly into the browser's GPU memory via WebGPU.
Document parsing is ran client-side.
Plain text and code files are read as UTF-8 streams.
PDFs are scanned with binary regex patterns to extract BT...ET text objects.
Office formats are unzipped and traversed for XML text nodes (<w:t>, <a:t>).
Unrecognized format falls back to printable UTF-8 character extraction, so no file is ever rejected.
Extracted content is chunked using 128-dimensional FNV-1a hash embeddings with cosine similarity retrieval, which grounds the Gemma prompt in the student's source material.
The Leitner scheduler lives in StorageService and advances cards through five boxes based on recall. State is persisted in localStorage.
For The design, I used the MLH's website style, because I found it original and unique, and because I really enjoyed my path through this challenge, including the programming, the debugging and the solution seeking.

I used the DevRelay on Antigravity, using Gemini 3.8 medium, and Google AI Studio for troubleshooting.

Antigravity created the skeleton code and architecture. I coded the style layout myself, but it resulted in numerous bugs.
Therefore, I used Google AI Studio to polish the syntaxes, correct my own codes, and compile the final result.

Challenges

My first tiny bug through this challenge was the prompt: usually, I would define every single detail myself, and since I have researched and prepared before constructing this project, I knew almost everything about this product. Therefore, I already knew how I would want it to look like and how it would work. But having to study at the same time and lead my team in our college projects, I kind of just typed the overall features for the constraints, even though I explained the role, the context and the task well. Outputs were already explained in detail for the project parts.
The second problem was the syntaxes: I knew how to code the layouts, but I forgot some details and overlooked them (especially about the syntaxes and the animation) and it resulted on multiple bugs. I had to check all my errors myself, and when I didn't see any more, I finally used the Google AI Studio to check it out.
Getting Gemma to reliably produce large JSON payloads (for the roadmap stages, flashcards, quizzes) without malformed output required strict schema enforcement in the prompt.
A silent fallback bug was returning generic stage names whenever the primary endpoint was slow; fixing it meant rebuilding the /api/session route with a proper model cascade rather than a static fallback.

Running Gemma 2-2B in-browser via WebGPU required careful GPU adapter detection and graceful degradation for devices without hardware acceleration support.

Why Open Innovation Matters

To me, young students shouldn't have their learning notes, cognitive profiles, and personal study habits uploaded to proprietary platforms: for security and policies, mostly, but also for the token limits and paid plans.
Open-weight models running locally mean private inference with zero subscription cost.
For students with unreliable internet or families who can't afford private tutoring, open innovation represents a huge advantage for access. A closed API would leave people with limited, incomplete content, or lots of hallucinations.

Open innovation matters for my project, because it helps people widen their horizon and improve themselves by accessing to the most complete and detailed content possible. It represents a way for people to enjoy learning, access quality content with a more accurate, hallucination-reduced content. Hence, it helps people in their personal development.

What I Learned

On the AI side, I learned that prompt architecture is its own discipline. Getting Gemma to autonomously size a curriculum rather than filling a fixed template required understanding how temperature, schema constraints, and instruction phrasing interact. For example, too much freedom gets the output generic, but if it's too rigid, it feels canned and kind of boring, hostile, even, for some people. Finding that balance through my own experiences taught me more about how language models actually behave than any theoretical reading had.
On the engineering side, building client-side binary parsers for PDFs and Office XML gave me a real understanding of how those formats work. This knowledge is something I wouldn't have touched in a normal university project unless asked to. The same goes for implementing the vector index from scratch: writing FNV-1a hash embeddings and cosine similarity retrieval in plain TypeScript, rather than dropping in a library, forced me to actually understand what semantic search is doing.
Learning about people's struggles and challenges in learning taught me about the diversity of people, different ways on how our brain works, and how science has managed to design and tailor different ways for everyone to learn at their pace but not to fall behind. and designing for short attention spans taught me that information architecture matters as much as content.
Building Gemma Atlas also taught me to pay more attention to details and be careful with my actions (like typing correct syntaxes).
And on a personal level, I learned that having a real motivation to build something for people you actually care about changes how you solve problems. I wouldn't have really thought about every detail of this project about this learning topic if I did not have a clear overview of my siblings' situation, even if I have experienced the same problems.

Prize Categories

I entered in the Best Use of Gemma, because I was curious about this AI. I've heard of it frequently, but never used it, and this challenge brang me to research and explore it's potential.
Gemma is the core AI in my project.

My Agent Session

Gemma Atlas: Adaptive Private Study Workspace Architecture

You

I need a study workspace for my siblings. They need a full workspace that helps them master any topic they need, and they don't know what to start with. The workspace should be empathetic, adaptive and private, supporting Gemma 2 9B IT in the cloud and Gemma 2 2B IT locally offline.

Final Note

I built Gemma Atlas for my siblings and my mother. She works hard enough, but when she's not home, they deserve something that meets them where they are, helps them move forward and ease the teachers' work.