DEV Community: Aishani

Wabi-Sight: The Stone & Cloud Oracle — Nature Micro-Meditations with Gemma 2 & TabPFN

Aishani — Fri, 09 Oct 2026 20:36:34 +0000

🪨 The Stone & Cloud Oracle (Wabi-Sight)

Micro-Meditations from Nature's Shapes — Built for Hacktoberfest 2026: "Touch Grass"

🌿 What I Built — The Real Problem

We live in an era of epidemic screen fatigue. Even apps designed for the outdoors (trail navigators, species classifiers, AR games like Pokémon GO) suffer from the same fundamental flaw: they keep our eyes glued downward to a piece of glowing glass. Hikers walk past ancient canopies while staring at map pins; park visitors miss the rustle of leaves because they are managing digital notifications.

I built The Stone & Cloud Oracle (Wabi-Sight) to solve this exact problem by flipping the role of AI upside down.

Instead of demanding screen time, the Oracle uses open-source multimodal AI to make the screen the shortest, most disposable part of the experience (under 15 seconds). You spot an ephemeral organic pattern outdoors—puddle ripples, dried mud fissures, leaf veins, lichen on stones, or wandering cloud silhouettes—and take a quick snapshot.

The Oracle extracts the pattern's geometry, crafts a poetic metaphor, and then deliberately commands you to turn your phone face down in the grass, close your eyes, and listen to the world around you.

The screen dims into an unhurried breathing circle matching your local circadian rhythm, accompanied by gentle voice guidance and a resonant Tibetan singing bowl chime that tells you when your meditation is complete.

⚡ How It Works — The Core Idea & Technical Approach

The Stone & Cloud Oracle is built with a Dual-Engine architecture engineered to unite cutting-edge open-weight foundation models, tabular machine learning, and screen-minimizing ambient audio.

1. The 5-Step User Journey:

Spot Organic Geometry (< 5s): Spot a fleeting natural pattern on a trail or park.
Consult the Oracle (< 10s): Snap a quick photo with the mobile camera.
Multi-Model Synthesis:
- Prior Labs' TabPFN evaluates local circadian conditions (hour_of_day, temperature, cloud_cover, day_of_year) from a reference ecological dataset to calculate your optimal resonant breathing cadence (e.g., 5.5s resonant rhythm for midday equanimity, 7.0s deep exhale for dusk).
- Google Gemma 2 (Open-Weights) acts as a poetic naturalist (inspired by Bashō, Thoreau, and Mary Oliver), delivering a poetic metaphor, a philosophical reflection, and a tactile somatic grounding command.
- ElevenLabs Voice Synthesis speaks the guidance aloud in an unhurried, serene naturalist voice so you never have to read the screen.
The Screenless Immersion: The screen dims to a tranquil near-black view (#030604). You place the phone face down in the grass, close your eyes, and breathe with the spoken guidance until a Tibetan singing bowl chime marks the end.
Nature Scrapbook: The observation, poetry, and element badge are preserved in an offline field journal.

2. Prize Categories:

Best Use of Gemma ($200 Target): Powered by Google's open-weight Gemma 2 (gemma2-9b-it via Groq Cloud acceleration and local Ollama support). Gemma 2 serves as the poetic brain that transforms raw visual geometries into contemplative literature.
Best Use of TabPFN ($200 Target): We integrated Prior Labs' TabPFN tabular foundation model (pip install tabpfn). TabPFN analyzes ecological weather and seasonal data to dynamically predict circadian respiration cycles.
Best Use of ElevenLabs ($100 Target): Uses ElevenLabs' newest eleven_flash_v2_5 model (Voice ID: "Adam") for realistic documentary-style narration, backed by native browser Web Speech API offline fallbacks.
Best Use of Render ($200 Target): Fully containerized and deployed via a 1-click render.yaml blueprint on Render's free tier.
Best Use of Sentry Agent Tracing ($100 Target): Instruments the pipeline with Sentry spans (tabpfn.forecast, vision.analyze, gemma.synthesize) to monitor end-to-end latency.

3. Why Open Innovation Matters:

100% Offline Trail Independence: Deep forest ravines and national parks rarely have 5G signal. Proprietary cloud APIs fail in the wild; open-weight models run locally on edge hardware with zero external dependencies.
Privacy of Inner Life: Meditative reflections, backyard nature snapshots, and personal breath logs belong solely to the user. No personal biometric or location data is ever mined by ad networks.
Zero Cost Forever: Free open-weights allow anyone, any school outdoor group, and any community nature club to run the software forever at $0.

🎬 Demo — Real Scenarios & Live Links

🚀 Try the Live Web App:

👉 Live Deployed URL: https://stone-cloud-oracle.onrender.com

(Designed mobile-first: open on your smartphone or browser!)

🎥 Video Walkthrough

Watch the Wabi-Sight Demo Video:
(https://1drv.ms/v/c/bbab0edd6634b417/IQDno30x6Th_QJY10OC6etXrAa2IKleXIkxP4uOMRHWGv-k?e=uXjl1E)

🌲 A Real Trail Scenario:

The Spotting: While walking in a park, I spotted concentric ripples in a shallow rainwater puddle.
The Oracle's Revelation: > 🌿 **The Vision: "These ripples mirror the annual growth rings of an ancient cedar, expanding toward the mossy bank." > 🪨 **The Reflection: "Disturbance marks the surface for only a moment; the depth remains undisturbed and still." > 🌬️ **The Grounding: "Place your phone face down on the grass. Close your eyes, feel the sunlight on your eyelids, and take 5 slow breaths."
The Grounding: I set the phone face down in the grass. The screen dimmed completely, soft audio guided my breaths, and a peaceful Tibetan bowl chime sounded at the 30-second mark.
The Result: I spent 10 seconds on my phone and 5 minutes actually listening to the birds and feeling the grass beneath me.

💻 Code — The GitHub Repository

The entire codebase is open-source under the Apache 2.0 license:

👉 GitHub Repository: https://github.com/aisha453/Wabi-Sight

Built with ❤️ for Hacktoberfest 2026. Remember to close your laptop and go touch grass!

TinkerSight: An AI Friend for When You're Stuck With a Device

Aishani — Sun, 04 Oct 2026 05:27:11 +0000

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend

What I Built

I built TinkerSight, a vision-first AI technical guide for my friend who lives alone and sometimes struggles with unfamiliar appliances and everyday devices.

The problem is simple: when you're standing in front of an unfamiliar AC remote, washing machine, or device, the manual often isn't written for the one question you actually have.

TinkerSight lets someone show a device through a photo, explain what they want to do, and get simple, step-by-step guidance.

Its core idea is:

See → Ground → Simplify → Guide

It doesn't just try to recognize what's in the image. When possible, TinkerSight retrieves relevant official manufacturer documentation and uses it to ground the guidance. If the image is unclear or the situation could be dangerous, it asks for clarification or stops instead of confidently guessing.

I built TinkerSight because I wanted to create something that felt less like searching through a manual and more like having someone standing beside you when you need help.

Demo

Here is a short demo showing three situations:

Using Turbo Cool on a Blue Star AC remote, with the answer grounded in official manufacturer documentation.
Getting guidance for a washing-machine control.
A safety scenario involving a burning smell, where TinkerSight refuses to provide hazardous repair instructions and tells the user to stop.

Demo video:

Watch the TinkerSight Demo video:
https://1drv.ms/v/c/bbab0edd6634b417/IQDjln7aXVbrR5HvUco5p-YwAXIP2aJejHEnnWeCJaIsJGc?e=KX55aw

Code

The complete source code is available on GitHub:

https://github.com/aisha453/TinkerSight

The repository includes the React frontend, FastAPI backend, local AI integration, manufacturer-document grounding, and safety handling.

How I Built It

TinkerSight is built around Qwen3-VL 2B, an open-weight vision-language model running locally through Ollama.

The architecture is:

React + Vite → FastAPI → Ollama → Qwen3-VL 2B

The user uploads an image and describes what they want to do. The vision model first identifies the device, visible controls, brand information, and relevant context.

For supported manufacturers, TinkerSight then retrieves information from selected official manufacturer webpages and uses that information to ground the response.

I also added a deterministic safety layer for obvious hazardous situations such as exposed electrical wiring, gas leaks, smoke, sparks, burning smells, and internal appliance repairs.

One important design principle was: don't pretend to know something that isn't visible or verified.

For example, TinkerSight does not claim an exact appliance model unless it can actually read the model information.

Why Does Open Innovation Matter?

Open innovation made this project possible in a way that would have been difficult to achieve with a closed API alone.

Because TinkerSight uses an open-weight vision-language model through local inference, the AI can run on the user's own machine without sending every image to a proprietary vision API.

It also means the model layer can be replaced or improved without rebuilding the entire application.

More importantly, using open AI made it possible for me to experiment with the behavior of the system itself: how it interprets images, how it handles uncertainty, how manufacturer information is introduced, and when it should refuse to provide instructions.

For a tool dealing with someone's home, appliances, and potentially sensitive images, having the ability to run the AI locally is especially meaningful.

Prize Categories

No partner prize category claimed.

TinkerSight currently uses open-source/open-weight AI and local inference, but I did not add a partner technology solely to qualify for a prize category.

#grinding mode on

Aishani — Fri, 02 Oct 2026 16:01:39 +0000