What I Built
PIANIST — Touch Grass is an open-source AI music-learning experience designed to get learners away from the screen and turn the real world into part of their music lesson.
Instead of spending an entire lesson looking at an app, learners receive an Outdoor Music Mission:
GO OUTSIDE → LISTEN → DISCOVER → RETURN → PLAY → CREATE
They can put their phone away and listen to the world around them—footsteps, birds, water, traffic, repeating patterns, musical contours, or even silence.
When they return, PIANIST transforms what they discovered into a musical activity: a rhythm exercise, melody exercise, piano challenge, improvisation prompt, or creative composition idea.
The goal isn't simply to teach songs. It is to develop active listening, musical awareness, creativity, and musical independence.
Don’t use AI to keep people on the screen. Use AI to help them experience the world.
Demo
https://youtu.be/iKHdvasGzMs?si=MUS-lLfoZWNJorGA
Code
https://github.com/Zenieverse/pianist-touch-grass
Supporting documentation:
How I Built It
PIANIST Touch Grass was built with Google AI Studio and uses Gemma as the musical reasoning layer.
The architecture is:
Real-world sound → Acoustic features → Gemma → Musical interpretation → Personalized exercise → Piano → Learner
The browser captures a short listening window and extracts structured acoustic features such as energy, spectral characteristics, tempo/pulse information, rhythmic patterns, and pitch-direction trends.
Raw audio is not sent to the Gemma endpoint. The server receives structured acoustic features plus an optional learner description.
Gemma then reasons about those observations and generates a structured, pedagogically meaningful musical activity.
The current implementation uses:
- gemma-4-31b-it as the primary model
- gemma-4-26b-a4b-it as an automatic fallback
- a deterministic local heuristic provider as an offline fallback
- Web Audio API for local acoustic feature extraction
- an AI-provider abstraction
- interactive piano exercises and performance verification This makes Gemma a genuine part of the product's core learning loop—not just a chatbot added to the interface. ## Why Does Open Innovation Matter? Open AI models make it possible to experiment with new ways of connecting artificial intelligence with real-world learning. For music education, that means we can explore experiences where AI doesn't demand more screen time. Instead, AI can encourage people to listen, move, observe, create, and then return with something meaningful to play. An open-source implementation also makes the architecture inspectable and gives educators, musicians, developers, and researchers a foundation they can extend with new outdoor missions, musical exercises, models, instruments, and learning approaches. ## My Agent Session Built using Google AI Studio Build Mode. I did not use a separate DEV Agent Session / DevRelay coding-agent session for this project. ## Categories ### Best Use of Gemma PIANIST Touch Grass uses Gemma as the reasoning layer between real-world listening and musical practice. This is not a generic chatbot wrapper. The learner goes outside first. The browser extracts structured observations from what they hear. Gemma interprets those observations and generates a musical exercise that changes what the learner actually plays.
Top comments (0)