Veronika Kashtanova

Posted on Feb 19 • Edited on Mar 4

Gemini Tales: Turning Screen Time Into Active Adventure🧸

#agents #buildmultiagents #gemini #adk

Education Track: Build Multi-Agent Systems with ADK

This post is my submission for DEV Education Track: Build Multi-Agent Systems with ADK.

What I Built

I built Gemini Tales, an interactive storytelling experience that blends real-time AI conversation with physical activity. It solves a haunting statistic: 80% of children today don't move enough. While technology is often seen as the cause of sedentary behavior, I wanted to turn the screen into a catalyst for movement.

📹 Watch the Vision: See how we turn sedentary screen time into an active adventure.

Gemini Tales doesn't just tell a story—it sees your child, hears their voice, and asks them to ACT. Every physical movement becomes part of the magic.

Cloud Run Embed

The project is currently running in Google Cloud Run:

🧚 The Experience: Live Multimodal Storytelling

The frontend is a direct bridge to Gemini Live API, enabling unified Voice + Vision interaction in real-time.

Features That Create Magic ✨

Feature	What It Does	Tech Stack
🎙️ Stable Voice Live	Interruption-aware, low-latency conversation. Child speaks and changes story path anytime.	Gemini Live API
📸 Visual Awareness	Real-time video stream (1 FPS) lets AI "see" costumes, toys, movements.	Gemini 2.5 Flash Native Vision
🎨 Dynamic Illustrations	Watercolor-style art that evolves with the plot.	Gemini 3.1 Flash Preview
⚡ Agent-Driven Context	Before the story starts, Storysmith agents research & craft unique plot.	Google ADK Backend
🎮 Interactive Challenges	AI pauses story for "Hero's Challenges"—physical actions detected via video.	Computer Vision

🤖 The Brain: Multi-Agent Story Engine

The backend is a distributed multi-agent system built with the Google Agent Development Kit (ADK) and the A2A (Agent-to-Agent) protocol. This ensures specialization, reliability, and scalability.

🎭 Meet the Agents

Agent	Role	Specialty	Model
🔍 Adventure Seeker	Physical activity planning	Multi-step reasoning for movement-based quests	Gemini 2.5 Flash + Google Search
⚖️ Guardian of Balance	Safety gatekeeper	Strict age-appropriate & movement-focused validation	Gemini 2.5 Flash (Temp 0.1)
✍️ Storysmith	Narrative weaver	Advanced storytelling & character depth	Gemini 2.5 Pro
🪄 Orchestrator	Pipeline coordinator	Manages state & flow between agents	Gemini 2.5 Flash

Architecture Highlights

┌─────────────────────────────────────┐
│    Frontend (React + Live API)       │
│  Voice • Vision • Real-time Feedback │
└────────────────┬────────────────────┘
                 │ WebSocket
┌────────────────▼────────────────────┐
│   API Gateway (FastAPI)              │
│   Authentication & Rate Limiting     │
└────────────────┬────────────────────┘
                 │ A2A Protocol
    ┌────────────┼────────────┬────────────┐
    │            │            │            │
┌───▼──┐  ┌─────▼──┐  ┌─────▼──┐  ┌────▼───┐
│Seeker│  │Guardian│  │Storysmth   Orch.   │
└──────┘  └────────┘  └─────────┘  └────────┘

For a detailed deep-dive into the system design, see ARCHITECTURE.md.

🚀 From Tutorial to Hackathon Reality

This project started as a journey through the Build Multi-Agent Systems with ADK track. The tutorial taught me Agent-to-Agent (A2A) communication and state-based orchestration—the exact foundation I needed for my entry in the Gemini Live Agent Challenge 🎬

I took those core architectural patterns and pivoted toward something bigger: an AI Nanny that inspires children to move. The same ADK logic that powers educational tools now powers interactive, movement-based storytelling.

🛠️ Tech Stack

Layer	Technology
Frontend	React 19, Vite, TypeScript, Tailwind CSS
AI Intelligence	Gemini Live API, Gemini 2.5 Flash/Pro, Gemini 3.1 Flash Preview
Backend Framework	Google ADK, Agent-to-Agent (A2A) Protocol
Infrastructure	FastAPI (Python 3.12), WebSockets, Google Cloud Run
Observability	OpenTelemetry, Google Cloud Trace
Dev Tools	Antigravity IDE for agentic orchestration

📂 Getting Started

Prerequisites

Python 3.10+ & Node.js 20+
uv for Python dependency management
Google Cloud Project with Vertex AI enabled

1. Backend Launch

Easy Mode (Windows):

# Starts all 5 services with automatic cleanup
.\run_local.ps1

Manual Launcher:

# Start distributed agents (Leaf Services)
uv run shared/adk_app.py agents/researcher --port 8001 --a2a
uv run shared/adk_app.py agents/judge --port 8002 --a2a
uv run shared/adk_app.py agents/content_builder --port 8003 --a2a

# Start Orchestrator & API Gateway
uv run shared/adk_app.py agents/orchestrator --port 8004
uv run app/main.py

2. Frontend Launch

cd frontend
npm install
npm run dev

Visit http://localhost:5173 and start creating stories! ✨

📚 Key Learnings

Specialization > Monoliths

I was surprised at how much more reliable the system became when I stopped relying on one giant prompt and started treating agents like a specialized team with distinct responsibilities.

The Power of A2A Protocol

Implementing Agent-to-Agent communication was challenging, especially handling Google Application Default Credentials (ADC) on Windows. But once it clicked, the elegance of distributed agents became clear.

Movement Changes Everything

The most rewarding part? Seeing a child leap off the couch when the AI asked them to "show me how you jump." Screen time transformed from sedentary consumption into active play.

📂 Open Source & Reproducibility

The full source code—including ADK orchestration logic, deployment scripts, and frontend—is available on GitHub:

👉 GitHub: vero-code/gemini-tales

Features:

✅ Full Docker & Cloud Run deployment
✅ Multi-agent architecture with A2A protocol
✅ Live API integration with fallbacks
✅ Comprehensive ARCHITECTURE.md for deep-dives

🚧 Current Status & What's Next

Gemini Tales is in active development for the Gemini Live Agent Challenge. The multi-agent backend is fully operational, and I'm continuously refining:

🎬 Multimodal streaming: Synchronizing text, generated images, and voice.
🎯 Real-time movement detection: Analyzing video feed for physical actions.
🏆 Gamification System: Badges and rewards to motivate physical activity.

New updates coming soon:

🛡️ Strict Interaction Logic: Implementing "anti-cheat" verification so the agent visually confirms actions instead of relying on voice.
⚡ Performance: Optimizing agent loading times for instant storytelling.
🎥 Live demo video showcasing the full experience.

Stay tuned as we turn screen time into move time! 🚀🧸