This is a submission for Weekend Challenge: Dog Days Edition
What I Built
We know what our dogs do all day.
We don't really know what that day feels like to them.
I built A Dog's Day to explore that idea as a single interactive experience rather than another dog utility.
You start by scrolling through a Golden Retriever's morning: waking up, waiting for breakfast, heading outside, meeting another dog, losing a favorite red ball, and eventually finding it again.
The story is built around scroll-driven visual beats, so the page behaves more like a short interactive film than a collection of sections.
Then, instead of simply ending the story, it asks:
What if you could experience the world through the eyes of your favorite animal?
That leads into the second half of the project.
Choose an animal and talk to it.
The Dog character is a fictional Golden Retriever with context from the story you just experienced. Ask why the ball mattered, why everything outside smells interesting, or what it noticed when it met another dog, and the conversation is grounded in that character's perspective rather than a generic "tell me about dogs" chatbot.
The experience currently includes five animal perspectives:
- Golden Retriever
- Domestic Cat
- European Rabbit
- Red Fox
- Gray Wolf
The idea is simple: the same world can feel very different depending on who is experiencing it.
Demo
Live: https://dev-cha-git-main-jyotishpabbisetti2025-1209s-projects.vercel.app/
Video: https://youtu.be/WRw55khrIBI
The demo shows the project as one continuous experience:
- the opening story
- the missing-ball sequence
- the transition into the animal experience
- a voice conversation with the Dog
- interruption while the character is speaking
The part I wanted to get right is the transition from watching the story to being able to talk to the character inside it.
Code
GitHub: https://github.com/Jyotish08/dev-cha
The project is a single Next.js application using the App Router.
The story is organized around reusable components for scroll progress, scene beats, media, typography, and the transition into the interactive chapter.
The conversation system has separate animal configurations, a voice conversation state layer, a Gemini Live WebSocket client, and a fallback API path.
How I Built It
The first half is a small story engine.
Each chapter has authored visual beats and text, and the user's scroll position determines which beat is active. The story uses sticky full-screen sections with Motion to control the pacing and crossfade between the story frames.
For example, the four main chapters are:
Morning Comes Quietly — the dog wakes.
Breakfast — anticipation, eating, and the empty bowl.
The Walk — the neighborhood, another dog, and the dog's sense of smell.
The Missing Ball — chase, loss, searching, discovery, and getting the ball back.
The visual layer combines 1536×1024 story frames with short ambient MP4 overlays. The frames carry the actual narrative beats, while the videos add movement without making every moment depend on a full video sequence.
The second half is where the architecture changes.
Instead of sending recorded speech to a normal text-generation endpoint and then using browser text-to-speech, I connected the conversation directly to Google's Gemini Multimodal Live API over a persistent WebSocket.
Microphone audio is captured as 16 kHz linear PCM and streamed to Gemini. The returned 24 kHz audio is scheduled through the Web Audio API for continuous playback.
That matters because the interaction is supposed to feel like a conversation, not:
record → wait → generate → speak.
The Live connection also supports interruption. If the Dog is speaking and the user starts talking, the current playback can be stopped and the conversation can continue from the new turn.
The animal personalities are configured separately. The Dog's system instructions include the fictional events from the story itself, including breakfast, the walk, the other dog, and the red ball. That lets the conversation continue from the story instead of starting from zero at the bottom of the page.
There is also a fallback path for the parts of the experience that depend on external AI services. Gemini rate limits, network conditions, microphone permissions, and temporary connection failures can all affect a live session, so the project does not silently pretend a failed request succeeded. A secondary Gemini text-streaming path and an offline scripted mode are available when the live connection cannot be established.
The goal was not to make the project depend on AI for every part.
The goal was to use Google AI exactly where it changes the experience:
the story shows you the animal's world; Gemini Live lets you talk from inside it.
Prize Categories
Best Use of Google AI
I am submitting A Dog's Day for Best Use of Google AI.
Gemini is not being used as a wrapper around a normal chatbot.
The interactive chapter uses the Gemini Multimodal Live API for the core voice experience: continuous microphone input, streamed spoken responses, story-aware animal personas, and interruption handling.
Remove Gemini Live from that part of the project and the central interaction changes completely.
The technology is what allows the final chapter to move from:
"Here is a story about a dog."
to:
"You can actually talk to the animal whose morning you just followed."
Top comments (0)