YouTube Storybook Converter

#devchallenge #googleaichallenge #ai #gemini

This is a submission for the Google AI Studio Multimodal Challenge

What I Built

I created the YouTube Storybook Converter, an applet that magically transforms any YouTube video into a beautifully illustrated, narrated digital storybook. As a parent and creator, I've always been fascinated by the idea of making vast educational and entertainment content on platforms like YouTube more accessible and engaging for young children. My applet bridges this gap, taking a simple URL and turning it into a captivating, multi-sensory reading experience.

The process is simple: a user pastes a YouTube link, and the applet intelligently crafts a child-friendly narrative, illustrates each page with whimsical, AI-generated art, and even provides an audio track to read the story aloud. It’s designed to spark imagination and offer a fresh, interactive way for kids to enjoy content.

Demo

Live Applet: Link to my deployed applet

Here’s a walkthrough of the experience:

Start with a Spark: The user is greeted with a clean, inviting interface where they can paste any YouTube video URL.
The Magic Unfolds: With a single click, the AI gets to work. A friendly loader keeps the user informed as the story is written, illustrations are painted, and the narration is prepared.
A Storybook Comes to Life: The final storybook is presented with vibrant illustrations, clear text, and intuitive navigation. The "Read to Me" feature brings the story to life with voice narration.
Become the Artist: What makes this truly special is the ability to edit the illustrations. By clicking the 'Edit' button, the user can type a simple command like "add a happy little cloud" to modify the image, making each story uniquely their own.

(Note: Since gemini-2.5-flash-image-preview is a preview model, a video is the best way to capture the full, interactive magic of the image editing feature in action.)

How I Used Google AI Studio

Google AI Studio was my creative sandbox for this project. I extensively used it to prototype and refine the prompts that power the entire experience. The ability to quickly test different models and configurations was crucial.

Here’s the lineup of Google AI models I orchestrated:

gemini-2.5-flash: This was the storyteller. I prompted it to take a YouTube URL, imagine its content, and then spin a simple, multi-paragraph tale suitable for children. The key was using its JSON output mode to get a structured story that my app could easily parse into pages.
imagen-4.0-generate-001: This was the artist. For each paragraph of the story generated by Gemini, I used Imagen to create a unique, storybook-style illustration. The text from the story became the direct input for the image generation, creating a seamless link between words and visuals.
gemini-2.5-flash-image-preview: This is where the true multimodal magic shines. For the image editing feature, this model takes both an existing image and a user's text prompt as input to generate a new, modified image. This powerful capability is what makes the storybook interactive and deeply personal.

Multimodal Features

The heart of this applet is its rich blend of multimodal features that create a cohesive and immersive experience.

Conceptual Interpretation (URL -> Story): The journey begins with gemini-2.5-flash interpreting the concept behind a YouTube URL and translating it into a new modality: a written narrative. This leap from a link to a creative story is the first step in the multimodal transformation.
Text-to-Image Synthesis: The applet creates a direct and meaningful link between the written word and visual art. Each story paragraph (text) is passed to imagen-4.0-generate-001 to generate a corresponding illustration (image). This ensures that the visuals perfectly match the narrative on every page.
Interactive Image Editing (Image + Text -> Image): This is the flagship feature. The user can provide feedback in natural language (text) to alter a visual (image). When a user types "make the robot smile," gemini-2.5-flash-image-preview processes both the current illustration and the text command to produce a new image. This interactive loop between user text and AI-generated imagery is a powerful demonstration of multimodal input leading to a creative visual output.
Text-to-Speech Narration: To complete the sensory experience, the generated text for each page is converted into spoken word using the browser's built-in capabilities. This adds an auditory layer, transforming the digital book into a listen-along experience, which is perfect for younger audiences.