I built DreamLens, a multimodal applet powered by Google AI Studio that transforms any idea, story, or doodle into a short animated movie.
Everyone has imagination—kids tell fantasy tales, writers create worlds, gamers describe epic battles—but most can’t turn them into visuals and sound. DreamLens solves this by using text, voice, and image understanding to automatically generate storyboards, narration, and background audio.
How I Built It
Frontend: React (simple input box + doodle/image upload + voice mic)
Backend: Python Flask on Cloud Run
AI: Gemini 2.5 Pro for multimodal input
Text + Voice processing via Gemini Live API
Image/doodle understanding via Gemini 2.5 Flash Image
Script/narration generation via Gemini text model
Deployment: Google Cloud Run
Other Tools: Tailwind (UI), Firebase (story saves), optional text-to-speech
Key Features
📝 Story-to-Movie → User types or narrates a story, Gemini generates visuals + narration.
🎤 Voice-to-Story → Kids or users can simply tell a story and see it animated.
🖼️ Doodle-to-Scene → Upload a doodle/sketch, Gemini turns it into a polished scene.
🎙️ Custom Narration Styles → Funny, dramatic, bedtime storyteller, etc.
🎶 Background Music → Automatically matched to story mood (epic, spooky, calm).
🔄 Interactive Regeneration → Users say “Add a villain,” “Make it funnier” → instantly updated.
Challenges I Faced
Synchronizing story + visuals + narration into one flow.
Handling latency in image + audio generation without breaking immersion.
Designing an interface that’s both simple for kids and powerful for creators.
What’s Next
📚 Education Mode → Teachers can create animated lessons (history, science).
🎮 Gamers Mode → Auto-generate cinematic recaps of role-playing campaigns.
📱 Mobile App → “Tell a story, watch it come alive” in seconds on phones.
🌍 Community Sharing → Users can share their AI mini-movies with friends.
Submission Category
🔹 Innovation and Creativity
🔹 Technical Implementation
🔹 User Experience
🔹 Effective Use of Multimodal Features
Team Members
Rohit Mane
Top comments (0)