DEV Community

Cover image for CineMuse 🎬 — an AI film mentor for the short films we can't stop making
Kaustubh Upadhyay
Kaustubh Upadhyay

Posted on

CineMuse 🎬 — an AI film mentor for the short films we can't stop making

#devchallenge #weekendchallenge #ai #webdev

This is a submission for Weekend Challenge: Passion Edition

What I Built

Everyone I know who makes short films has the same problem: nobody watches their drafts.

Friends say "nice!" after three seconds. Feedback threads die. So the film you've poured your nights into gets edited on instinct alone, and you publish it still wondering — does the opening drag? Does the ending land?

CineMuse is a private screening room for those films. You upload your short film (animated or live action, up to 12 minutes), tell it your plot and what you meant to say, and Google Gemini watches the whole thing — every frame. Then it hands you:

  • 🎯 The annotated cut — time-coded notes pinned as clickable markers on your timeline, Frame.io-style, except the AI wrote them. Click a marker, the film seeks there.
  • 🎬 A scene-by-scene breakdown with real thumbnails (captured in your browser)
  • 📈 An emotional arc graph of your story's beats — it draws itself as you scroll
  • 🩺 A plot doctor — stakes, alternative directions, ending ideas, what to cut
  • 💬 "Ask the Muse" — a chat grounded in what the AI actually watched. Ask "how do I fix my opening?" and it answers with timestamps from your footage.

No login. Sessions save in your browser. And because it's a passion project about passion projects, the whole UI is a cinema: velvet curtains that part when your film lands on the silver screen, a NOW SHOWING marquee with chasing bulbs, a film-leader countdown while the muse watches, and film strips rolling through the footer.

Demo

🔗 Try it live: https://cinemuse-orcin.vercel.app

Frame-by-frame Film Analysis

Emotional Arc throughout the film

Plot Doctor and Craft Report

Code

🗂️ GitHub: https://github.com/KaustubhUp025/cinemuse (MIT)

You can run your own CineMuse in about 3 minutes — all it needs is Node 18+ and a free Gemini key (no credit card):

git clone https://github.com/KaustubhUp025/cinemuse.git && cd cinemuse
cp .env.example .env.local   # paste your key from aistudio.google.com/apikey
npm run dev                  # → http://localhost:3000
Enter fullscreen mode Exit fullscreen mode

There's deliberately nothing to install: no framework, no build step, zero npm dependencies. The frontend is vanilla HTML/CSS/JS (every animation hand-rolled), the backend is four small Vercel serverless functions, and the local dev server is a single Node file that mimics Vercel's runtime.

How I Built It

The interesting problem: how do you analyze a whole video on a free tier?

Video tokens are expensive, so CineMuse runs a map-reduce pipeline over the film:

  1. The browser uploads the film directly to Gemini's Files API (a resumable, CORS-enabled upload — my server only signs the session, so the API key never leaves the backend and there are no proxy size limits).
  2. MAP: gemini-flash-lite watches the film in ~20-second clips using videoMetadata time offsets at low media resolution — cheap, parallel, timestamped observations.
  3. REDUCE: gemini-flash never sees a single video token. It gets the text observations + your plot + your focus areas, and writes the full critique as structured JSON (responseSchema) — scores, annotations, scenes, arc, plot doctor.
  4. "Ask the Muse" reuses the saved observations as chat context — follow-up questions cost zero video tokens.

A 60-second film ≈ 7 API calls. The whole thing is vanilla JS + Vercel serverless functions — no framework, no build step, all the motion is hand-rolled CSS.

Fun bugs from the weekend: Google's resumable uploads demand 8MB chunk granularity (which killed my proxy plan — direct browser upload saved it), and Gemini's thinking tokens silently ate my chat token budget, truncating the muse mid-sentence.

Prize Categories

Cinemuse is built with extensive use of Gemini AI and that is why I would like to submit this project under Best Use of Google AI

The Passion Angle

CineMuse exists so that work finally gets seen properly: watched all the way through, taken seriously, and answered with the kind of specific, warm, timestamped feedback a mentor would give.

Built solo over the weekend. If you've got a draft sitting in a folder — give it a screening. 🎞️

Stack: Google Gemini (Files API + multimodal video understanding) · Vercel serverless · vanilla HTML/CSS/JS

Top comments (0)