I recently researching how to make a short vertical video with vibe coding with painfully detailed prompts.
I wrote the narration, timing, aspect ratio, color palette, subtitle rules, transitions, evidence screenshots, music direction, and even a list of visual clichés in the prompts to avoid mistakes. I described the result as an “AI search investigation,” with real model responses, highlighted fields, budget comparisons, and a clean monitoring interface.
What came back was a valid MP4 that the resolution was correct, the scenes appeared in roughly the right order, the numbers were accurate; but it also looked like an animated PowerPoint.
There were circles, rectangles, progress bars, and very large text. One scene showed “300,000” inside a ring. Another showed a range next to a flat horizontal bar. Most of the actual visual story—AI interfaces, search evidence, cost items, ranking changes—had quietly disappeared.
The AI had followed the checklist. It had also completely missed the point.
The prompt was detailed, but not visually specific
My first reaction was that the prompt still wasn’t detailed enough.
Then I looked at it again. It was already several pages long. Adding another paragraph about making the video “more polished,” “more cinematic,” or “more high-tech” probably would not have fixed anything.
The problem was that I had mixed two different things into one document:
- Production requirements
- Creative direction
Production requirements are easy to verify. The video is either 1080×1920 or it isn’t. A scene lasts six seconds or it doesn’t. A subtitle appears inside the safe area or it doesn’t.
Creative direction is much harder.
“Make it feel like a real investigation” sounds clear to a person, but it leaves dozens of decisions unresolved. Should the interface resemble a browser, a terminal, or an analytics dashboard? How dense should the screen be? How large should the subtitles feel on a phone? What makes a frame feel investigative instead of corporate?
When these choices were missing, the agent used the safest visual vocabulary it knew: cards, circles, large numbers, and smooth slide transitions.
In other words, it converted ambiguity into geometry.
Programmatic video solves a different problem
This experience made me look more carefully at the tools underneath AI-generated videos.
Remotion treats React code as the source of truth. Scenes can be written as components, connected to data, previewed in the browser, and rendered in batches. It is especially appealing when the video contains repeated layouts, dynamic data, reusable components, or content that needs to be generated at scale.
HyperFrames takes a different approach. A composition is written with HTML and CSS, while timing is described through data attributes and seekable animations. It can work with GSAP, Lottie, Three.js, CSS animations, and other browser-native tools, then render the result through headless Chrome and FFmpeg.
The distinction matters.
Remotion gives you the structure and component model of React. HyperFrames gives an agent a format it already generates constantly: HTML and CSS. Neither one automatically provides taste, but both make timing, layout, assets, and rendering editable instead of hiding them inside a generated video file.
That is the real advantage of programmatic video for me. It makes mistakes repairable.
If a subtitle is too large, I can change one token. If every scene has too much empty space, I can update the shared layout. If the timing changes after I record the narration, I can adjust a configuration file instead of manually moving every clip along a timeline.
But code gives you control only after someone has made the right visual decisions.
What I would do differently next time
The biggest mistake was generating the entire video before validating its visual language.
Next time, I would begin with the hardest 10 to 15 seconds. In this case, that would be the moment when three AI models recommend the same brand and the fourth gives a surprising answer.
That sample should answer a few questions before the full render starts:
- Do the screenshots feel like evidence or decoration?
- Is the text still readable on a phone?
- Does the animation have variation in speed and pauses?
- Does the interface feel like an investigation?
- Are the visual layers doing more than repeating the narration?
If those seconds do not work, rendering another minute in the same style only produces a longer mistake.
I would also replace vague aesthetic words with visible rules.
Instead of saying “make it feel advanced,” I could specify a graphite background, one accent color, small technical labels, restrained scanning animations, real screenshots as the main layer, and no oversized rounded cards.
Instead of saying “use modern subtitles,” I could define the font, weight, maximum width, line count, position, shadow, and which three sentences are allowed to become full-screen titles.
Reference frames would help even more. A few annotated screenshots can explain hierarchy, density, and texture far better than another 500 words of adjectives.
Assets should be treated as dependencies
Another lesson was that missing visual assets should not silently become abstract shapes.
If a scene requires four real AI responses and only two screenshots exist, the agent should stop and mark the other two as missing. A clearly labeled placeholder is more useful than a polished animation that pretends the evidence was never needed.
I now think video assets should be handled much like code dependencies:
- List every required asset.
- Map it to a scene.
- Validate that it exists.
- Flag missing or low-resolution files.
- Only then build the full composition.
That sounds less exciting than prompting a complete video in one shot, but it prevents the entire production from drifting away from the original idea.
Maybe AI should build the system before it builds the video
I still think AI coding agents are genuinely useful for video production.
They can scaffold scenes, centralize timing, create reusable animation components, generate subtitle files, inspect keyframes, and render multiple versions. Tools like Remotion and HyperFrames make that workflow much more accessible.
What I trust less now is the one-shot request: “Here is my script. Please make the finished video.”
A better division of work might be:
- - The human defines the visual language.
- - The agent builds a reusable system around it.
- - Both iterate on a short sample.
- - The full video is rendered only after that system survives review.
That process is slower during the first 15 seconds and much faster during the remaining 60.
I’m still figuring out where prompting ends and creative direction begins. Maybe programmatic video will eventually make that boundary disappear. For now, it seems that AI is very good at building the thing it can verify—and very willing to approximate everything else with a rectangle.
How do you handle this gap? Do you provide visual references, prototype one scene first, or accept that the final layer of creative direction still needs to be done by hand?
Top comments (0)