DEV Community

Dhardingsea Developer
Dhardingsea Developer

Posted on

I asked an agent to make a product video. It wrote HTML and rendered an MP4 - Dota Companion

#ai #webdev #video #showdev

I needed two videos for a Chrome extension I built: a promo and a walkthrough. I did not want to open a video editor.

So I used HyperFrames — HeyGen's open-source (Apache 2.0) HTML-to-MP4 framework. Compositions are plain HTML and CSS. No JSX, no bundler, no build step. Timing lives in data attributes; motion comes from GSAP. A headless Chrome seeks the timeline frame by frame and FFmpeg encodes it.

That design matters for one specific reason: an agent can read and edit a plain HTML file. It cannot meaningfully iterate on a timeline in a GUI.

Both videos below were authored as HTML, narrated with a local TTS model, and rendered locally. Here's what came out, then the three things that actually cost me time.

The promo (29s)

The walkthrough (58s)

Numbers, for calibration: 1080×1080, 867 frames for the promo, 56 seconds to render. The walkthrough took just over 2 minutes. Both passed the framework's check gate — 0 runtime errors, 0 layout issues, and 22/22 and 23/23 WCAG AA contrast checks respectively.

Failure 1: the CDN script that fails silently

The init scaffold pulls GSAP from a CDN:

<script src="https://cdn.jsdelivr.net/npm/gsap@3.14.2/dist/gsap.min.js"></script>
Enter fullscreen mode Exit fullscreen mode

My render container proxied the shell but not the browser. curl worked. Chrome had no network.

So GSAP never loaded, window.__timelines was never registered, and the render would have produced a static video — with a zero exit code and no error. Nothing in the output says "your animation library is missing."

curl -s -o vendor/gsap.min.js https://cdn.jsdelivr.net/npm/gsap@3.14.2/dist/gsap.min.js
Enter fullscreen mode Exit fullscreen mode

Vendor everything. Fonts too. This is the failure mode I'd most expect someone else to hit, because it doesn't announce itself — you just get a video where nothing moves, and you assume you wrote the timeline wrong.

Failure 2: timing the cuts before generating the audio

My first instinct was to write the storyboard, assign each frame a duration, then generate narration to fit. That's backwards. Speech doesn't land where you guess it will, and the drift compounds — by the third frame the voice is talking over the wrong slide.

Invert it. Generate audio first, one file per sentence, and read the real duration of each:

npx hyperframes tts seg/s1.txt --voice am_michael -o seg/s1.wav --json
# → {"durationSeconds": 4.48, ...}
Enter fullscreen mode Exit fullscreen mode

Then derive every cut point cumulatively — start[i] = start[i-1] + dur[i-1] + gap — and let those offsets become the data-start values. A 0.3s inter-segment gap reads as natural breathing.

Result: a 28.90s video against 28.904s of audio, and 58.26s against 58.26s. No drift, no nudging.

Render silent, mux at the end, and verify both streams actually exist:

ffmpeg -i video.mp4 -i vo.wav -c:v copy -c:a aac -shortest out.mp4
ffprobe -v error -show_entries stream=codec_type out.mp4
Enter fullscreen mode Exit fullscreen mode

That last check matters — a mux that silently drops the audio stream still produces a file that plays.

Failure 3: real screenshots when the browser has no network

The framework is strict that screenshot slots hold real captures, not mockups. Correct rule. But my capture environment couldn't reach the API, so loading the extension gave me an empty UI.

The wrong fix is to mock the interface. The right one is to fetch the real payload out-of-band and seed it through the app's own cache path:

await page.evaluate(d => new Promise(r => chrome.storage.local.set({
  settings: { refreshOnOpen: false },     // the app's documented cache-only path
  ['cache:123']: { ts: Date.now(), ...d } // shape the app itself writes
}, r)), realData);
await page.reload();
Enter fullscreen mode Exit fullscreen mode

Real data, real render code, real pixels. Two conditions make this legitimate and both are required: the data is genuinely fetched, and the cache shape is one the application actually writes. Invent a shape the app never produces and you're fabricating a screenshot.

One caveat I'd underline: inspect every offline capture. Mine painted the full dashboard but left one field empty. Ship that to a store listing and it reads as a broken product.

An honest note on the content

The first cut of the promo showed my real stats — including a 0 improvement streak and a loss streak — directly under narration about tracking improvement.

I swapped the shot rather than faking the numbers. Worth saying out loud, because the tempting move when your generator can produce any frame you describe is to describe a better one.

Would I do it again

For this job, yes. The whole pipeline is text: HTML, CSS, a script file, and CLI verbs. Everything is diffable, re-renderable, and re-narratable by changing a sentence and re-running. Editing a word of narration is editing a text file, not re-recording.

It is not a replacement for a real editor when you want craft. It's very good at "generate this deterministically and regenerate it when the product changes" — which is most product video.

The extension in the videos is Dota Companion — free, local-first Dota 2 stats. More of my projects at dhseadev.online.

Top comments (0)