HyperFrames — Blackbox in a Whitebox

M Habib ur Rehman — Thu, 10 Sep 2026 20:10:23 +0000

For the last few days, I've been trying to build a simple video editor for my own use case. And honestly, the rendering wasn't the problem — the export wasn't either.

It was the timeline. The playhead. The clip bars. The drag. Split. Resize. Track layering. The scrub loop that has to stay in sync with the preview while the user drags something. All of it — I was forced to write myself.

And it was a headache. A real one.

I started looking for something that could handle these common issues for me — and that's when I found something super interesting: HyperFrames.

And then I watched every tutorial on it

Every article. Every YouTube video. Every thread. And everyone explained it the same way.

"It's just an HTML file."

"Your video is just an HTML file."

"Use HyperFrames in the back and it renders."

Cool. Great. Very clear.

And then — without fail — every single one of them would bring in Claude Code. Or GPT Codex. Or some other AI agent. And they'd say: tell the AI to build a video following this prompt. And the AI would build it. And the best part is, it actually would. But that was it. That was the tutorial.

And I sat there thinking... okay, but what actually happened?

That was a blackbox to me.

Even though, everyone kept saying "nothing is a blackbox, everything is in front of you, it's just HTML, it's open source, you can read it." And yeah. Technically true. But I couldn't see how it worked. I could see the inputs. I could see the outputs. I could not see the middle.

And then, somewhere in the middle of exploring things here and there, a thought hit me.

What if this handles all the behind-the-scenes for me?

What if I don't have to write the timeline. The playhead. The scrub loop. The clip model. What if HyperFrames already solved all of that, and I just build my own UI on top of it?

That thought is why I'm writing this article.

So what is HyperFrames, really?

Let me put it in the simplest way I can, for someone who's never touched it.

HyperFrames — an AI agent framework — turns HTML into video.

That's it. That's the pitch. You tell the AI what you want, it writes the HTML, you press render, and you get an MP4.

Sounds too simple to be useful, right? Stay with me, because the simplicity is the whole point.

Think about it. Animation, transitions, text flying in, images sliding, colors fading — all of this already exists on the web. It's called HTML, CSS, and JavaScript. Every website you've ever visited uses it. And every AI you've ever talked to can write it.

So I asked myself: if the browser can already do all of this, why do we need a separate tool to make videos?

The answer: we don't. We just need to teach the browser to hand over the frames one at a time.

That's what HyperFrames does. It opens your HTML in a headless browser, moves a playhead through time frame by frame, takes a screenshot at every step, and stitches them together into a video.

Simple.

But that's not where the story ends. That's actually where the interesting part begins.

The question that I believe every tutorial was missing

So we or an AI can write any animation in HTML and that becomes a video, right?

but then why?

Why does that HTML only work with HyperFrames' video editor studio? Why won't it magically work with any famous editors like CapCut or so?

Why do we rely on HyperFrames to have a Studio — their version of CapCut — which knows exactly what to do with the HTML you wrote?

Why can't you just take any random HTML animation and drop it into any random video editor and have it work?

Because it won't. It can't. And that's the answer I was missing for the past week.

Because they did two things, not one

Here's what I missed at first, and I think most did too.

HyperFrames didn't just say to the AI "write a video in HTML, do whatever you want."

Because if they had, every AI would write the same scene differently. And that's the whole problem.

Think about it. You open Claude and you say: "Make a 5-second scene where the title fades in and slides up."

It gives you something like this:

<div class="scene">
  <h1 id="title">Hello World</h1>
</div>

<style>
  #title {
    opacity: 0;
    animation: fadeIn 0.8s ease forwards;
  }
  @keyframes fadeIn {
    to { opacity: 1; transform: translateY(0); }
  }
</style>

Clean. Works in a browser. Perfectly fine.

Now you open ChatGPT and ask the exact same thing. And it gives you this:

<div class="scene">
  <h1 id="title">Hello World</h1>
</div>

<script>
  const el = document.getElementById("title");
  el.style.opacity = 0;
  setTimeout(() => {
    el.style.transition = "opacity 0.8s";
    el.style.opacity = 1;
  }, 200);
</script>

Also clean. Also works in a browser. Also perfectly fine.

Now you open Gemini. Same prompt. And you get a third thing — maybe a GSAP timeline. Maybe a Web Animations API call. Maybe inline styles being toggled. Every AI, its own way. Every time, a different file.

All three look like videos. None of them are video projects.

Because now try to open any of those in a video editor. Try to drag that title clip on a timeline. Try to change its duration by dragging the edge. Try to split it. Try to know where it starts and where it ends without reading the code line by line.

You can't.

And that's the real problem. Not that AI writes bad HTML. AI writes great HTML. The problem is that AI writes a different shape every *single** time.* There's no agreement. There's no contract. Every video-shaped HTML file is its own little island, and no editor can dock at any of them. and that is what the real problem was.

And to address this, HyperFrames did the two things that actually matter.

First, they wrote SKILLS files.

Skills files are basically instruction manuals for the AI. They spell out exactly how to write a video in HTML if you want it to be a real, editable, renderable project.

Not "do whatever you want." Not "make it look nice." They said things like:

Use data-start and data-duration for every element's timing
Register your animation on window.__timelines[composition-id], not somewhere else
Always use the .clip class convention for timed elements

A bunch of rules. That's it. And if you follow them — all of them — your HTML stops being a webpage and becomes a proper video project.

That's the first half of the trick. The input side. It's how you make sure the AI's output is always in the right shape — the same shape, every time, no matter which AI you use.

Second, they built the Studio.

Now here's the part that makes the whole thing click.

Studio is HyperFrames' visual editor. Their version of CapCut. It looks like a normal editor — timeline, playhead, clip bars, drag, split, resize.

But the reason Studio works at all is because it knows the exact language the SKILLS files told the AI to write in.

When Studio opens your HTML, it doesn't guess. It looks for data-composition-id on the root. It looks for data-start and data-duration on every element. It looks for window.__timelines to find the animation. It knows the half-open visibility rule. It knows the track system.

And once it has all of that, it renders your HTML file as a full visual project.

Every scene, every clip, every track, every timing — all of it shows up as a real timeline you can edit.

And once you're at that point? It's just a normal video editor. Split. Resize. Drag. Rearrange. Overlay audio. Add transitions. Every basic editing feature works, because under the hood it's just editing a timeline model that was built from the HTML.

So the two halves fit together perfectly:

The SKILLS files teach the AI exactly what to write.
The Studio reads exactly what the SKILLS files teach the AI to write.

They designed both sides of the bridge to match. And that's what makes the whole thing work.

That's the insight. That's the thing.

Let me show you what I mean

Let's say you open any AI coding tool — Claude, GPT, Gemini, whatever — and you say:

"Put a 5-second animation on this clip. Text fades in, slides up, and fades out."

As I just showed you, you'll get three different answers. All of them work in a browser. None of them are a video project.

Now ask the same thing, but with the HyperFrames skills files in place.

It doesn't do whatever it wants. It doesn't guess. It always gives you the same shape, because the skills files told it exactly what shape to use:

<div id="root"
     data-composition-id="main"
     data-width="1920"
     data-height="1080"
     data-duration="5">

  <section class="clip" data-start="0" data-duration="5">
    <h1>Hello World</h1>
  </section>

</div>

<script>
  const tl = gsap.timeline({ paused: true });
  tl.from("h1", { y: 40, opacity: 0, duration: 0.8 }, 0.2);
  tl.to("h1", { opacity: 0, duration: 0.5 }, 3);
  window.__timelines["main"] = tl;
</script>

Same prompt. But every AI, every time, produces this structure. Because the skills files removed the guessing.

That's a real, complete, renderable video. Five seconds long.

And here's the beautiful part — that exact HTML can now be opened in HyperFrames Studio and edited visually.

You can scrub through it. You can drag the clip around on the timeline. You can split it. You can duplicate it. You can add another clip after it. You can change the duration by dragging the edge. All the things you'd do in CapCut, you can do with this HTML.

Not because the HTML is magic. Because the HTML follows the rules, and the Studio knows the rules.

That's what I mean when I say HyperFrames didn't build magical HTML. They built the rules, they taught the AI the rules, and they built an editor that speaks the rules.

The magic isn't in the code. It's in the agreement between the two sides.

And this is when it hit me

Building a visual video editor used to mean solving five hard problems.

What's the project file format? How do you represent clips and timing and layers? How do you render it? How do you build the editor UI — timeline, playhead, drag, split, resize? And how do you exchange projects with anything else?

The first three are the ones that kill you. They take months. They require decisions you can't undo.

HyperFrames solved all three by fiat and documented the answer.

And once you see that — once you realize the format is just HTML, following some simple rules, and the AI can already write it, and the editor already knows how to read it — the whole thing flips.

You can now build your own mini editor.

Not from scratch. Not in six months. Not even in six weeks.

Because the hard problems were already solved — by the Standard the SKILLs files introduced.

So now it's just a question of how you want to plug in.

You can take any editor that already handles the timeline, the playhead, the drag, the split — and write one parser on top of it that reads the HyperFrames standard. That's the whole job.
You can fork Studio and reskin it. They already built the parser. They already built the timeline model. They already built the scrub loop.

you can sit down and write your own parser from scratch, and then everything else — resize, drag, split, effects, audio overlay — is just normal editor work you'd be doing anyway.

Three routes. One outcome. And none of them require you to solve the same five problems every editor before you had to solve.

Because that was the headache I was trying to escape. The timeline. The playhead. The scrub loop. The clip model. All of it. HyperFrames already wrote it.

I just have to plug in.

The thing I keep coming back to

Standards sound boring. They sound like paperwork. But they're actually the most powerful thing in technology.

Think about how we used to have a different charger for every phone. Then USB-C came along. Now one cable charges your laptop, your phone, your headphones.

Or a more recent one — MCP.

Before MCP, every AI app wrote its own way to get the weather. Its own way to read the news. Its own way to send an email. If you wanted your tool to work with Claude, you built one thing. If you wanted it to work with GPT, you built another. If you wanted it to work with Gemini, another.

Then MCP came along. One protocol. One format. And now any tool that speaks MCP works with any agent that supports MCP — Claude, GPT, Gemini, whoever comes next.

Standards do that. They turn messes into ecosystems.

After a standard exists, everything connects. Anyone can build on top. Nobody has to build from underneath anymore.

HyperFrames is doing that for video. A small contract — A bunch of rules — and suddenly the whole game changes.

But the part I want you to remember is the pair. The SKILLs files and the Studio. The instruction and the editor that reads exactly those instructions. That's the two-sided trick. That's why it works when nobody else's does.

And the twist that makes it urgent rather than just neat — in a world where AI is going to write most of our video content, the format AI can write is the format that wins.

AI already writes HTML. Fluently. Every day.

So if the video format is HTML, and the AI has instructions to write it correctly, and an editor exists that reads exactly what the AI was told to write —

Then the AI's output is the video project. Ready to edit. Ready to render. Ready to ship.

That's not a feature. That's the future.

A note before I go

I'm writing this because I struggled.

I spent the past week staring at videos, reading documentation, watching people do it on YouTube — and every time, they'd say "it's just HTML, it's easy, you just ask the AI" and move on. And I kept thinking, okay, but how.

And I think maybe some of you have felt the same. You see people use something magical, everyone tells you it's simple, and somehow you still can't see how the trick works.

And honestly, maybe this blackbox was never a blackbox at all.

Maybe it was obvious from everything that had been handed to me all along.

Maybe I just didn't see it.

But I felt the pain. So I shared the cure.

If you've been through the same struggle with HyperFrames, or Remotion, or anything else — doesn't have to be a video tool — I'd love to hear about it. And if you find anything wrong in this article, please point it out. I'd be thankful for the generosity.

Thanks for being here.

Stay happy, stay blessed.

DEV Community: M Habib ur Rehman