A generated video can look beautiful and still tell the wrong story. A character enters from the wrong direction. A close-up replaces the shot that should establish the room. A prop removed in one scene reappears in the next.
In Timeline Studio, our open-source, local-first video editor, we built a usable browser-based 3D blockout workspace to make those directing decisions visible before final generation.
Source code on GitHub · Try the Anna edition
This article covers the architecture and lessons from a separate AI short-film experiment that helped motivate it. The experiment's blockout footage was made independently; the editor did not generate that finished film in one click.
What a blockout makes explicit
A blockout is a simplified rehearsal: basic characters, primitive props, camera positions, and timed movement. Materials and detailed appearance can wait while we check where people stand, what the camera sees, and whether an action has enough time to finish.
“Walk to the desk, put down the milk, and return a hair clip” hides several decisions: which side of the desk, when the hand releases the carton, where the handoff happens, and when to cut to the other character's reaction.
The goal is to give these decisions an editable representation.
The workspace is directly editable. The voice-binding area shown in this screenshot is future work.
Separate scene data from rendering
The workspace uses Three.js and loads separately from the main editor. Its scene is a versioned, serializable document with objects, timed motions, camera shots, script segments, and a total duration.
A simplified view of the data model is:
{
revision,
duration,
objects,
motions,
shots,
script
}
Three layers share responsibility:
| Layer | Responsibility |
|---|---|
| Scene commands | Validate operations and produce the next scene snapshot |
| Three.js viewport | Selection, transforms, playback, and camera-view output |
| Workspace and chat | Editing state, timeline, script, and proposal review |
The scene command layer is the common boundary for manual controls and AI proposals. The renderer displays the scene; it does not define a second set of editing rules.
One clock for actors, cameras, and script
At playback time t, the workspace computes actor poses, selects the current camera shot, and finds the corresponding script segment.
For a linear move between P0 and P1 over [t0, t1]:
u = clamp((t - t0) / (t1 - t0), 0, 1)
P(t) = P0 + (P1 - P0) * u
Preview and video export use the same actor-pose and camera-selection helpers. That reduces the risk of a preview showing one arrangement while an export uses another.
Clicking an action, shot, or script segment seeks to its start. Script text and time ranges remain editable. Changing the total scene duration scales timed content together. Script timing is an editable estimate, not automatic alignment to recorded speech.
AI proposes edits through a constrained command layer
In the Anna integration, the model receives the current scene metadata and a scene-edit catalog. It submits structured proposals rather than arbitrary executable code.
The flow is:
Inspect scene snapshot
→ Propose structured operations
→ Validate commands
→ Review the proposed edit
→ Check that the scene still matches
→ Apply as one undoable edit
Validation checks object identities, values, and supported timing constraints. Unknown objects, unsupported operations, arbitrary code, and external assets are rejected. If the scene changes while a proposal is pending, that old proposal cannot overwrite the newer edits.
This keeps natural-language assistance connected to an inspectable, editable project. Manual editing works locally; remote planning needs a working Anna host session.
Export a rehearsal you can actually watch
The workspace exports scene JSON and a real 1280 × 720 WebM camera-view video, with progress and cancellation. Video capture uses the browser canvas stream and MediaRecorder; available codecs depend on the browser.
Scene changes persist per project in the same browser. This does not imply cross-device synchronization. The blockout workspace is also separate from the main video timeline.
What we learned from a short-film experiment
In a campus short-film experiment, we compared independent blockout footage against AI-generated shots. The comparison images accompanying this article show corresponding narrative moments sampled proportionally from clips of different lengths. They are not exact timestamp matches or proof of frame-by-frame adherence. All realistic character images are AI-generated.
G02 r5: blockout above, AI-generated image below. Proportional narrative-moment sampling, not frame-exact alignment.
1. Similar composition does not mean the whole shot passed
Comparisons made it easier to inspect character positions, desk layout, occlusion, and camera direction. But a matching still did not establish that a whole sequence followed the rehearsal.
One generated segment preserved the broad order—enter, approach, put down milk, return the hair clip—while changing medium shots into close-ups or over-the-shoulder views and delaying a reaction shot. We did not mark it as strict adherence.
2. Sometimes the rehearsal contains the mistake
A character needed to return to his seat before a segment ended. Initially, we blamed generation for not completing the action. Reviewing the blockout showed that the original staging itself had left too little time.
We moved the return approximately 0.7 seconds earlier and adjusted the ending shot. The lesson was practical: inspect the reference before adding stronger wording to the prompt.
3. Prop interactions need motion review
Putting down milk requires a stable contact with the table before release. Returning a hair clip requires an actual handoff and the correct resulting prop state.
A still helps locate a problem. It cannot establish whether the fingers, object, and movement remain coherent throughout the action. Full playback is still necessary.
4. Continuity references can propagate errors
Using the previous clip's real final frame helped carry appearance and pose forward. It also carried an incorrect hair clip into the next segment. A conflicting environment reference contributed to a window being replaced by a blackboard.
We preserved the original final frame, made a corrected reference copy, and removed the conflicting environment image.
| Reference | Main information |
|---|---|
| Blockout video | Space, movement, cameras, and pacing |
| Character image | Appearance and clothing |
| Previous final frame | State at the end of the preceding segment |
| Environment image | Setting and layout |
More references help only when their information is compatible.
What this contributes—and what remains open
The clearest benefits we observed were earlier staging checks, a concrete comparison baseline after generation, and more precise discussions about revisions.
We have not measured a reliable percentage reduction in retries or cost. A blockout also cannot guarantee that a generation model will execute its camera plan.
The browser workspace already supports direct editing, timed actions and shots, script timing, reviewed AI edits, and JSON/WebM export. Voice binding, complex facial animation, collision and IK, external mesh imports, and full integration with the video timeline remain unfinished. There is no automatic end-to-end blockout-to-final-video pipeline.
The direction is to keep directing decisions visible and revisable before committing to expensive final imagery.
If you work on browser 3D, AI filmmaking, or agent-assisted editors, I'd welcome a concrete shot-control problem or an implementation discussion.
Feature availability can differ between the public editor and the Anna edition.
Top comments (0)