One Prompt, Four Modalities: What a Unified Generation Agent Actually Has to Solve

FunGen — Sun, 18 Oct 2026 16:13:05 +0000

Most generation tools are one modality wide. You have a favourite for images, a
different one for video, something else for voice, and if you touch 3D at all it is
a fourth tab with its own account and its own export quirks.

The obvious fix is to put them behind one text box. That sounds like a UI decision.
It is not. Once a single prompt field has to serve image, video, voice and 3D, three
genuinely hard problems show up, and none of them are about layout.

Problem 1: the modalities disagree about what a prompt is

A good image prompt is a dense noun phrase. Adjectives, materials, lighting, lens.
Nothing about time, because there is no time.

a worn brass desk lamp, single hard key light from the left, dusty highlights

A good video prompt is a verb phrase with a camera in it. The same noun phrase
produces a static shot that technically moves.

a worn brass desk lamp; camera pushes in slowly as the filament flickers on

A good 3D prompt is neither. It is a set of geometric constraints, and half the
adjectives that improve an image actively hurt it — "dusty highlights" is a
texture instruction with no geometry behind it, and "hard key light" is
meaningless in a mesh.

a desk lamp, hollow base, flat bottom, thick shade walls, no thin wires

A voice prompt is a performance direction: pace, register, emphasis.

So a single prompt box is lying a little. The same sentence cannot be optimal in
four places at once. The design question is what to do about that, and there are
three honest answers:

Make the user rewrite it per modality. Correct, and nobody does it.
Rewrite it silently. Fast, and it hides why a result was bad.
Rewrite it and show your work. Slower, and the only option where the user learns anything.

The third is what "agent" should mean in practice: the thing plans, tells you what
it is about to do, and lets you disagree before spending the compute.

Problem 2: each modality fails in an unrelated way

The failure modes have almost nothing in common, which means one generic error
state is useless:

Modality	Typical failure	What the user needs to see
Image	Composition drifts from the brief	A/B against the prompt
Video	Frame-to-frame identity drift	Which frame it broke on
Voice	Right words, wrong performance	The specific phrase to re-read
3D	Geometry is plausible, topology is not	Mesh stats, not a render

That last row is the one people underestimate. A 3D render can look flawless while
the underlying mesh is unusable — hundreds of thousands of triangles, disconnected
shells, no consistent scale. The render is not evidence about the geometry. Any
tool that shows you only the pretty picture is answering a different question from
the one you asked.

Problem 3: chaining is where the value is, and where the state lives

The single-modality workflow is: prompt, generate, download, done.

The interesting workflow is: generate an image, keep that exact subject, put it in a
scene, animate it, add a voice over it, and separately turn the subject into a mesh.

Each arrow in that chain is a place where identity has to survive. That is a state
problem. The system has to carry a persistent handle on "this specific thing" across
model boundaries that were never designed to agree with each other. Tool-hopping
loses that handle at every step, which is why the manual version of this workflow
produces four assets that do not quite look like the same object.

Keeping the chain inside one project is not a convenience feature. It is the only
place the continuity can live.

What this implies for exports

If the chain is the point, the exports have to be ordinary files — MP4, PNG, WAV,
OBJ — and they have to be downloadable without ceremony.

A generation tool that will not hand over the file is not a tool, it is a demo. This
matters most for 3D, where the file is the entire deliverable: nobody wants a mesh
they can only look at inside someone else's viewer.

The short version

Putting four modalities behind one prompt is not a consolidation exercise. It
requires the system to translate intent per modality, surface modality-specific
failure honestly, and keep subject identity alive across model boundaries. Get those
three right and the single text box is genuinely simpler. Get them wrong and you
have built four tools that happen to share a font.