How to Create AI Video From Audio With an Explicit Input Contract

Treat audio-driven video like an API with required inputs: one still image, one clean
audio file, and a motion prompt that describes only what the sound should drive. If
you leave any contract ambiguous, the model invents motion to fill the gap.

This cover is an input-flow illustration, not generated footage.

Input contract

type AudioVideoJob = {
  image: "licensed portrait or character still";
  audio: "trimmed voice or vocal clip";
  motion: {
    mouth: "follow spoken phonemes";
    head: "small nod on emphasized phrases";
    eyes: "natural blinks, gaze near camera";
    camera: "locked";
  };
  exclusions: ["camera cuts", "new objects", "large hand gestures"];
};

Prepare the audio first. Trim dead air, remove obvious clipping, and use a single
speaker for the first test. Choose an image with an unobstructed mouth, visible jaw,
and enough space around the head.

The audio-to-video generator
currently exposes Podcast VO, Singing/vocal, and Explainer presets. The July 25 form
required an image and audio file, accepted a prompt, used Public visibility, and
showed 240 credits at its checked default.

Minimal motion prompt

Single-speaker podcast close-up. Preserve the uploaded face, hair, clothing, and
background. Match lip movement to the uploaded speech. Add one subtle nod on strongly
stressed words and natural blinks. Keep shoulders relaxed and gaze near the lens.
Locked camera, no cuts, no hand gestures entering frame, no background motion.

Validate like a media pipeline

Check the result at normal speed and then at 0.5×:

• Consonant closures roughly align with p, b, and m sounds.
• The mouth returns to rest during pauses.
• Head motion follows emphasis rather than oscillating continuously.
• Teeth, jawline, earrings, and glasses remain stable.
• The frame does not crop differently from the source without instruction.

For lip-sync drift, test a shorter clean excerpt before changing the portrait. For
facial distortion, use a more frontal image and remove requests for expressive
gestures. For robotic motion, name two low-amplitude behaviours rather than asking
for “more natural movement.” If upload fails, confirm the file types and retry with a
shorter, locally exported audio clip.

Failure boundaries

This workflow does not create consent. Use your own voice and portrait or obtain
clear permission. Do not impersonate another person, misrepresent recorded speech,
or present a synthetic presenter as a real endorsement. Final captions, loudness,
and accessibility review still belong in an editor. The current estimate and model
selection can change, so recheck them before generating; Public is not appropriate
for confidential recordings.

Once the three inputs satisfy the contract, run a short synchronization test.