Talking heads are where most AI video still looks like a puppet. The mouth lags, the voice belongs to someone else, and the body stays still while the jaw chatters. Vidu Q4 treats sound as part of the shot, not as a track glued on afterwards.
The usual pipeline exports a silent clip, runs a separate voice model, then stretches the chin with a lip-sync plugin. That is slow, and the patch sits on a finished face, so the skin smears. Vidu Q4 maps the audio waveform and the facial motion in the same generation pass. A short voice sample is enough for the line to come out in that voice, with the lips, teeth, and breath landing on the same beats.
Why a plugin still looks like a puppet
An external lip-sync tool only repaints the mouth on frames that already exist. It does not tilt the head, blink, or move the chest with the breath. Vidu Q4 uses the audio as a guide for the whole diffusion, so a hard consonant can come with a small lean and a narrower eye. The motion follows how a person actually speaks, instead of a mask sliding on the jaw.
What goes into one voiced shot
The job is three inputs: a character still, up to three voice samples, and the line with its tone marked. One branch keeps the timbre and the pace. The other shapes the teeth, the lips, the throat, and the small motion around the eyes. They meet in one picture with sound, rather than a silent video plus a dub.
Five steps
- Record 8 to 10 seconds of one dry voice. WAV or MP3 is fine if the room is quiet and the mic is not popping.
- Break the line on the breaths, and mark the delivery. Example: a tight, shaking tone for "You thought you could hide this from everyone?"
- Load the still and the sample in the same slot so the face and the voice stay bound. The voice panel on viduai.co is set up for that drop.
- Turn on high-precision mouth tracking and let the audio lead the face a little more than the default if the consonants are soft.
- Listen on the timeline. Export only after the words and the mouth land together, with a 48 kHz track in the file.
Against a two-step dub
| Video, then TTS, then a plugin | Vidu Q4 native voice | |
|---|---|---|
| Mouth and teeth | Flat lips, a smeared chin | Teeth and lips move as one shape |
| The rest of the body | The torso freezes | Breath and small head motion stay in time |
| Whose voice | A generic reader | Up to three samples of the character |
| Turnaround | Export, dub, patch, re-export | One voiced render |
Practical notes
A noisy sample will leak room tone into the line. Record dry. If two characters speak, give each one a separate sample and do not reuse the same clip for both. Mark the emotion in the script. The model follows the written stress more reliably than a plain paragraph of dialogue.
This is for short-drama teams, commercial spots with a spokesperson, and anyone who has been fixing mouths in post. Start with one clean 10-second sample and one still, and check a single line before you build a longer scene.
Top comments (0)