Image to Talking Video: The Uncanny Valley Isn't About Lip Sync

My first image to talking video attempt had perfect lip sync and still looked wrong. I spent an afternoon assuming the mouth timing was off before I actually watched it frame by frame. The mouth was fine. Everything around it was dead still.

This post covers what actually causes the uncanny valley in talking-head clips, two scripts that fix the boring inputs before generation, and where I generate the clip itself.

The uncanny valley isn't lip sync, it's stillness

Lip sync is the easy part to notice and the easy part most tools get right by now. What actually breaks the illusion is everything a face does between words: a blink, a small head tilt, a shift in the eyes, breath moving the shoulders. Freeze all of that and sync only the mouth, and you get a face that talks but doesn't live.

That reframes the problem. You're not troubleshooting sync anymore, you're troubleshooting motion, and the motion mostly comes from what you feed the model:

• A stiff source photo. A rigid, straight-on portrait gives the model nothing to build natural movement from.
• Flat audio. Monotone, unedited audio produces monotone motion, because pacing and emphasis in the voice are what drive expression.
• Clips too long for one take. Long single takes drift or loop oddly. Short, clean segments hold up better.

Two of those three are things you can fix before you ever generate anything.

This sits next to the plainer image to video AI case rather than replacing it. A product demo that doesn't need a face is still the faster path to animate a photo of an object. Talking video is specifically for when a presenter, not a product, needs to carry the message, and an AI product video generator reaching for a spokesperson clip is exactly that case.

Clean the audio before it drives a face

Silence at the start, a level that's too quiet, or a stray click all show up as dead time or a flinch in the generated motion. I run every voice track through this first:

#!/usr/bin/env bash
# audio_prep.sh - trim leading/trailing silence and normalize loudness
# usage: ./audio_prep.sh input.wav output.wav
IN="$1"; OUT="$2"
ffmpeg -y -i "$IN" -af "silenceremove=start_periods=1:start_threshold=-40dB:start_silence=0.1:detection=peak,areverse,silenceremove=start_periods=1:start_threshold=-40dB:start_silence=0.1:detection=peak,areverse,loudnorm=I=-16:TP=-1.5:LRA=11" "$OUT"

It trims silence from both ends by reversing the file, stripping silence again, and reversing back, then normalizes to a broadcast-standard loudness. A voice track without dead air at the edges and with consistent level gives the model a much cleaner signal to animate from.

Split long scripts before they become one long, drifting take

If your script runs past what one generation comfortably handles, don't feed it in as one block. Split on sentence boundaries and estimate timing from a speaking rate:

# script_split.py - split a voiceover script into clip-sized chunks
# usage: python script_split.py script.txt --wpm 150 --max-seconds 20
import argparse
import re

def sentences(text):
    return [s.strip() for s in re.split(r'(?<=[.!?])\s+', text.strip()) if s.strip()]

def chunk(text, wpm, max_seconds):
    chunks, current, current_words = [], [], 0
    limit_words = wpm * max_seconds / 60
    for s in sentences(text):
        w = len(s.split())
        if current and current_words + w > limit_words:
            chunks.append(" ".join(current))
            current, current_words = [], 0
        current.append(s)
        current_words += w
    if current:
        chunks.append(" ".join(current))
    return chunks

if __name__ == "__main__":
    p = argparse.ArgumentParser()
    p.add_argument("path")
    p.add_argument("--wpm", type=int, default=150)
    p.add_argument("--max-seconds", type=int, default=20)
    args = p.parse_args()
    text = open(args.path).read()
    for i, c in enumerate(chunk(text, args.wpm, args.max_seconds), 1):
        secs = len(c.split()) / args.wpm * 60
        print(f"--- clip {i:02d} (~{secs:.1f}s) ---")
        print(c)

--wpm should match how your voiceover is actually paced, not a generic average; a slow, deliberate read needs a lower number than a fast one. Each chunk becomes its own generation instead of one long take that starts to wander.

Where I generate: image to talking video in one workspace

For the generating itself, I used VOKOO, a multi-model AI creation platform built around video. Its tagline is "Create more. Switch less." The relevant feature here starts from one photo plus one voice track, which is exactly the input shape the two scripts above are meant to clean up. I dropped in a portrait and a prepped audio clip and had a result to review before I finished my coffee.

One photo, one voice track, one clip

The AI avatar feature takes one photo plus one voice track and animates lip movement and expression from it. One photo plus one voice track, that's the starting point. Feed it the trimmed, normalized audio from audio_prep.sh rather than a raw recording.

Start from a photo with something to work with

A photo with a slight head turn or natural expression gives the model more to animate than a stiff, straight-on shot. The AI photo editor and image upscaler let me edit, refine, and make small or blurry images crisp and usable without leaving the flow before I use it as the source.

Switch models if one reads flat

The AI agent lets me try different models without rebuilding my workflow. Some models add more natural micro-movement than others by default; if a result still reads stiff after clean audio and a good photo, the model itself is the next thing to change, not the inputs.

Check the cost per chunk

I can pick quality and generation specs per stage and see the estimated credit cost before I submit. With a script split into several chunks, that cost preview matters more than usual, since you're paying per clip, not per script.

Keeping a multi-clip script affordable

Draft the first chunk at a lower spec to confirm the voice and photo pairing works before committing to the rest. Once the pairing looks right, the remaining chunks are a much safer bet at full quality.

If you'd like an LLM to tighten a script down to punchier sentences before you split and generate it, RouteAI provides a cost-effective, OpenAI-compatible API gateway with multiple models, so setup stays simple.

Try this next

Image to talking video stops feeling uncanny once you stop treating lip sync as the whole job. Clean the audio, give the model a photo with something to move from, and keep clips short enough to hold together. VOKOO handled the animating; the two scripts handled the inputs. Stop managing tools. Start making things.

Here's a short test you can run today:

1. Run one real voice recording through audio_prep.sh.
2. Split a short script with script_split.py --max-seconds 15.
3. Generate the first chunk with a photo that has a slight, natural expression.
4. Check the estimated cost before generating the rest of the chunks.

If you want an easy AI video generator that keeps simple AI video creation simple and still leaves room to explore, try VOKOO at https://vokoo.ai.