DEV Community

AI-KSK
AI-KSK

Posted on Edited on

LTX 2.3 LipDub: Rewrite What a Person Says Without TTS or Voice Cloning

Load a clip, give it new lines, and LTX 2.3 regenerates the speech and the mouth motion in one pass while trying to keep the person and the timbre. The official IC-LoRA for lip dubbing removes the TTS-then-clone-then-stitch chain that used to sit in front of this job.

Evidence note: this episode ships no workflow JSON, so no node counts or sampler values are claimed. Everything below comes from the narration and the published mirror page.

When this is useful

  • Retiming or re-scripting a talking-head clip without re-recording the presenter.
  • Producing the same on-camera segment in several languages from one take.
  • Second-use content: change the line inside an existing clip instead of shooting a new one.

How this differs from the previous build

The input shape is what changed. Older dubbing pipelines took a video, synthesised speech with TTS, cloned the speaker, then drove a lip-sync model — three tools and three failure points. This one takes a video plus text and produces video with new audio already aligned.

  • One fewer input: no separate audio track to supply.
  • The text encoder is Gemma 3 in FP4 form, chosen by the author for 5090 behaviour and lower VRAM; prompts in this task are short, so the smaller encoder is enough.
  • A new node carries the audio reference tokens (LTXVSetAudioRefTokens), which is what the IC-LoRA needs to keep timbre.

Running it

  1. Update the LTXVideo nodes to the current release before loading the official example graph.
  2. Deploy the mirror and open ComfyUI; the graph is in the left workflow list.
  3. Load the source clip, then supply the reference video and reference audio the graph asks for.
  4. Set frame rate and frame count, then write the prompt in the fixed order: who the person is and what they look like, what they are doing, the scene, the language they speak, then the line itself inside double quotes.
  5. Run the two-stage pass: about 600k pixels first, then about 1.2M pixels.
  6. Keep the second-stage pixel budget at roughly twice the first — the spatial upscaler is a 2x model.

Honest limits

  • The author reports visible artefacts in some results, and expects an upscale pass to handle them; that is a workaround, not a fix.
  • Frame size advice is the author's, not an official spec: 600k pixels then 1.2M pixels, kept at a 2x ratio because the upscaler is 2x.
  • No workflow JSON ships with this episode, so nothing here is a parameter claim.
  • Peak VRAM of about 27GB and the 1080p claim are single-machine observations from the author.

Episode resources

Mirror (Compshare, referral code preserved)

Optional API platform - 88API (affiliate link)

Explore 88API for access to multiple AI model APIs. Available models, routes and prices are listed on the platform; Alipay top-ups are supported.

https://88api.ai/sign-up?aff=usx3

Disclosure: I may earn a commission from eligible purchases after you register through this referral link.

Top comments (0)