Load a clip, give it new lines, and LTX 2.3 regenerates the speech and the mouth motion in one pass while trying to keep the person and the timbre. The official IC-LoRA for lip dubbing removes the TTS-then-clone-then-stitch chain that used to sit in front of this job.
Evidence note: this episode ships no workflow JSON, so no node counts or sampler values are claimed. Everything below comes from the narration and the published mirror page.
When this is useful
- Retiming or re-scripting a talking-head clip without re-recording the presenter.
- Producing the same on-camera segment in several languages from one take.
- Second-use content: change the line inside an existing clip instead of shooting a new one.
How this differs from the previous build
The input shape is what changed. Older dubbing pipelines took a video, synthesised speech with TTS, cloned the speaker, then drove a lip-sync model — three tools and three failure points. This one takes a video plus text and produces video with new audio already aligned.
- One fewer input: no separate audio track to supply.
- The text encoder is Gemma 3 in FP4 form, chosen by the author for 5090 behaviour and lower VRAM; prompts in this task are short, so the smaller encoder is enough.
- A new node carries the audio reference tokens (
LTXVSetAudioRefTokens), which is what the IC-LoRA needs to keep timbre.
Running it
- Update the LTXVideo nodes to the current release before loading the official example graph.
- Deploy the mirror and open ComfyUI; the graph is in the left workflow list.
- Load the source clip, then supply the reference video and reference audio the graph asks for.
- Set frame rate and frame count, then write the prompt in the fixed order: who the person is and what they look like, what they are doing, the scene, the language they speak, then the line itself inside double quotes.
- Run the two-stage pass: about 600k pixels first, then about 1.2M pixels.
- Keep the second-stage pixel budget at roughly twice the first — the spatial upscaler is a 2x model.
Honest limits
- The author reports visible artefacts in some results, and expects an upscale pass to handle them; that is a workaround, not a fix.
- Frame size advice is the author's, not an official spec: 600k pixels then 1.2M pixels, kept at a 2x ratio because the upscaler is 2x.
- No workflow JSON ships with this episode, so nothing here is a parameter claim.
- Peak VRAM of about 27GB and the 1080p claim are single-machine observations from the author.
Episode resources
Mirror (Compshare, referral code preserved)
- Official LipDub: change a talking video's lines — https://www.compshare.cn/images/NiTes5O1EVZr?referral_code=A7fTQUYY9lCDFJwA6MMvoq
- Video tutorial (YouTube): https://www.youtube.com/watch?v=TCwPQbEyxHM
- Chinese original (Bilibili): https://www.bilibili.com/video/BV1455g6yEqG
Optional API platform - 88API (affiliate link)
Explore 88API for access to multiple AI model APIs. Available models, routes and prices are listed on the platform; Alipay top-ups are supported.
https://88api.ai/sign-up?aff=usx3
Disclosure: I may earn a commission from eligible purchases after you register through this referral link.
Top comments (0)