DEV Community

AI-KSK
AI-KSK

Posted on

LTX 2.3 digital human in three passes — and the honest case for keeping LTX 2 around

#comfyui #ai #tutorial

Three-stage rendering has been the thread through several episodes; this one is where it pays off for talking heads. The author's claim is not peak quality but probability: the share of attempts that produce a usable lip-synced result goes up sharply, and the picture stays sharp through two latent upscales. He also does something rarer — he ships the LTX 2 version alongside, and explains when the older model is still the right answer.

Evidence note: both graphs ship as JSON and are parsed below. The face-proportion rule (about one twentieth of the frame), the 0.7/1/1 image-to-video weights and the two-person caveat are the author's reported observations in 字幕.srt. Note that this folder's YouTube and Bilibili titles describe different angles of the same episode; both are recorded.

Where this fits

You have one portrait and one audio clip and you need the face to speak. If the face occupies a reasonable share of the frame, LTX 2.3 with this graph does it. If the face is tiny in frame, the author says to use the LTX 2 graph instead — and he means it, because he ships both.

The two graphs

Graph Nodes Model line RunningHub
LTX 2.3 三阶渲染 1+3 数字人 100% 性能释放 64 LTX 2.3 22B https://www.runninghub.ai/post/2034922238251634690/?inviteCode=rh-v1111
LTX 2 三阶渲染 1+3 数字人 100% 性能释放 69 LTX 2 19B https://www.runninghub.ai/post/2034922123667447810/?inviteCode=rh-v1111

Both fetched once: HTTP 200, titles match.

JSON basis — LTX 2.3 (64 nodes)

  • Checkpoint ltx-2.3-22b-dev.safetensors
  • LoRA ltx-2.3-22b-distilled-lora-384.safetensors at 0.5 and ltx-2.3-22b-ic-lora-motion-track-control-ref0.5.safetensors at 1.0
  • First pass: euler_ancestral_cfg_pp, CFGGuider 1, sigmas 1.0, 0.99375, 0.9875, 0.98125, 0.975, 0.909375, 0.725, 0.421875, 0.0
  • Passes 2 and 3: euler_cfg_pp, CFGGuider 1, sigmas 0.85, 0.7250, 0.4219, 0.0 each
  • Latents 320x288 with 97 frames; audio latent 97 frames at 25 fps
  • Upscaler ltx-2.3-spatial-upscaler-x2-1.0.safetensors

JSON basis — LTX 2 (69 nodes)

  • Checkpoint ltx-2-19b-dev.safetensors
  • LoRA ltx-2-19b-lora-camera-control-static.safetensors at 0.5 and ltx-2-19b-distilled-lora-384.safetensors at 0.4
  • Same three-pass sampler pattern and the same 320x288x97 / 97-frame latents
  • Upscaler ltx-2-spatial-upscaler-x2-1.0.safetensors

That camera-control-static LoRA is the interesting carry-over: in the LTX 2 line its job was locking the camera down, and the author found it also raised the success rate. In 2.3 he swaps in the motion-track-control IC-LoRA and gets the same effect.

Step by step

  1. Load one image and one audio clip.
  2. Set the audio length. It is measured in seconds — the demo takes 18 seconds from the start, and you can start from, say, 5 seconds instead. That length defines the video.
  3. Write a short prompt describing the shot. The author includes an explicit "keep the camera static" instruction, because for a talking head the lip-sync is the whole point and a subject that walks off frame cannot be synced.
  4. Run. The graph does one low-resolution latent pass, two latent upscales, and keeps an image-to-video node in all three passes with weights 0.7 / 1 / 1. The author's intent for the first pass is to trade some similarity for motion — at the very low initial size there is not much similarity to protect anyway.
  5. If you raise the size, use the megapixel control: he moves the first-pass render from 0.1 to 0.15.

The two technical details people get wrong

  1. Set Latent Noise Mask width/height must be raised in passes 2 and 3. After a latent upscale, if you leave the mask at the old size, the audio condition is misaligned and the lip-sync degrades. The audio reaches the pipeline through the encoded audio input, so it must be preserved at the right resolution.
  2. Two-person digital human: do not load the audio-driven route. With two speakers the model cannot tell which line belongs to which person. The author's advice is to write the dialogue into an image-to-video prompt instead — his two-person demo, generated that way, keeps the speakers distinguishable.

Face proportion: the rule that decides everything

He states it plainly: when the character's face occupies no less than about 1/20 of the frame, lip-sync essentially always works. Go smaller and LTX 2.3 produces a static video — which is why the LTX 2 graph exists, and why he shows results where a large scene with a small figure still animates on 2.3's older sibling. For a small figure in a wide shot, use LTX 2.

Limits

  • The success-rate improvement is the author's observed probability, not a measured benchmark.
  • Camera-static guidance is load-bearing: moving subjects break lip-sync.
  • Two-speaker audio driving is explicitly discouraged.
  • The LTX 2 graph is not deprecated here — it is presented as the better tool for small faces.
  • Latent upscale requires matching the noise mask size in later passes, or quality silently drops.
  • Both graphs still depend on a single still image; there is no multi-view identity lock.

Episode resources

Top comments (0)