DEV Community

AI-KSK
AI-KSK

Posted on

LTX 2.3 control was unusable — then a second upscale pass fixed it

#comfyui #ai #tutorial

LTX 2.3's IC-LoRA control flow has a specific failure mode: you feed it a skeleton or depth video, it follows the motion, and the result is visibly degraded — soft, wrong at the edges, occasionally gridded. The episode's answer is unglamorous and effective. Render first, then run the output through a video-to-video spatial upscale pass. That single change is what moves IC-LoRA control from "demo" to "usable".

Evidence note: node counts, model and LoRA filenames, sigma lists, CFG values and latent sizes below are parsed from the two workflow JSON files in 工作流及封面(视频216). The verdicts — Dev-versus-distilled, why the third pass hurts, why 56/24 frame chunking keeps VRAM flat — are the author's reported observations in 字幕.srt.

Where this fits

You need motion control (pose, depth, canny) rather than prompt-only generation, and you are willing to spend a second render pass to get the picture back. The author's framing: this pipeline is worth using as the post-process stage of some other tool.

The two graphs

Graph Nodes Purpose RunningHub
LTX 2.3 视频控制高清增强 80 Control-driven generation (IC-LoRA) + high-definition enhancement https://www.runninghub.ai/post/2040812593555709953/?inviteCode=rh-v1111
LTX 2.3 循环采样放大优化 45 Any video in, upscaled video out — the "loop sampling" upscale https://www.runninghub.ai/post/2041055276513628161/?inviteCode=rh-v1111

Both fetched once: HTTP 200, titles match.

Graph 1 — control + enhancement

JSON basis (80 nodes; groups: Set prompts → use API nodes to save on memory / Preprocess / Load Models / Load Image (set bypass=True if t2v) / Generate / Decode / Prepare Reference Video / Load Reference Video / Depth Annotation / Canny Annotation / Pose Annotation):

  • UNET ltx-2.3-22b-dev_transformer_only_fp8_scaled.safetensors
  • Dual CLIP gemma_3_12B_it_fp8_e4m3fn.safetensors + ltx-2.3_text_projection_bf16.safetensors (type ltxv)
  • VAE LTX23_video_vae_bf16.safetensors; checkpoint ltx-2.3-22b-distilled.safetensors
  • LoRA ltx-2.3-22b-distilled-lora-384.safetensors at 0.7 and LTX2.3_Reasoning_V1.safetensors at 1.0
  • Latents: 960x544 with 121 frames; audio latent 97 frames at 25 fps
  • Samplers euler_cfg_pp with CFGGuider at 1.3; plus a res_2s_ode branch with a 9-value ManualSigmas chain starting at 1.0
  • Second-pass sigma chains: 0.85, 0.7250, 0.4219, 0.0 and a 10-value chain 0.85 … 0.15, 0.0
  • Spatial upscaler ltx-2.3-spatial-upscaler-x2-1.1.safetensors

Quick start: load a video, load a reference image (or bypass it — then generation depends only on the prompt and the control video), write the positive prompt, keep the negative prompt short and specific, set frames to the loaded cap (121 here), and do not fight the width/height. The graph's guidance is that an over-detailed positive prompt pulls the result away from the reference image.

Control preprocessor choice: the official example offers three — depth, Canny, Pose. The author uses SD Pose on RunningHub because it detects more accurately, at a higher VRAM cost. For a result with a body turn, he expects the SD Pose preprocessed video to be clearly better.

The crop that matters: before decoding, the graph crops the video length. In latent space the frame count sits around 200-plus; after the crop it matches the 121 frames you asked for. Miss this and your frame count silently disagrees with the target.

Why one render pass is bypassed: the three-pass group is deliberately cut down. The author found that a third render after the second degrades the result, so after the IC-LoRA control pass he runs one enhancement pass and stops. On a basketball clip he still sees grid-like artefacts when he zooms in, and he expects feeding a reference image would make it worse.

Graph 2 — the loop-sampling upscale

This is the graph that rescues blurry output, including output from graph 1. It is a video-to-video spatial upscale that works on any video, and — the author's point — the less sharp the source, the better the improvement looks. It is closer to a "deep" upscale than to a plain resize.

JSON basis (45 nodes; groups Step 1 - Load Video / Step 2 - Load Model / Step 3 - Sampler, duplicated for two sampling branches):

  • Checkpoint ltx-2.3-22b-distilled.safetensors
  • LoRA ltx-2-19b-distilled-lora-384.safetensors at 1.0 and ltx-2-19b-ic-lora-detailer.safetensors at 1.0 — note these are the LTX 2 19B LoRAs, used as the detailer inside an LTX 2.3 graph
  • KSamplerSelect = euler; CFGGuider = 1
  • Sigma chains 0.909375, 0.725, 0.421875, 0.0 and 0.85, 0.7250, 0.4219, 0.0

What was changed versus the source graph: a custom sigma chain that makes the redraw more aggressive (so the effect is visible), a NAG node to force the negative prompt to have effect, a VRAM reserve, and automatic prompt reverse-engineering.

Performance note: parameters generally do not need touching. Each processing chunk is 56 frames with 24 frames of overlap, which is why raising the longest-edge target does not blow up VRAM. That is also why it is safe to run on ordinary machines.

Limits

  • Control output still shows grid artefacts at high zoom; the upscale hides a lot, not everything.
  • The third pass is removed on purpose — more passes is not better here.
  • The author's recommendation is to move to the distilled 22B model instead of Dev: Dev is too slow and does not improve results enough to justify it.
  • SD Pose costs more VRAM than depth or Canny.
  • This pipeline is a post-process, not a replacement for a proper control model.

Episode resources

Top comments (0)