DEV Community

AI-KSK
AI-KSK

Posted on

Colorizing black-and-white video with LTX 2.3 IC-LoRA: one reference frame, no preprocessor

#comfyui #ai #tutorial

Black-and-white to colour is a different job from video generation: the motion, framing and timing already exist, and the model's only task is to rebuild skin, clothing, environment, light and mood along the original footage. This episode does it with an LTX 2.3 IC-LoRA colourisation LoRA, and the result is stable enough that the author recommends trying it.

Evidence note: both graphs ship as JSON and are parsed below — node counts, model filenames, the Qwen 2511 first-frame branch, sampler and sigma values. The claims about frame counts (400 to an optimised 800), why the CN preprocessor is no longer needed, and the "reference latent prevents pixel shift" point are the author's reported findings in 字幕.srt.

Where this fits

You have archival or newly shot monochrome footage and you want colour that follows the existing camera movement rather than a hallucinated new scene. The author deliberately used royalty-free clips for testing.

One structural note worth understanding before the steps: LTX 2.3's ControlNet is loaded as a LoRA — an IC-LoRA, a context LoRA — so "colourisation ControlNet" and "IC-LoRA" are the same mechanism here.

The two graphs

Graph Nodes Difference RunningHub
LTX 2.3 IC LoRA 视频上色（手动提示） 44 Reference branch bypassed; you write a detailed positive prompt https://www.runninghub.ai/post/2035643489333022722/?inviteCode=rh-v1111
LTX 2.3 IC LoRA 视频上色 81 Adds a Qwen 2511 first-frame colourisation branch as global reference https://www.runninghub.ai/post/2035010437032448002/?inviteCode=rh-v1111

Both fetched: HTTP 200, titles match.

Shared graph basis (from the JSON)

Both graphs share the LTX side:

  • Checkpoint ltx-2.3-22b-dev.safetensors
  • KSamplerSelect = euler_ancestral_cfg_pp; CFGGuider = 1
  • Sigma chain 1.0, 0.99375, 0.9875, 0.98125, 0.975, 0.909375, 0.725, 0.421875, 0.0
  • Latents 960x544 with 121 frames; audio latent 97 frames at 25 fps
  • Group layout: Set prompts → use API nodes to save on memory / Preprocess / Load Models / Load Image (set bypass=True if t2v) / Generate / Decode / Prepare Reference Video / Load Reference Video / Depth-Canny-Pose annotation

The 81-node version additionally carries the Qwen 2511 branch:

  • UNET qwen_image_edit_2511_bf16.safetensors; CLIP qwen_2.5_vl_7b_fp8_scaled.safetensors (type qwen_image); VAE qwen_image_vae.safetensors
  • A KSampler at 4 steps, CFG 1, euler, simple, denoise 1 — the standard 2511 four-step edit setup

Step by step

  1. Prepare a monochrome clip. The author uses royalty-free footage.
  2. Set the frame count you want to process. He states that for LTX 2.3 400 frames, and up to roughly 800 after optimisation, is fine.
  3. Choose your route:
    • Automatic (81 nodes): the graph extracts the first frame, hands it to Qwen 2511 with a short "colourise this" prompt, and uses the colour result as the global reference / colour guide. Because it also uses a reference-latent node, pixel shift is effectively prevented and the whole flow is smoother. This is the more automated option.
    • Manual (44 nodes): bypass the reference image and write a more detailed positive prompt. The author ships a generic colour prompt inside the graph as a starting point, and says the best result comes from watching the loaded video and writing colour words that match what you actually see.
  4. Feed the original frames as control. This is the biggest change from the previous generation of this workflow: the depth / Canny / Pose preprocessing group is not needed for colourisation. The black-and-white video goes through a size constraint and straight in as control.
  5. Decode and check. Audio is taken from the source video by default; you can wire in new audio from the front audio input, but then the prompt has to describe that audio.

What changed versus the CN-era workflow

  • The CN model slot is replaced by the colourisation LoRA, which the author uploaded to RunningHub. Locally you drop it into the loras folder.
  • The Load Image group is now used for the first frame of the black-and-white clip, extracted with a first-frame node, rather than as a video-to-video input image.
  • The preprocessor group is bypassed entirely — this is the real simplification.
  • IC-LoRA's principle is unchanged: concatenate two streams (in the fill-model era, two images; for video, two videos) and let the right side be guided by the left. That is also why the author expects the same mechanism to support general "everything transfer" tasks once it is developed further.

Limits

  • The LoRA is explicitly a test version in the author's own words.
  • Frame counts are stated as workable up to about 400, and 800 after optimisation — not unlimited.
  • Route 2 (manual prompt) is faster because it skips the Qwen render, but it demands a much more specific prompt.
  • Reference-latent nodes are doing real work here; removing them can shift pixels.
  • Colour plausibility is a judgement call: the model rebuilds plausible colour, not archival truth.
  • Audio defaults to the source video; changing it means rewriting the prompt around it.

Episode resources

Top comments (0)