DEV Community

AI-KSK
AI-KSK

Posted on

Two-Person Dialogue That Holds: Dual-Character LoRA Plus Prompt Relay, and a Subtitle Fixer

#comfyui #ai #tutorial

Two-person dialogue used to fail in a specific way: the model would answer its own question, or swap who was speaking. The combination here fixes both — a dual-character LoRA pins identity, Prompt Relay pins who speaks in which segment, and a 0.5-version repair graph cleans up the Chinese gibberish the base model leaves behind.

Evidence note: five workflow JSON files ship with this episode and were parsed directly; node counts, checkpoint, LoRA strengths, samplers, sigma schedules and latent sizes below come from those files.

When this is useful

  • Two characters on screen where both must keep their identity across a long exchange.
  • Scripted dialogue that needs per-segment control of who speaks when.
  • Cleaning the garbled Chinese text that appears once dialogue runs past about five seconds.
  • Audio-driven variants where the performance must follow a supplied track.

How this differs from the previous build

The improvement is prompt architecture plus one LoRA, not a new base model.

  • Dual-character LoRA at 0.75 pins both identities. It is the single largest change.
  • Prompt Relay supplies the storyboard. Two prompt dialects — the LoRA author's and Relay's — are merged, which is why the author built a prompt app rather than documenting a template by hand.
  • A transition LoRA at 0.5 smooths shot changes when the dialogue has cuts.
  • The repair graph is a 0.5, deliberately provisional. The author shipped a loop version in the same batch, and says a fuller version was still coming.
  • Resolution was raised on the repair path to reduce latent-space loss on the source clip, at a small speed cost.

Running it

  1. Load an image containing both characters. Non-human is acceptable, but faces must be readable — eyes and mouth at minimum.
  2. Set the initial size in the 100k-150k pixel band.
  3. Set frames and frame rate. The author has run 480 frames, over twenty seconds, without problems.
  4. Write the prompt using both rule sets: the LoRA's per-speaker structure and Relay's segment structure. Use the author's app or feed both rule sets to an LLM.
  5. Run the dialogue graph. For audio-driven output, load the audio track instead of relying on segment prompts.
  6. Add the transition LoRA and raise its weight if the cut is jarring.
  7. For the text artifacts, load the clip plus the same reference image used for the dialogue, paint the subtitle region, and run the repair graph.

Parameters, and where the numbers come from

Item Value in the shipped graphs
Checkpoint ltx-2.3-22b-distilled-1.1.safetensors
Text encoder gemma_3_12B_it.safetensors
Samplers euler_ancestral_cfg_pp, euler_cfg_pp
CFG 1 (CFGGuider)
Sigma schedule, main 1.0, 0.99375, 0.9875, 0.98125, 0.975, 0.909375, 0.725, 0.421875, 0.0
Sigma schedule, refine 0.85, 0.7250, 0.4219, 0.0
Video latent 320x288, length 97 (dialogue); 768x512, length 89 (loop repair)
Audio latent 25 fps (dialogue), 24 fps (repair)
NAG scale 11, alpha 0.25, tau 2.5 (dialogue); 15 / 0.4 / 0.47 (audio-driven dialogue)
LoRA, dual character LTX2.3-IC-LORA-Dual-Character.safetensors at 0.75
LoRA, VBVR Ltx2.3-Licon-VBVR-I2V-240K-R32.safetensors at 1.0
LoRA, accelerate ltx-2.3-22b-distilled-lora-384.safetensors at 0.5
LoRA, motion track ltx-2.3-22b-ic-lora-motion-track-control-ref0.5.safetensors at 1.0
LoRA, transition ltx2.3-transition.safetensors at 0.5
Repair UNET ltx-2.3-22b-distilled_transformer_only_fp8_input_scaled_v3.safetensors
Repair text encoders gemma_3_12B_it_fp8_e4m3fn.safetensors + ltx-2.3_text_projection_bf16.safetensors
Repair scheduler bong_tangent, steps 8, denoise 1

Note the repair graphs load the transformer-only FP8 UNET rather than the full checkpoint — they are materially different graphs, not a mode switch.

Honest limits

  • The repair graph only works on low-motion footage. Basketball or fast action will not resolve cleanly.
  • The subtitle fix is explicitly a 0.5 — the author calls it provisional and points at the loop variant for longer clips.
  • Garbled subtitles appear once Chinese dialogue exceeds roughly five seconds; shortening shots is the alternative to repairing them.
  • The dual-character LoRA needs a prompt that names who stands where and who speaks when; vague prompts still drift.
  • The 240K VBVR build here is older than the 390K release used in later episodes.

Episode resources

RunningHub workflows

Top comments (0)