DEV Community

AI-KSK
AI-KSK

Posted on

Training an LTX 2.3 LoRA with AI Toolkit: the parameter set that actually matters

#comfyui #ai #tutorial

LTX 2.3 is an audio-video model, so its LoRA training has one foot in video consistency and one foot in voice. This episode is a full walkthrough on a rented 4090 48G through the author's own one-click AI Toolkit image — dataset layout, trigger word, quantisation, rank, steps, learning rate, and the switches most people get wrong (audio training, latent caching, and why 4-bit main-model quantisation is a bad idea here).

Evidence note: this episode ships no workflow JSON (工作流链接：无). The image page itself was verified by GET and contributes a few hard facts; everything else — training hyperparameters, step counts, resolutions — is what the author states in 字幕.srt and is reported as his recommendation, not as a reproduced benchmark.

Where this fits

You want a LoRA that keeps a character's face consistent across LTX 2.3 clips, and optionally carries that character's voice. The author's framing is blunt: the 22B Dev base model is enormous, so train on a rented instance rather than locally, and use a 4090 48G as the practical floor.

The image

  • Compshare image: LTX-2.3 LoRA训练 +AIToolkit 4090/48G/5090 一键开训
  • https://www.compshare.cn/images/xahyFyiJf09D?referral_code=A7fTQUYY9lCDFJwA6MMvoq
  • Page facts confirmed by GET: 110 GB image, PyTorch 12.8 / CUDA 12.8, JupyterLab on port 8888, presets for 4090 48G and 5090, launch command cd /workspace/ai-toolkit && source myenv/bin/activate && cd ui && npm run build_and_start.
  • The author recommends at least a 5090 as the minimum configuration and says going lower costs quality noticeably. He personally rents the 4090 48G and calls it the most cost-effective option.

Step by step

  1. Open the image page, copy the launch command from the description, click deploy.
  2. Enter JupyterLab, open a terminal anywhere, paste the command, press enter.
  3. When the address appears (port 8675), click through to AI Toolkit.
  4. The training queue ships two presets: one for 4090 48G, one for 5090. The author points out his preset is misnamed — the published version is the 5090 one — and that he is actually renting the 48G 4090.
  5. Create a dataset. Any name. This walkthrough includes audio, so the dataset contains video and audio.
  6. Tagging. LTX 2.3 does not officially document a tagging scheme; the author considers the AI Toolkit's own convention sufficient, and says you can hand the rule to an LLM to tag for you.
  7. Trigger word. He puts a fixed phrase — "a man named Tommy Shelby" — in every caption, repeated identically. More complex phrasing is fine as long as it is identical everywhere.
  8. Video format. Plain MP4, 1920x1088, length 5 seconds = 121 frames, fps 24. These are his recommended LTX settings.
  9. Pick the task that matches your GPU, select the dataset, update the task, run.

The hyperparameters

Setting Value Note
Low-VRAM layer offload on author: mandatory below 80 GB
Transformer offload 100%
Text-encoder offload 0
Main model quantisation Float8 4-bit visibly degrades consistency
Text encoder quantisation 4-bit separate, and acceptable
Rank 64
Save precision default
Save interval every 250 steps, max 10
Batch size 1 "do not go above 1" for video
Gradient accumulation 1 follows batch size
Steps 10,000
Optimizer default AI Toolkit default
Learning rate 0.0001
Text embedding cache on lowers the performance cost
Latent cache on
Audio training on only if the dataset really contains the voice
Regularisation off
I2V off not used here
Audio normalisation off only needed if levels differ a lot
Preserve pitch off not needed at 121 frames / 24 fps
Horizontal / vertical flip off flipping breaks camera language
Training resolution 768 48G 4090 could push 1024-1280; 5090 only 512
Sampling off samples rarely match test results

Two of those deserve emphasis. Float8 for the main model, not 4-bit — the author is explicit that 4-bit hurts consistency, which is the entire point of a character LoRA. And batch size 1 — for video training he says not to exceed 1.

How long is enough

The author observes that consistency is already holding well at step 3,250, and he runs the full 10,000. He also says plainly that LTX 2.3 training is not very different from LTX 2 training — the parameters are similar, the performance cost is higher.

Getting the file out

If you keep the page open, click download. If you navigated away, the output sits in the AI Toolkit workspace under the output folder.

Limits

  • No workflow JSON is shipped, so nothing here is a graph parameter — these are training settings from the video.
  • The author states that most of the settings do not strongly affect the final result; they mostly affect performance. Without A/B runs he will not rank them.
  • Tagging has no official LTX 2.3 specification; the AI Toolkit convention is a judgement call.
  • Sampling is disabled on purpose because samples do not correspond to his test results.
  • Resolution is a VRAM trade-off, not a quality setting you can max out.
  • Training on a 5090 caps the resolution at 512 in his configuration.
  • The 3,250-step observation is his own impression, not a measured convergence curve.

Episode resources

Top comments (0)