DEV Community

AI-KSK
AI-KSK

Posted on Edited on Originally published at huggingface.co

MiniMax H3 Video Control: Eight ControlNet 2.0 Modes, Motion Guidance and 1080P Two-Pass

ControlNet on video is a different problem than on images, because the control signal has to stay coherent frame to frame. This episode maps eight ControlNet Union 2.0 modes across four generation tasks and documents where each one holds.

The four workflows

# Workflow Task
01 CN Union 2.0 video control + local inpaint, VDN 12-step Guided generation
02 Audio-driven digital human, VDN 12-step Talking head
03 General video generation, VDN 12-step Baseline
04 CN Union local inpaint only Pure repaint

The eight modes, ranked by usefulness on video

Strong on video:

  • Depth — the workhorse. Preserves scene geometry across a cut and survives camera movement.
  • Canny — works for hard-edged subjects (architecture, props) and falls apart on hair and foliage.
  • Pose — good for single-subject motion, degrades when limbs occlude each other.

Conditional:

  • Lineart — clean results but sensitive to input line quality; a wobbly line becomes a wobbly render.
  • Normal — useful for lighting-consistent repaints, rarely used alone.

Weak on video:

  • Tile — designed for upscaling stills; on video it produces frame-to-frame texture flicker.
  • Seg — segmentation maps jitter between frames, and the jitter transfers directly into the output.
  • The eighth mode — same problem as seg: per-frame instability is amplified by the model.

That ranking is the practical content of the episode. The modes that work on stills are not the modes that work on video, because stills have no temporal coherence requirement.

The two-pass 1080P finish

VDN 12-step at native resolution gives a clean base; the second pass upscales to 1080P and re-renders at low denoise. The reason to split it: a single pass at 1080P produces softer motion because the sampler spends its budget on resolution instead of on movement.

Audio-driven digital human

Workflow 02 is the one with the most obvious commercial use — one portrait plus one audio track, 12 steps per frame. The limitation is the same as any audio-driven approach: change the audio and you re-render, because the mouth timing is derived from it.

Files

Video walkthrough (Chinese): https://youtu.be/WdAaPq2Xa0I
Bilibili: https://www.bilibili.com/video/BV1SEJm6PE22/

Notes

  • ComfyUI workflow exports; the H3 weights and CN Union 2.0 adapter must exist locally.
  • Tutorial material, not a product.
  • Independent AI-KSK release; not affiliated with the model authors.

Optional API platform - 88API (affiliate link)

Explore 88API for access to multiple AI model APIs. Available models, routes and prices are listed on the platform; Alipay top-ups are supported.

https://88api.ai/sign-up?aff=usx3

Disclosure: I may earn a commission from eligible purchases after you register through this referral link.

Top comments (0)