Most reference-edit workflows use one reference. This one uses two simultaneously: an RGB reference for appearance and a depth reference for geometry. The combination unlocks doing four things in a single pass that normally need four passes.
What the two references each carry
RGB reference — who the character is. Face structure, hair, costume, colour.
Depth reference — where everything is. Camera position, subject distance from lens, scene layout.
Using both means the new character inherits appearance from one and spatial position from the other. Using only RGB, the character lands wherever the model feels like putting it.
The four jobs in one pass
- Character replacement — the subject becomes the reference character.
- Action inheritance — the motion of the source performance transfers.
- Scene and camera preservation — because depth is conditioned, the camera does not wander and the set does not rebuild itself between frames.
- Audio-driven lip sync — when audio is supplied, mouth movement is driven by it.
Doing these separately is how most pipelines end up with a character that has the right face but the wrong head angle, because each pass re-decides.
Three variants
| File | Use |
|---|---|
| Standard | Default settings |
| Silent (无声版) | No audio conditioning |
| Face Refine | Extra pass on the face after the main render |
The Face Refine variant is worth the extra time whenever the character is in frame larger than about a third of the image height. At smaller sizes the refine pass adds nothing.
Where it breaks
Depth conditioning is precise but rigid. If the source performance moves the subject closer to camera in a way the depth map does not describe (because the depth was extracted from a single frame), the composite drifts at that moment. The fix is a per-frame depth extraction, which is much more expensive — and the episode shows the difference.
Files
- HF: https://huggingface.co/AI-KSK/minimax-h3-role-swap-reference-edit
- ModelScope: https://modelscope.cn/models/Sunjian520/minimax-h3-role-swap-reference-edit
Video walkthrough (Chinese): https://youtu.be/UespHFT9cjc
Bilibili: https://www.bilibili.com/video/BV1VjaD6jEU3/
Notes
- Three ComfyUI workflow exports; H3 weights and the depth extractor must exist locally.
- Tutorial material, not a product.
- Independent AI-KSK release; not affiliated with the model authors.
Top comments (0)