Swapping a character in a video splits into two problems that people usually conflate: deciding what gets replaced, and making the replacement move the way the original did.
Character Swap 1.5 handles both in one graph, with SAM 3.1 for the first and motion transfer for the second. One workflow, eight steps.
What each half contributes
SAM 3.1 runs the masking. Prompt-segment the subject you want to replace and it returns a per-frame mask. This is the part that used to be manual rotoscoping: a fighter turning through 180 degrees means the mask has to track limbs, hair and any held props across every frame.
Motion transfer runs the replacement. The source clip's motion drives the new character. Body shape, limb timing and the general read of the movement come from the original performance.
Why the mask quality decides everything
The motion transfer half is actually well-behaved. The failures almost always trace back to the mask:
- Occlusion. A hand passing in front of the torso needs two mask regions, not one. Single-region masks ghost at the crossing.
- Fast motion. Between two frames with large displacement, the mask can lose the limb entirely for a frame or two. That reads as a flicker.
- Soft edges. Hair and loose fabric need a feathered mask. A hard-edged mask produces a visible halo.
SAM 3.1 is considerably better than the previous generation at the first two. The third is still on you — feather the mask.
The eight-step claim
The VDN8 variant uses a distilled sampler, eight steps per frame. In practice that is the difference between a swap that renders overnight and one that renders during a coffee break. The quality cost lands where you would expect: fine fabric texture and small facial details are slightly softer than a 20-step pass.
Where this is used
- Replacing a performer in an existing shot without a reshoot.
- Casting a specific character into existing motion reference.
- Localisation: same performance, different character per market.
The last one is the commercially interesting case — one shot, several faces, no second shoot.
Files
- HF: https://huggingface.co/AI-KSK/h3-character-swap-sam-vdn8
- ModelScope: https://modelscope.cn/models/Sunjian520/h3-character-swap-sam-vdn8
Video walkthrough (Chinese): https://youtu.be/iuEmeHgHGIg
Bilibili: https://www.bilibili.com/video/BV1GLnH6NEPC/
Notes
- Single ComfyUI workflow export; SAM 3.1 and the H3 weights must exist locally.
- Tutorial material, not a product.
- Independent AI-KSK release; not affiliated with the model authors.
Optional API platform - 88API (affiliate link)
Explore 88API for access to multiple AI model APIs. Available models, routes and prices are listed on the platform; Alipay top-ups are supported.
https://88api.ai/sign-up?aff=usx3
Disclosure: I may earn a commission from eligible purchases after you register through this referral link.
Top comments (0)