Skeleton-guided video generation is one of the more reliable control signals available: a pose sequence is unambiguous in a way that text is not. This workflow merges SkyReels V3's R2V capability with VACE's control path on a Wan 14B base.
The graph
| Node | Value |
|---|---|
UNETLoader |
wan-14B_vace_skyreels_v3_R2V_e4m3fn_v1.safetensors, dtype fp8_e4m3fn
|
CLIPLoader |
umt5_xxl_fp8_e4m3fn_scaled.safetensors, type wan
|
VAELoader |
Wan2.1_VAE.pth |
66 nodes.
The merged checkpoint
The UNET filename describes exactly what it is:
-
wan-14B— the Wan 14B base -
vace— VACE control path included -
skyreels_v3— SkyReels V3 R2V behaviour -
R2V— reference-to-video -
e4m3fn— FP8 quantisation, e4m3 -
v1— first merged version
A merged checkpoint like this bundles behaviour that would otherwise be separate LoRAs or adapters. The advantage is that you load one file instead of three and there is no ordering problem between them. The disadvantage is that you cannot tune the individual contributions — you get the merge as designed.
The VAE is .pth, not .safetensors
Wan2.1_VAE.pth — a PyTorch pickle rather than safetensors.
This matters for two reasons. First, .pth files can execute arbitrary code on load, so they should only come from a source you trust. Second, tooling that expects safetensors will not find it. If you have been collecting safetensors only, this workflow will fail at the VAE load, not at the model load.
The umt5-xxl text encoder
umt5_xxl_fp8_e4m3fn_scaled with type wan.
UMT5-XXL is a large multilingual encoder — this is why Wan-family models handle Chinese prompts better than CLIP-based pipelines. The scaled suffix means the weights have been rescaled for FP8, which is different from simply quantising them.
Why skeleton guidance works
A pose skeleton supplies: joint positions, limb angles, and a temporal progression. There is no ambiguity in the way there is with "the person raises their arm" — the arm either goes where the skeleton says or the render is visibly wrong.
The failure modes are correspondingly specific:
- Skeleton and body proportions disagree — the model produces a body that has to stretch to hit the joints
- Occlusion is unconstrained — a skeleton does not say which limb is in front
- Hand and face detail is absent — skeletons stop at the wrist
Those three are why skeleton control produces strong overall motion and weak detail.
Where this breaks
-
.pthVAE — tooling incompatibility and a trust question. - Merged checkpoint means no component-level tuning.
- Hands. Skeletons stop at the wrist; the model has to invent fingers from nothing.
- Occlusion ordering. With two limbs crossing, the skeleton does not say which is in front and the model guesses.
- FP8 quantisation on both UNET and encoder — detail loss compounds across two quantised stages.
Practical approach
Start with a single-subject, single-limb action where occlusion is not an issue. Confirm the skeleton aligns with the rendered body. Only then try two-limb interactions, and expect to need a mask or a second pass for hands.
Workflows
One graph, published on RunningHub with invite code rh-v1111: VACE + SkyReels V3 R2V merged skeleton-guided video.
Files
https://huggingface.co/AI-KSK/ksk-video-186-comfyui-workflows
Notes
- Workflow file only. Model weights are not included; each keeps its own licence.
- Tutorial export, not a product.
- Node count and loader values quoted above are read from the published JSON file.
- Independent AI-KSK release. Not affiliated with the model authors or RunningHub.
Optional API platform - 88API (affiliate link)
Explore 88API for access to multiple AI model APIs. Available models, routes and prices are listed on the platform; Alipay top-ups are supported.
https://88api.ai/sign-up?aff=usx3
Disclosure: I may earn a commission from eligible purchases after you register through this referral link.
Top comments (0)