DEV Community

AI-KSK
AI-KSK

Posted on

SkyReels V3 R2V x VACE Merge: Skeleton-Driven Character Video With a Stronger Reference

#comfyui #ai #tutorial

Merging two models is usually a lottery. This one has a clear story: take SkyReels V3's R2V path and merge it with VACE, and you get better consistency, better background similarity and better motion following than stock VACE. The graph keeps both model variants side by side so you can see the difference on the same inputs.

Evidence note: this episode ships one workflow JSON (66 nodes). Node-level values below are read from it. The author's model naming in the narration differs from the filenames, and that is flagged rather than smoothed over.

When this is useful

  • You have a driving video of a person moving and one or more reference images, and you need the subject to follow the motion without losing identity or background.
  • You are still on Wan 2.1 tooling and do not want to rebuild the whole stack for a merged model.
  • You want skeleton guidance (not just depth) and up to four reference images in one pass.

What the graph does

  1. Video in, at Wan's rate. VHS_LoadVideo runs at force_rate 16 with frame_load_cap 81, which is the ~5 second baseline the author keeps. Output size and length are detected from this node, and the author notes the 81-frame cap is a default, not a ceiling, when you are doing guidance.
  2. Skeleton preprocessing. A DWPreprocessor (DWPose, yolox_l + dw-ll_ucoco_384) converts the video to open-pose. Depth is a valid alternative; the author picks skeleton here.
  3. Reference images into Phantom. LoadImage feeds ImageBatchMultiple+, which is labelled as accepting up to four images, and the author says four is already plenty. References go to the Phantom side, not to the VACE side.
  4. Phantom + VACE sampling. WanVacePhantomSimpleV2 is configured at 832x480, 81 frames, batch 1, with a strength field at 1.5. A ModelSamplingSD3 sits at shift 9, which is the value the author recommends. The KSampler runs 15 steps at CFG 3, euler, simple: more steps than a typical Wan graph, and the author says he may add an acceleration model later.
  5. Two model variants in parallel. One branch loads wan-14B_vace_skyreels_v3_R2V_e4m3fn_v1.safetensors with umt5_xxl_fp8_e4m3fn_scaled and Wan2.1_VAE.pth; the other loads wan-14B_vace_skyreels_v3_R2V_fp32_v1.safetensors with the NSFW text encoder and the same VAE. Both run the same sampler settings, and both write comparison videos through VHS_VideoCombine.
  6. Size constraints after pose. ImageResize+ at 832x832 (nearest, keep proportion) normalises the pose frames before they reach the sampler. The author explains why this second constraint exists: DW OpenPose generates at the size it is told, so re-applying the size constraint afterwards is what makes the final value correct. Lowering the size produced visibly blurry results in his tests.

A naming mismatch you should know about

The narration describes the two model files as an FP32 version and an FP16 version, and says the FP32 one threw errors in his runs. The filenames in the graph are ..._fp32_v1 and ..._e4m3fn_v1, and both UNET loaders are set to fp8_e4m3fn_fast weight dtype. So treat "FP16" in the video as shorthand for the non-FP32 variant rather than a literal dtype claim.

Related: the author also says he recommends downloading the smaller variant because the large one may not fit and he hit problems with it. That is consistent with the file names even if the spoken labels are loose.

Step by step

  1. Load the workflow. If you are on RunningHub, it runs online as-is.
  2. Load your driving video into VHS_LoadVideo. Keep force_rate 16 to match Wan 2.1.
  3. Put one to four reference images into the Phantom image batch.
  4. If you do not want video guidance at all, disconnect the video branch or set the VACE reference strength to 0; the author notes the node accepts 0 for exactly this case.
  5. Set width and height. The author uses a standard 720x1280 and warns that dropping below it gets blurry.
  6. Enable the Wan format constraint so the resolution does not silently break.
  7. Keep shift 9, 15 steps, CFG 3.
  8. Choose your text encoder: the graph ships a general umt5_xxl_fp8 and an NSFW variant; the author notes RunningHub's online encoder list is broad enough to swap either.
  9. Write the prompt. Because reference strength is high, a short description of who is doing what is enough; longer descriptions are allowed but not required.
  10. Run and compare the two variant outputs before deciding which to keep.

Parameter basis

From VACE_Skyreels_V3_R2V_Merge骨骼引导视频.json (66 nodes):

Node Value
UNETLoader (A) wan-14B_vace_skyreels_v3_R2V_e4m3fn_v1.safetensors, fp8_e4m3fn_fast
UNETLoader (B) wan-14B_vace_skyreels_v3_R2V_fp32_v1.safetensors, fp8_e4m3fn_fast
CLIPLoader umt5_xxl_fp8_e4m3fn_scaled / nsfw_wan_umt5-xxl_fp8_scaled, type wan
VAELoader Wan2.1_VAE.pth
KSampler 15 steps, CFG 3, euler, simple, denoise 1
ModelSamplingSD3 9
WanVacePhantomSimpleV2 832 x 480, 81 frames, batch 1, strength 1.5
VHS_LoadVideo force_rate 16, frame_load_cap 81
ImageResize+ 832 x 832, nearest, keep proportion, multiple of 16
ImageResizeKJv2 512 x 512, bilinear, pad edge
DWPreprocessor yolox_l + dw-ll_ucoco_384, 512
Output VHS_VideoCombine, crf 19, yuv420p

Limits

  • Two variants, one honest result. The author reports the FP32 variant erroring on his machine; the comparison in the graph is a convenience, not a controlled A/B.
  • Still Wan 2.1 rates. 16 fps is a Wan 2.1 constant, so output is 16 fps regardless of source.
  • Quality is assessed visually. "Higher consistency, better background, better motion following" is the author's comparison against stock VACE, with no metric.
  • The 81-frame default is easy to misread as a hard cap; it is a frame_load_cap default.
  • Naming drift between the narration (FP16/FP32) and the filenames (e4m3fn_v1/fp32_v1) will confuse anyone who follows the audio only.
  • Acceleration is not wired in yet. The author explicitly says 15 steps is heavy and he is looking at adding an accelerator.

Episode resources

Top comments (0)