The Cause of Wrist Skin Collapse is Skinning, Not Pose

📝 Originally published (in Japanese) at forge.workstyle.tech.

"It still looks like the wrist is twisted." When measured after hearing this, the median twist of the wrist was 6–8°. It's a small value. Instead, when the base model was bent by hand and measured, just bending the wrist by 30° caused the edge in the skin weight blend area to shrink to 0.32 times its original length. This is a worse figure than 0.52, which occurred when the same base model was twisted by 85°.

What was being crushed was not the twist but the bend. And this couldn't be fixed on the posing side. To fix it, the skin weights needed to be adjusted. However, smoothing the entire arm resulted in the elbow becoming thinner. Ultimately, the solution settled on processing only the area within a sphere of radius 8 cm centered at the wrist joint.

This is the final installment of the series on creating VRM animations from fixed-camera live-action videos. This time, we'll address the breakdowns caused by issues with the rig and mesh, as well as operational problems found when batch-processing the extracted live-action clips. The first five installments focused on the algorithm—that is, "how to solve it." This time, it's about "why the solved result breaks down visually."

Links to each installment of the series are compiled at the end of the article.

In this article, the process of transferring an actor's movements to a character of a different build is referred to as "retargeting," adjusting the bone directions to match the character's bone lengths is called "lifting," and creating poses from bone rotations is termed "FK application."

Wrist Crushing Is Skin Weights, Not Posing

First, we considered whether changing the avatar would fix the issue. Among the 28 models in the library, all were VRoid base models, and counting the edges in the wrist blend area revealed that all 28 models had 131 edges, which matched. This measurement indicates that the topology (how vertices and edges are connected) is consistent, but it doesn't mean the mesh shapes or weight values are identical. Nonetheless, since it's highly likely that all models inherited the same structure from the base model, we didn't assume that replacing the model would fix the issue. Instead, we decided to investigate the blend area weights themselves.

Smoothing the Entire Arm Made the Elbow Thinner

Initially, we applied Blender's vertex_group_smooth to the entire arm chain. While the wrist issue was resolved, the elbow became thinner, which was pointed out by users. When the affected area expands, more vertices participating in the fold increase, leading to greater volume loss.

Here are the results of measuring the edge length ratios (deformed ÷ rest) in the blend area for various test poses (manually bending the base model). Each cell shows "minimum / maximum."

Blend Area Edge Length Ratio (Min / Max)	VRoid Base Model	Entire Arm Smoothed	Wrist Only
Wrist bent 30°	0.32 / 1.57	0.77 / 1.34	0.76 / 1.16
Wrist bent 45°	0.28 / 1.82	0.56 / 1.48	0.71 / 1.24
Elbow bent 110°	0.12 / 2.24	0.12 / 1.27	0.12 / 2.24 (same as base model)

From this table, we can see that the minimum edge length ratio for the wrist improved from 0.32 in the base model to 0.77 when smoothing the entire arm and 0.76 when smoothing only the wrist. Additionally, when only the wrist is smoothed, the elbow values return to exactly the same as the base model. This allows us to narrow down the changes while maintaining the wrist improvement, so we adopted the method of processing only the wrist.

The visual effect of "the elbow becoming thinner" was separately measured using dance material. The minimum edge length ratio for the elbow worsened from 0.23 to 0.12 when smoothing the entire arm, but returned to 0.23 when only the wrist was smoothed. Properly, the visual difference should be confirmed by comparing images of the same pose.

The current implementation averages only the forearm and hand weights for vertices within a radius of 8 cm from the wrist joint (repeat=30, factor=0.5). Two points were carefully considered:

# pipeline/smooth_arm_weights.py:59-68 (excerpt)
for v in region:
    keep = sum(w for g, w in nxt[v].items() if g not in groups)
    movable = sum(w for g, w in nxt[v].items() if g in groups)
    room = max(totals[v] - keep, 0.0)
    if movable > 1e-12 and room > 0.0:
        s = room / movable
        for g in groups:
            if g in nxt[v]:
                nxt[v][g] *= s
    nxt[v] = {g: w for g, w in nxt[v].items() if w > 1e-6}

• Only the forearm and hand groups are moved. Other groups are carried over without omission (leaking into the torso would cause other breakdowns).
• The weight sum for each vertex is renormalized to the original value (room is the remaining space after subtracting other groups).

Conditions for Corrections to Reach the Viewer

In this process, we re-exported the VRM with the corrected weights and confirmed the effect in the three.js viewer. Since vertex weights are included in glTF, simply replacing the animation won't apply the corrections. The corrected avatar file (in this case, models/kivi_smooth_weights.vrm) needs to be set as the default in the viewer. Conversely, if the exporter settings or baking process differ, the same corrections may not be applied.

Separating Swing and Twist in Range of Motion

There was another cause for the wrist crushing. The range of motion restriction was insufficient, allowing the wrist to twist up to 179° relative to the forearm. This is a different issue from the "small twist causing crushing" mentioned earlier, and it requires pose-side correction. Initially, range restrictions were only applied to the knees, fingers, and neck.

The model works like this: Bone rotation from the rest pose is divided into two components:

• Swing: Where it's pointing. Consists of flexion and lateral angles.
• Twist: Rotation around the bone's own axis.

Swing is clamped by an ellipse.

# pipeline/joint_limits.py:162-172
def clamp_swing(flex, lat, lim: Limit):
    if lim.flex is None or lim.ext is None or lim.lat is None:
        return flex, lat, 0.0
    F = lim.flex if flex >= 0.0 else lim.ext
    L = lim.lat if lat >= 0.0 else (lim.lat_neg if lim.lat_neg is not None else lim.lat)
    s = math.hypot(flex / F, lat / L)
    if s <= 1.0:
        return flex, lat, 0.0
    before = math.hypot(flex, lat)
    flex, lat = flex / s, lat / s
    return flex, lat, before - math.hypot(flex, lat)

The key is to divide both flexion and lateral angles by the same coefficient s. This maintains the ratio between the two angle components while fitting them within the ellipse's boundary (it doesn't preserve 3D orientation in general).

Twist is restricted to the set range, and any excess is passed to the parent. In this implementation, axis-centered rotations concentrated at the wrist are distributed to the forearm.

If Restrictions Are Frequently Hit, Check Thresholds and Axes

The FK application report (apply_result.json's joint_limits) shows the percentage of frames where each joint hits the restriction. There's a rule for interpreting this:

Joints hitting restrictions in many frames may indicate not only movement anomalies but also issues with thresholds or misalignment with rest pose axes.

We actually fixed two restriction-related issues:

• Ankle dorsiflexion: With a 25° setting, about 10% of dance material frames hit the restriction. Relaxed to 38°.
• Toe lateral angle: With a 5° setting, 70% of a specific material hit the restriction. The estimated value had a median of 4° and a p95 of 17°, so it was relaxed to 20°.

Let's clarify something. The restriction values are sourced from comments in the implementation:

# pipeline/joint_limits.py:197-199
# Pipeline joint name -> limits (degrees), against the Mixamo T-pose rest.
# The joint names are the CHILD bone of the joint ("l_elbow" is the forearm).
# Sources: AAOS / Kapandji ranges, rounded up a little for a dancer.

These are AAOS and Kapandji range of motion tables, slightly rounded up for dancers. The explanation "25° is for non-loaded conditions, with 35–40° possible during landing or pushing" was written as a reason for this adjustment, not as a general claim about human anatomy with measurement methods. Implementation comments don't serve as anatomical evidence. What's certain here is the procedure: "The distribution with restrictions removed was measured first to determine if the restrictions were appropriate."

Parent Receiving 350° Twist

Here's a subtle but critical issue:

Twist angles wrap around at ±180°. Angles crossing 180° are read as the opposite sign, and clamping them directly causes flipping between two restriction values. So, the previous frame's value is checked to choose the branch.

However, the side calculating excess twist to pass to the parent and the side clamping the child used different branches based on the same previous frame value. The parent absorbed angles the child hadn't lost, resulting in a maximum twist of 350°. The fix was simple: Both sides now use the same previous frame value and calculate from the same branch.

No Bend Plane for Straight Arms

Another issue: The forearm was completely straight (zero bend) but twisted 72° relative to the upper arm.

The cause was that the upper arm's roll reference was the "plane where the elbow bends." Straight arms have no such plane, so it kept aligning with the old plane from the previous frame. Meanwhile, the forearm rolled based on the absolute reference of the back of the hand, creating a twist across the elbow.

As the arm straightens, the upper arm's roll reference is blended toward the same target as the forearm (the back of the hand). In T-pose dance material, the elbow twist changed from 72° to 0.1° on the left and from -85° to 0.0° on the right. Bent frames remained unchanged.

Pitfalls with Mixamo Rig

Here are issues encountered during implementation related to the rig itself. These are specific to this setup and aren't guaranteed as general specifications.

Issue Encountered	Resolution
IK constraint application caused feet to jump to −81m	In this rig and setup, constraints weren't used; corrected rotations were written directly (all FK from now on)
Confusing world and armature spaces caused hands/feet to jump to 100x positions	Included armature rotation (X=90°) and scale (0.01) in coordinate transformations. Target was armature space (cm-based)
Bone position read from pb.head didn't match expectations	After view_layer.update(), read using (arm.matrix_world @ pb.matrix).to_translation() and matched coordinate spaces for comparison

Two more points: Forgotten location key deletion caused keys from past paths to survive rotation-only rewrites. Since only Hips translate in this rig, location for other bones is reset to 0 every frame. Additionally, in Blender 5.1.2, action.fcurves didn't exist, so fcurves were obtained from the target slot's channel bag.

For reference, here are measured values for Mixamo Y Bot (rest pose, world space, meters):

Hips        (0, 0, 0.998)
Ankle       z = 0.105          ← Ground height
Clavicle    0.129              ← Upper limit for "movable socket" shoulder
Upper Arm/Forearm/Hand  0.274 / 0.276 / 0.110
Thigh/Shin/Foot 0.406 / 0.421 / 0.157
Arm Reach   About 0.55 (0.66 including hand)

With these bone lengths, when the knee is bent approximately 90°, the distance from the hip joint to the ankle is sqrt(0.406² + 0.421²) ≈ 0.58m. The reference point here is the hip joint, not the waist (Hips). To bring the ankle closer from there, the knee needs to be bent more deeply.

Standing Pose Adjustment: Align Foot Width with Midpoint

The estimator accurately captures the actor's standing posture characteristics. In dance material, SMPL-X's knees and ankles were within 5% of 2D detection reprojection. Therefore, instead of dismissing the estimate as incorrect, we considered how to adjust the character's standing pose.

In other words, if the actor stands with toes pointed outward at 10–20° and slightly bow-legged, the character will adopt the same posture. Slender characters may appear "bow-legged." Note that while GVHMR, PromptHMR, and Human3R used here all regress to neutral SMPL(-X), this doesn't mean they lack prior distributions for gender or body type. Using a neutral model and having priors are separate issues.

Adjustments are made via constant offsets in the lifting stage (changing standing pose without affecting choreography). One mistake was made here:

Aligning each leg with its hip joint caused weight shifting to reverse. The more outward a foot is, the more it moves, so an actor standing on one leg appeared to be on the opposite leg.

# pipeline/stance.py:120-126 (excerpt)
mid = 0.5 * (mp["left_ankle"] + mp["right_ankle"])
off = ((ankle - mid) * lat).sum(-1)
shift = (-step_in_pct / 100.0 * off)[:, None] * lat
for name in ("knee", "ankle", "heel", "foot_index"):
    key = f"{side}_{name}"
    if key in out:
        out[key] = out[key] + shift

The metric was the correlation between where the estimate placed the pelvis relative to both feet and where the character placed it. In desk material, this correlation flipped from +0.89 to −0.39. After fixing the implementation to align with the midpoint of both feet, it returned to +0.88. Left-right sway correlation was 0.233 (0.213 for estimates), and foot width narrowed from 1.49 times waist width to 1.11 times.

A similar mistake: Elbow tightening corrections moved the entire forearm parallel while keeping the wrist fixed. In desk material, during intervals where only the actor's fingertips touched, the character's hands were 3.6cm apart. In another interval with elbows 10% narrower, they were 7.3cm apart. The fix was to keep the wrist stationary while changing the forearm's orientation while maintaining its length.

Batch Processing 41 Clips

Now for operational aspects. As my own material increased, I created a browser tool to select and extract clips. Extracted clips were automatically processed in batches, shortest first, using generated specs.

As extraction progressed, batches were also run, and two out of 25 clips failed in the batch processed after extracting 25 clips. Ultimately, all 41 clips were processed. The two failures were of the same type:

• Clips where both hands never came close: Hand linking processing returned numeric 0 early (other paths return dicts). Calling report.update(0) caused failure.
• Clips without faces: Face processing called enumerate(None) without None checks, causing failure.

Both issues arose because return values or missing value handling weren't consistent when there was nothing to process. Paths doing nothing killing the entire execution was the pattern. When writing batch processing, it's better to first confirm what each stage's "do nothing" path passes to the next stage.

Series Summary

Throughout six installments, three things needed confirmation before adding corrections:

1. Where did the error originate? Head orientation required rebuilding downstream three times before realizing the issue was with facial landmark creation.
2. What do the numbers measure? Approximate shapes, different joint points, old logs. Even if values are correctly calculated, they may not match what you want to know.
3. Was the output checked after changes? Besides solver results, check saved animations and viewers. When removing processing based on old measurements, remeasure with current code.

Transferring live-action movements to VRM required examining estimation, measurement, correction, mesh, and export. I hope this series' failure examples provide clues for investigating visual breakdowns.

And leaving records of what didn't work in code comments prevents your future self from retracing the same path. This series itself was written from such comments.

Appendix: Issues Encountered in This Environment

Below are issues encountered when running this pipeline in the following environment: WSL2 Ubuntu 24.04 / Blender 5.1.2 / RTX 4080 Laptop GPU. Symptoms, troubleshooting, and effective countermeasures are summarized. The same countermeasures may not work in different environments.

Blender Segfaults in WSLg

In this environment, running plain blender via WSLg caused segfaults. Setting WAYLAND_DISPLAY or XDG_SESSION_TYPE=x11 didn't help. Only one thing worked:

Point XDG_RUNTIME_DIR to a directory without Wayland sockets

Additionally, GALLIUM_DRIVER=d3d12 and LD_LIBRARY_PATH=/usr/lib/wsl/lib were specified as part of this setup. This combination confirmed Blender's system info returned GPU_RENDERER: D3D12 (NVIDIA GeForce RTX 4080 Laptop GPU) (not llvmpipe). This doesn't guarantee real GPU usage in all cases.

WSL Instance Terminated After 90 Seconds

"Launcher-started Claude Code terminated after a few minutes." Initially, the tool was suspected, but even a simple sleep script terminated after 87–90 seconds. Console presence or sleep granularity didn't matter.

• /proc/uptime kept increasing (VM was running).
• Yet, each wsl.exe launch showed systemd[1]: Startup finished in 1.5s, and journal was corrupted or uncleanly shut down.
• No entries in Windows event logs.

The VM side ran continuously, but the distribution side recorded restarts. After wsl --shutdown, the issue stopped (confirmed over 180 seconds of survival), but the cause of termination remains unconfirmed based on observations alone.

What helped in troubleshooting was running just sleep with the same launch method. Confirming whether application-specific processing was involved narrowed down the investigation range.

Examples of Halts Due to GPU, Memory, Intermediate Files

• Wrong human model: This setup required SMPLX_NEUTRAL.npz. SMPL and SMPL-X have separate distribution sites and registrations, causing acquisition delays.
• GPU shared with other sessions: In an environment with VRAM shortages, hand estimation took 431 seconds per frame with zero detections (normally 5–6ms/frame, ~2.6GB VRAM). File existence alone wasn't enough; processing time and detection counts also needed checking.
• Editing running shell scripts: In this operation, editing running files broke processing twice. Fixed by using a fixed version for execution.
• RSS increased to 47GB during rendering: In this EEVEE animation render, RSS increased ~20MB per frame, causing OOM. Restarting Blender every 400 frames kept RSS under 3.4GB.
• Lost intermediate generation files after reboot: Intermediate files were moved to persistent storage, allowing resumption from mid-process using output reuse.

Stopped Healthy Job After Reading Previous Log

Finally, a self-inflicted accident:

Seeing "1 frame 431000ms" in an append-style log, a healthy job was stopped. That number was from a previous run. The number was correct, but what it pointed to was wrong.

Logs are now separated per run, matching logs being checked with running jobs. This prevents mistaking previous numbers for current anomalies.

Where to Reread From

The installment to revisit depends on the type of visual breakdown: For measurement doubts, Installment 2; for grounding, Installment 3; for arms and hands, Installment 4 and Installment 5; for mesh and rig, this installment.

Series Installments

• Installment 1: Animating VRM from a Single Live-Action Video ── Fixing Remaining Foot, Arm, Finger Breakdowns After Estimation
• Installment 2: Head-Chest Angle Difference 0.000° in All Frames ── Motion Measurement Breakdown Story
• Installment 3: 1% Shorter Legs Caused 16° Knee Bend ── Pitfalls in VRM Grounding Correction
• Installment 4: Arm Penetration Correction Flipped Front to Back ── Dynamic Programming Decides Escape Direction
• Installment 5: "Moving" Thumb Causes Thumbs Up ── Correction Maintaining Grip and Smoothing
• Installment 6 (This Article): Wrist Skin Crushing Cause Was Skinning, Not Pose

Notes

This pipeline is based on squall01337/mixamo-llm-mocap (MIT). The same repository name is explicitly referenced to avoid confusion. Among the quoted code, pipeline/joint_limits.py, pipeline/smooth_arm_weights.py, and pipeline/stance.py are parts added later.

Parts added after the fork aren't publicly available. Reproduction isn't possible just by looking at the repository (quotes include file names and line numbers).

Code is excerpted for explanation. For full implementations including omitted initializations and helper functions, refer to the file and line numbers provided in each section.