How to Build a Frame-by-Frame Hip-Drift Test for an AI Dance Clip

#video

A generated dance clip can look fine as a thumbnail and still fail in motion: the face drifts, feet slide, the torso stretches, or the apparent hip movement comes from the camera rather than the body.

This tutorial turns that subjective review into a small OpenCV test. The input fixture and five-second result come from the public example on the Voor AI Hip Shake generator; no generation credits are needed to reproduce the analysis.

Define the signals

For each sampled frame, track four normalized values:

face-center drift from the baseline;
torso-width ratio;
left and right foot displacement;
horizontal hip-center amplitude.

Normalize pixel distances by frame width or subject height. That makes thresholds portable across exports.

from dataclasses import dataclass

@dataclass
class FrameMetric:
    t: float
    face_drift: float
    torso_ratio: float
    left_foot_drift: float
    right_foot_drift: float
    hip_x: float

def failed(m: FrameMetric) -> list[str]:
    errors = []
    if m.face_drift > 0.035:
        errors.append("face drift")
    if not 0.90 <= m.torso_ratio <= 1.10:
        errors.append("torso stretch")
    if max(m.left_foot_drift, m.right_foot_drift) > 0.04:
        errors.append("foot slide")
    return errors

The values above are starter thresholds, not universal biological rules. Calibrate them against a small set of clips your team has already accepted.

Sample deterministically

OpenCV makes a reproducible one-frame-per-half-second pass straightforward:

import cv2

cap = cv2.VideoCapture("hip-result.mp4")
fps = cap.get(cv2.CAP_PROP_FPS)
step = max(1, round(fps * 0.5))
frames = []

i = 0
while True:
    ok, frame = cap.read()
    if not ok:
        break
    if i % step == 0:
        frames.append((i / fps, frame))
    i += 1

assert frames, "video produced no decodable frames"

Keep the raw timestamp with every metric. “The face drifts at 3.5 seconds” is actionable; “the face looks weird” is not.

Separate intended motion from instability

Hip amplitude should oscillate. Face identity and torso proportions should not. A simple release rule can require:

def release_ok(metrics):
    drift_failures = sum(bool(failed(m)) for m in metrics)
    hip_span = max(m.hip_x for m in metrics) - min(m.hip_x for m in metrics)
    return drift_failures == 0 and hip_span >= 0.06

This catches two opposite failures: unstable anatomy and a “dance” clip with almost no measurable lateral motion.

Preserve the generator state with the report

The current page shows a required image input, a 5-second default within a documented 4–30 second range, Public watermarked visibility, and 628 credits for the selected state. Save these observations with the test artifact because a model or setting change can invalidate old baselines.

Do not claim that a UI success toast equals a passing video. Decode the final MP4, count frames, and run the metric gate.

Turn failures into a compact CI artifact

Emit JSON plus a contact sheet of the worst timestamps. A pull request reviewer should be able to see the evidence without replaying the whole clip.

{
  "duration_s": 5.04,
  "sample_interval_s": 0.5,
  "release_ok": false,
  "failures": [{"t": 3.5, "type": "face drift", "value": 0.041}]
}

Use the public fixture page and its real result as a reproducible starting point. The aim is not to automate taste; it is to make identity, anatomy, and motion regressions visible before a clip ships.

DEV Community