While auditing robotics learning datasets, we stumbled onto a problem that seemed simple but turned out to have real depth: detecting frozen video frames.
The naive approach is obvious — compare the first and last frame of each episode. If they're identical, the video is frozen. This is what some popular tools do. But after testing it on real datasets, we found it misses the cases that actually matter.
What We Found in LIBERO
We ran our open-source audit tool RDA against the LIBERO benchmark — 10 manipulation tasks, 379 episodes total. The audit flagged 6 episodes with video freezes (1.6%), all short segments at episode boundaries.
But here's the thing: the first/last frame comparison method would have caught some of these by coincidence, while also producing false positives on episodes where the robot simply starts and ends in the same position (which is completely normal for manipulation tasks).
The real issue isn't whether frames are identical — it's whether the video fails to capture motion that should be happening.
The Action Cross-Reference Approach
This is the key insight: a frozen video isn't the problem. A frozen video while the robot is supposed to be moving — that's the problem.
RDA's video freeze detection works in four stages:
Decode to low-res grayscale (64×64) — noise reduction matters more than resolution
Adaptive thresholding — compute frame-to-frame pixel differences, use the p10 as a noise floor, set threshold at max(0.10, 0.25 × noise_floor). This automatically adapts to different camera qualities across datasets.
Continuous segment detection — only flag sequences of consecutive frozen frames (minimum 0.5s by default), filtering out single-frame noise
Action cross-validation — check the action data during frozen segments. If the robot's commanded actions show movement but the video doesn't change, it's a real freeze. If actions are also near zero, the robot is simply idle.
This distinction eliminates the two failure modes of simpler approaches:
False positives: Robot starts and ends at the same position → first/last frames match → incorrectly flagged as frozen
False negatives: Video freezes for 5 seconds in the middle of an episode → first/last frames are different → freeze completely missed
Validation on G1_WBT
To verify the method, we audited the Unitree G1_WBT_Brainco_Pickup_Pillow dataset — 300 episodes, 4 camera views (head stereo + wrist), 177,811 frames at 30fps.
Results on the 44 episodes with available video:
0 frozen segments detected across all camera streams
Camera sync: temporal offset = 0ms, drift rate = 0ms/min — excellent hardware synchronization
Episode 18's wrist camera had a visual quality dip (score 0.5/1.0, blurry frames), but RDA correctly classified this as "degraded quality" rather than "frozen" — because the frame pixels were changing, just not sharply
This is exactly where the action cross-reference matters: blurry frames have pixel variation, frozen frames don't. The tool distinguishes between them.
What Else We Found
Video freeze turned out to be the least interesting finding. The more impactful issues across these datasets:
LIBERO — Action discontinuities: 3,622 spikes across 100% of episodes
LIBERO — High idle ratio: Only 28.3% of frames show effective motion
G1_WBT — Missing video files: 278 out of 300 episodes have no video streams
G1_WBT — Joint limit violations: 7 out of 36 DOF exceed configured limits
These structural issues are harder to spot than frozen frames and likely have a bigger impact on policy training. Video freeze is visible and easy to understand; action discontinuity and low state-space occupancy are invisible but potentially more harmful.
The Bigger Picture
Robot learning datasets are growing fast — Open X-Embodiment, DROID, AgiBot World — but quality assurance hasn't kept up. Most datasets are published with basic sanity checks but no systematic audit of temporal integrity, action continuity, or state-space coverage.
We built RDA because we kept running into these issues during our own work. It's open source (v0.9.9 on PyPI), supports LeRobot v2/v3 format, and produces structured JSON reports. We've audited 12+ datasets so far and shared findings directly with dataset maintainers through GitHub issues.
If you're collecting robot data or training policies on open datasets, video freeze is worth checking — but it's probably not the biggest quality issue hiding in your data.
Tool: RDA (Robot Data Audit)https://github.com/liesliy/rda — pip install robot-data-audit
Top comments (0)