DEV Community

liesliy
liesliy

Posted on

I Audited AgiBot's New RL Dataset with RDA

#robotics #data #test #opensource

AgiBot just released AGIBOT WORLD 2026 Phase 3, a real-robot reinforcement-learning dataset. I used it as an independent test of RDA: can a third-party audit tool run against a fresh, large dataset without any tweaks?

Key Finding: RDA Spikes Align with AgiBot's Human-Takeover Labels


The real-robot HG-DAgger package includes an official intervened column: frame-by-frame human-takeover labels. This gave us a rare chance to validate RDA.

I aligned all 1,712 RDA action-discontinuity spikes against these labels:

  • 226 spikes (13.2%) fall within ±1 frame of a takeover transition
  • Random baseline: ~4.3%
  • That is a 3.1× enrichment overall (1.6×–5.2× per episode)

Most spikes are normal teleop motion — which is why RDA labels them REVIEW, not FAIL. But the clear enrichment at takeover boundaries shows the signal is grounded in real events, not pure noise.

Verdict Distribution: 0 EXCLUDE / 0 FAIL

Across 1,112 episodes (5 simulation tasks + one real-robot package):

Verdict Count
PASS 31
REVIEW 1,081
EXCLUDE 0
FAIL 0

All 31 PASS episodes come from simulation tasks. Verdict layering works — it is not a blanket REVIEW.

What Was Covered

I did not settle for a single tiny split. I ran RDA over all 5 simulation tasks and the smallest real-robot RL package (3.57 GB — the others are 17–50 GB each and would not fit on my drive):

Dataset Episodes Notes
Simulation × 5 tasks 1,102 200–283 episodes per task
Real robot HG-DAgger 10 23,658 frames, 30 fps

Data Engineering Passed the Audit

  • 4,448 integrity checks (missing frames, invalid values, schema, timestamps), 0 failures
  • Constant 30 fps
  • 6 sampled info.json files, all LeRobot v2.1

The data is ready for training. REVIEW items are worth a spot-check, but nothing needs to be cleaned before use.

Run It Yourself

Grab the dataset from the official HF repo (simulation splits are a few hundred MB; the audit doesn't read video files, so you can skip the videos/ tarballs), then:

pip install robot-data-audit
rda audit ./your_dataset_dir
rda recommend ./your_dataset_dir --offline
Enter fullscreen mode Exit fullscreen mode

The output is per-episode detail plus an overall verdict, in JSON so it plugs into any pipeline.

Scope Boundary (Said Honestly)

  • Tested: 5/5 simulation tasks (~500 MB) and 1/50 real-robot packages (2%).
  • Not tested: the remaining 49 real-robot packages (~3.9 TB), ImitationLearning 4.4 TB, and RichInteraction 2.3 TB.
  • Why the format conclusion generalizes: every sampled info.json uses LeRobot v2.1 with the same pipeline.
  • No video files were read, so you can skip the videos/ tarballs to save bandwidth.

The full 6 JSON audit reports are on hand for anyone who wants to reproduce or challenge the numbers.

RDA (Robot Data Audit) is an open-source tool I maintain. PyPI: robot-data-audit. GitHub: liesliy/rda.

Top comments (0)