DEV Community

Kunal
Kunal

Posted on

I built a tool that finds silent corruption in robot training datasets. Now it fixes it too.

#python #machinelearning #robotics #opensource

That audit became trajlens, an open source lint tool for LeRobot datasets. v0.1 could detect 20+ classes of these problems and give a dataset a trust score.

Today I released v0.2. It doesn't just detect anymore. It repairs.

What's new

Three fixers, one command.

pip install trajlens
trajlens fix ./my_dataset --out ./my_dataset_repaired --apply
Enter fullscreen mode Exit fullscreen mode
  • timestamp_dedrift rewrites drifted timestamps back to frame_index / fps, matching the exact float32 quantization the detection check uses
  • stats_recompute streams the full dataset through a Welford pass and rewrites stats.json with correct mean, std, min, max, and count
  • episode_reindex derives true episode boundaries from the per row index column in the data itself and corrects the metadata to match

Safety was the whole design problem. A repair tool that writes wrong data is worse than no tool. So every fixer is:

  • Copy on write. The original dataset is never touched. Ever.
  • Dry run by default. You see a full diff of what would change before anything is written.
  • Round trip tested. Every fixer's test suite proves that repair followed by re-lint clears the finding and introduces no new ones.
  • Fail closed. If a dataset is internally inconsistent in a way that makes repair ambiguous, you get a clear error, not a guess.

The hardest case was episode reindexing. The corruption it fixes is silent by nature, so a buggy fixer could manufacture the exact corruption it claims to repair. The fixer treats the actual data as ground truth and only ever corrects metadata to match it, never the reverse. I also verified the load bearing assumption (that every v3.0 data row carries a trustworthy global index column) against the lerobot source at the exact commit, not from memory.

A web dashboard.

trajlens web ./my_dataset
Enter fullscreen mode Exit fullscreen mode

One command, localhost only, renders the full lint report in your browser. Trust score gauge, severity color coding, per check drill down. No React build chain, no CDN calls, no analytics. It's a single static page served by FastAPI, because a tool you point at private robot data should not phone home through your browser.

Fun bug from building it: all 437 tests were green and the dashboard was completely broken in a real browser. The Content Security Policy I'd set (no inline scripts) was blocking the page's own inline JavaScript. Test clients check that headers exist. Only browsers enforce them. The fix was externalizing the JS and CSS rather than loosening the policy, plus a structural test that fails if anyone ever adds an inline script block again.

Why this matters

Imitation learning is only as good as its demonstrations. The ecosystem has great tools for collecting data and training policies, and almost nothing for answering "is this dataset actually what it claims to be." The bugs trajlens catches are real, documented issues in the wild (LeRobot #2401 and #3177 are two of them), and they don't announce themselves.

If you train on LeRobot format data, two minutes of linting might save you a week of debugging a policy that won't converge.

Links

v0.3 direction: more fixers, deeper per episode analytics, and upstreaming the boundary corruption fix to LeRobot itself. Issues and PRs welcome. If you run it on your dataset and it finds something weird, I genuinely want to hear about it.

Top comments (0)