DEV Community: liesliy

Robot Training Data Is Messier Than You Think: Auditing 4,959 Episodes with an Open-Source Tool

liesliy — Fri, 21 Aug 2026 08:48:10 +0000

Robot learning is getting better very quickly.

We now have better policy architectures, more capable simulation environments, standardized dataset formats such as LeRobot, and an increasing number of public robot manipulation datasets.

But there is a basic question that is surprisingly difficult to answer:

How do we know whether a robot training dataset is actually usable?

A dataset can be perfectly readable and still be a poor training dataset.

It can have valid Parquet files, correct schemas, and complete metadata while containing excessive idle motion, action discontinuities, sampling problems, distribution anomalies, or episodes that deserve human review.

I built RDA (Robot Data Audit) to explore this problem.

I recently ran it across 12 local robot datasets and 4,959 episodes.

Here is what I found.

"Can I Load It?" Is Not the Same as "Can I Train on It?"

Robot datasets can contain:

camera observations
joint states
actions
timestamps
task metadata
robot configuration
multiple sensor streams

Most data pipelines start with a simple question:

Can I load the dataset?

That's necessary, but it isn't enough.

I think robot data quality needs to be considered in several layers:

Layer	Question
Structural integrity	Does the data actually exist and load correctly?
Schema & temporal integrity	Are fields, values, and timestamps valid?
Behavioral quality	Does the robot motion contain suspicious patterns?
Task relevance	Are those patterns actually harmful for this task?

RDA currently focuses primarily on the first three layers and produces structured evidence that can be used to investigate the fourth.

This distinction matters.

A statistical anomaly is not automatically a failed demonstration.

And a dataset that passes structural validation is not automatically good training data.

What RDA Actually Measures

RDA is an open-source, local-first auditing tool for LeRobot-format robot manipulation datasets.

The core audit runs locally and does not require a GPU.

The current version checks 13 metrics across three layers.

Layer 1 — Structural Integrity

Deterministic checks include:

missing frames
NaN / Inf values
schema consistency
timestamp validity
joint limits

Hard structural failures can result in an EXCLUDE verdict.

Layer 2 — Temporal & Motion Quality

These checks look for statistical risk signals such as:

sensor synchronization
sampling jitter
velocity / acceleration anomalies
action discontinuity
temporal sufficiency

These are generally signals for investigation rather than proof that an episode is unusable.

Layer 3 — Dataset Utility

This includes:

idle ratio
distribution characteristics
state-space coverage

These metrics are particularly contextual.

For example, a high idle ratio may be perfectly reasonable for one task and problematic for another.

PASS, REVIEW, EXCLUDE — But Don't Confuse Them with Ground Truth

One design decision became particularly important during development.

RDA separates workflow verdicts from evidence levels.

The workflow verdict is:

PASS
REVIEW
EXCLUDE

The evidence level is:

HARD_FAIL
RISK_SIGNAL
UNVERIFIABLE

These are deliberately different concepts.

For example:

High action discontinuity → RISK_SIGNAL

does not mean:

The demonstration is corrupted.

Likewise:

Missing frames → HARD_FAIL

does not necessarily mean:

The entire upstream dataset is bad.

The purpose is to keep an audit tool from turning weak statistical evidence into overly confident labels.

4,959 Episodes: Almost Half Triggered a Review Signal

I ran RDA across 12 local dataset copies.

The overall result was:

Verdict	Episodes	Percentage
PASS	1,711	34.5%
REVIEW	2,473	49.9%
EXCLUDE	775	15.6%

These numbers are not a confusion matrix.

There was no independent human ground truth for this benchmark, so they cannot be used to claim precision or recall.

The interesting observation is simply:

A large fraction of the tested episodes contained signals worth investigating before blindly feeding them into training.

That alone is useful.

Finding #1: Idle Data Is Everywhere

The median idle ratio across the tested datasets ranged from:

20.8% to 93.3%.

Ten of the twelve datasets had median idle ratios above 65%.

That raises a surprisingly important question:

If a large fraction of your training data represents the robot doing very little, what exactly is the policy learning?

It might be useful temporal context.

Or it might simply teach the model that doing nothing is common.

The answer depends heavily on the task and model architecture.

This led to our first optimization experiment.

Finding #2: The Same Robot Can Have Very Different Data Characteristics

Two datasets from the same xArm platform showed a large difference.

xArm lift

Median idle ratio: 20.8%
767 / 800 episodes received PASS

xArm push

Median idle ratio: 83.3%
562 / 800 episodes received REVIEW

Same robot platform.

Different task.

The important lesson is:

Data quality is not completely independent of task structure.

A high idle ratio is not automatically a collection failure.

It may simply reflect how the task is performed.

That makes universal rules like:

"Delete every episode above 70% idle."

dangerous.

Finding #3: Action Discontinuity May Reveal the Controller

Another interesting pattern appeared in action discontinuity.

Some datasets showed spikes in essentially every episode.

Another xArm dataset had only a handful across 800 episodes.

For example:

simulated ALOHA datasets showed frequent spikes
the SO-100 dataset also showed frequent spikes
xArm lift had only 6 spike-containing episodes out of 800

This suggests that some data-quality signals may tell us as much about the collection or control stack as about the demonstrations themselves.

That could matter for:

smoothness-regularized policies
sim-to-real transfer
controller comparisons
teleoperation system evaluation

But again:

A discontinuity is a signal, not automatically a failure.

Finding #4: A Dataset Can Be Structurally Clean and Still Need Review

One of the most useful observations from the benchmark was the separation between integrity and behavior.

Outside of the intentionally corrupted ALOHA fixture, the tested datasets were largely clean at the basic integrity layer.

Yet the behavioral layer still flagged many episodes for review.

In other words:

"The files are valid" and "the demonstrations are ideal for training" are different questions.

This seems obvious after writing it down.

In practice, however, many pipelines stop at the first question.

Finding #5: Sometimes the Dataset Copy Is the Problem

One of the most interesting cases was LIBERO.

The local metadata declared:

1,693 episodes

But only:

920 episodes

could actually be read from the local Parquet files.

The remaining:

773 episodes

were missing from the local copy.

An earlier loader implementation could interpret this kind of metadata/data-layout mismatch as zero-frame episodes.

The loader was changed to scan the actual Parquet layout and build a fallback episode index.

The result was more honest:

The local copy is incomplete.

Not:

LIBERO contains 773 empty episodes.

This distinction matters.

A broken local copy is not necessarily a broken upstream dataset.

Then I Compared RDA with Other Tools

I also wanted to understand how different robot-data tooling behaves on deliberately corrupted data.

So I ran a small cross-tool benchmark involving:

RDA
trajlens
ORBIT
lerobot-doctor

The test included known injected defects.

One result looked like this:

Defect	RDA	trajlens	ORBIT	lerobot-doctor
NaN actions	EXCLUDE	WARN	Crash	FAIL
Timestamp reversal	EXCLUDE	FAIL	Crash	FAIL
Frozen actions	PASS	—	—	FAIL

The important result wasn't that RDA "won."

It didn't.

The frozen-action case exposed a real blind spot.

The Frozen-Segment Blind Spot

RDA's current idle_ratio metric is primarily episode-level.

That means a dataset can have a reasonable overall idle ratio while still containing a long locally frozen segment.

Another tool had a dedicated consecutive-identical-actions check and caught it.

This gives RDA a very concrete next feature:

frozen_segment

A good audit tool should be able to identify its own blind spots.

This one is now on our priority list.

Can Data Auditing Actually Improve Training?

Finding suspicious data is only half the problem.

The more interesting question is:

Can audit results tell us what to do with the dataset?

I ran several experiments around idle-data pruning.

The result was much more nuanced than I initially expected.

Moderate Pruning Helped — Aggressive Pruning Hurt

In one experiment:

Strategy	Data retained	MSE change
Baseline	100%	—
Keep 50%	50.1%	-28.9%
Keep 40%	40.1%	-12.5%
Keep 30%	30.1%	-13.3%
Keep 20%	20.0%	+11.5%
Keep 10%	10.0%	+98.2%

There was a clear dose-response pattern.

Moderate pruning helped in this experiment.

Aggressive pruning eventually became harmful.

So the conclusion is not:

"Remove idle data."

It is:

There may be a useful pruning region, but it depends on how the remaining data is used.

Then We Tested Different Model Architectures

This was probably the most interesting result.

We compared a frame-wise MLP with a temporal Transformer.

The result was almost the opposite.

MLP

Idle-data pruning:

-21.6% MSE

Temporal Transformer

Idle-data pruning:

+95.5% MSE

Why?

Because temporal models need context.

Suppose:

idle ratio ≈ 70%
active segments are scattered
sequence length = 10

After pruning, many active segments become shorter than the required temporal window.

The result:

You removed "low-value" frames and accidentally removed the context required to construct training samples.

There is another issue.

A temporal policy may need to learn the transition:

idle → movement

If all idle context is removed, the model may never see how movement starts.

The Practical Lesson

This changed how I think about automated dataset optimization.

A rule like:

idle_ratio > 70% → delete

is too simplistic.

The better question is:

What model are you training, and what temporal structure does that model need?

The current experiments suggest:

gentle trimming can be reasonable for frame-wise models
aggressive idle pruning can be harmful
temporal models need special handling
dataset/task domain matters
audit signals should not automatically become deletion rules

This is why RDA's recommendation layer takes the intended model type into account.

A Local-First Architecture

Robot datasets can contain commercially sensitive information.

They may reveal:

robot configurations
manipulation strategies
factory environments
camera observations
proprietary tasks
operator behavior

RDA's core audit therefore runs locally.

It can generate a blind report containing:

anonymized paths
dataset statistics
episode/frame counts
metrics
verdicts

without sending the raw trajectories or images.

The goal is simple:

Share evidence without sharing the dataset.

What RDA Does NOT Claim

This is probably the most important section.

The current experiments do not establish:

RDA precision
RDA recall
universally optimal thresholds
that every EXCLUDE episode should be deleted
that action discontinuity means corruption
that high idle ratio means useless data
that RDA improves training success rate
that RDA certifies compliance with any data-quality standard

The benchmark is based on real local dataset runs, but it does not have independent ground-truth labels.

So the next experiment is obvious:

Get real human or customer QC labels and compare them against RDA without exposing the RDA verdict first.

The Next Experiment: Blind Human Review

The proposed validation workflow is:

Select 100–500 episodes from one robot platform.
Generate anonymized samples.
Have reviewers independently label them:

KEEP
REVIEW
REMOVE
1. Do not show them RDA's verdict.
2. Preserve reviewer decisions and notes.
3. Reveal RDA results only after labeling.
4. Compare the two sets of evidence.

That would allow us to measure:

overlap between RDA HARD_FAIL and human REMOVE
overlap between RDA RISK_SIGNAL and human REVIEW
additional defects found by RDA
reviewer time per episode
review queue reduction
potential downstream training impact

Until that experiment is completed, these remain validation targets rather than marketing claims.

Try It

RDA is open source and currently supports LeRobot v2.1 and v3.0.

Install:

pip install robot-data-audit

Audit a dataset:

rda audit /path/to/lerobot/dataset -v

Generate a blind report:

rda audit /path/to/lerobot/dataset --blind --format json

Generate optimization recommendations:

rda recommend /path/to/dataset --policy frame-wise

or:

rda recommend /path/to/dataset --policy temporal

Launch the optional UI:

pip install robot-data-audit[ui]
rda ui

GitHub:
https://github.com/liesliy/rda

PyPI:
https://pypi.org/project/robot-data-audit/

What I Want to Test Next

The next step isn't another synthetic benchmark.

I want to test RDA against real robot data with independent QC labels.

In particular, I'm looking for:

real-world teleoperation datasets
datasets with existing QC labels
datasets containing known failure cases
datasets used for sim-to-real experiments
robot companies or research groups willing to run a local audit

The raw data does not need to leave your environment.

A small validation set — even 100–500 episodes from one robot and 1–3 tasks — would already be extremely useful.

If you work with robot manipulation data and are willing to challenge these results, I'd love to compare notes.

Final Takeaway

After auditing thousands of robot episodes, my conclusion is not that robot datasets are "bad."

It is more subtle:

Robot datasets contain many properties that are invisible to a simple "can I load this file?" check.

And those properties can matter differently depending on:

the robot
the task
the controller
the dataset format
the policy architecture
the intended use of the data

The goal of RDA is not to produce another mysterious quality score.

It is to make the path from:

measurement → diagnosis → human review → optimization

more reproducible.

And eventually, hopefully, to make robot data quality something we can discuss with evidence rather than intuition.

Before the GPU bill arrives, audit the data.

# Public Robot Dataset Health Check — as measured by RDA v0.5.2

liesliy — Wed, 19 Aug 2026 03:30:56 +0000

We ran RDA across 11 public LeRobot-format datasets — 4,909 episodes in total, spanning sim and real, scripted and human teleop, research arms and $100 hobby hardware. Summary:

Dataset	Type	Eps	Verdicts (P/R/E)	Action spikes	Idle median	Idle p5
aloha_sim_insertion_human	sim, human	50	5 / 45 / 0	1,338 (100% eps)	70.7%	65.1%
aloha_sim_transfer_cube_scripted	sim, scripted	50	5 / 45 / 0	2,489 (100% eps)	64.0%	44.2%
aloha_sim_insertion_scripted	sim, scripted	50	1 / 49 / 0	1,633 (100% eps)	63.7%	53.7%
droid_100	real Franka	100	34 / 66 / 0	1,428 (99% eps)	70.7%	55.7%
pusht	sim	206	43 / 163 / 0	1,148 (97% eps)	81.7%	59.5%
HuggingFaceVLA/libero	sim	1,693	0 / 3 / 1,690	34 (3 eps)	76.5% (3 eps)	74.6%
bridge_orig_lerobot (sampled)	real WidowX	25	4 / 21 / 0	91 (84% eps)	93.3%	20.5%
xarm_lift_medium	real xArm	800	767 / 33 / 0	6 (1% eps)	20.8%	12.5%
xarm_push_medium	real xArm	800	238 / 562 / 0	845 (62% eps)	83.3%	16.7%
svla_so101_pickplace	real SO-100	50	5 / 45 / 0	260 (100% eps)	86.7%	59.5%
jaco_play	real Jaco	1,085	390 / 695 / 0	11,958 (77% eps)	74.1%	52.7%

All integrity layers (NaN/Inf, timestamp validity, missing frames, schema) came back clean on all 11. libero note: 1,690 of 1,693 episodes read 0 frames — a dataset-side meta/layout mismatch, now flagged EXCLUDE instead of silently passing. bridge sampled 25 episodes; the other 10 datasets were audited in full. (aloha_sim_transfer_cube_human was audited earlier at v0.5.1 with matching results: 1/49/0, 1,535 spikes, 71.2% idle — it's now a gated repo, so we couldn't re-pull it.)

Five patterns worth knowing before you train

1. Median idle runs 20.8%–93.3%, and 8 of 11 datasets sit above 65%.

Loss functions trained on a 75%-idle distribution are structurally biased toward predicting "do nothing" unless you weight or curriculum it. Bridge data pushes it to 93%. Measure yours before the GPU bill, not after.

2. Same robot, same lab, four-fold idle difference.

xarm_lift_medium: 20.8% median idle, 767/800 episodes PASS. xarm_push_medium: 83.3% median idle, 562/800 REVIEW. Same xArm platform — the difference is task difficulty (lifting vs. pushing a flat object), not collection sloppiness. High idle isn't always a bug; it's a property you need to know and design around. RDA flags both sides of this honestly.

3. Action discontinuity tracks the controller, not the dataset's reputation.

Sim ALOHA and the SO-100 hobby setup spike in literally 100% of episodes; xArm lift data has 6 spikes across 800 episodes. If your policy uses smoothness regularization or you're doing sim-to-real action statistics, this number decides your curriculum.

4. Clean integrity ≠ good training data.

Integrity passed 11/11 — zero NaNs, zero timestamp reversals, zero missing frames anywhere. The behavior layer still flagged 45–98% of episodes for review in most datasets. Both layers matter; most pipelines check neither.

5. Cheap hardware produces the most expensive data.

The community SO-100 pick-place set: 86.7% median idle plus spikes in every episode. If you're fine-tuning on hobby-robot uploads, this is what you're inheriting.

Reproduce

pip install robot-data-audit
rda audit <any lerobot dataset> -v
rda recommend <dataset> --policy temporal   # or frame-wise, --lang en

Tool: https://github.com/liesliy/rda · PyPI: robot-data-audit · UI: rda ui (EN/中文)

Caveats: RDA flags statistical anomalies, not ground-truth errors. REVIEW means "look before you train," not "discard." All thresholds are open for debate — that's what the issue tracker is for.

I Open-Sourced a Data Quality Auditor for Robot Datasets — It Found 1,535 Action Spikes in the Official ALOHA Demo Set

liesliy — Wed, 19 Aug 2026 02:55:47 +0000

Everyone in embodied AI says "data is the new code." Nobody audits it like code.

We run linters, static analyzers, and CI gates on our source code. Then we feed 50GB of teleoperation recordings into a policy network and hope for the best. So I built RDA (Robot Data Audit) — an open-source CLI that treats robot datasets the way ruff treats a Python repo.

pip install robot-data-audit
rda audit /path/to/lerobot/dataset

It works natively on LeRobot-format datasets (v2.1 + v3.0), checks every episode across two layers — integrity (NaN actions, timestamp reversals, missing frames) and behavior (action discontinuity, idle ratio, frozen segments) — and outputs a per-episode verdict: PASS / REVIEW / EXCLUDE.

The dogfooding surprise

Before releasing it, I ran RDA against the datasets everyone treats as ground truth — starting with aloha_sim_transfer_cube_human, the official ALOHA simulation demonstration set that ships with the LeRobot ecosystem.

Result from the behavior layer, across 50 episodes:

Action discontinuity spikes: 1,535 (~30 per episode)
Median idle ratio: 0.71** — the arm is effectively stationary 71% of frames
Median effective motion ratio: 0.29

To be clear about what this does and doesn't mean: none of this is corruption. The integrity layer came back clean — no NaNs, no broken timestamps. The spikes are real discontinuities in the action space (large frame-to-frame joint jumps), and the high idle ratio likely reflects grasping/hover phases where the gripper holds still. Neither is necessarily a bug in the dataset.

But that's exactly the point. If you're benchmarking a policy on this data, or worse, fine-tuning on it, these numbers are context you didn't have. Is 30 spikes per episode normal for this task? Does 71% idle time skew your loss toward predicting "do nothing"? Nobody asks, because nobody measures.

The bug I found in my own tool (while writing this post)

Honesty section, because dev.to deserves better than marketing:

My first run reported all 50 episodes as PASS. Green across the board. Celebration ensued.

Then I cross-checked the behavior layer output against the verdicts and realized they weren't connected — the metrics were computing 1,535 spikes, and the verdict aggregator was ignoring behavior signals entirely. The tool had the evidence and wasn't reading it. The loudest silence in software is a metric that's computed but never consumed.

Fixed now: behavior signals feed the verdict through a dataset-utility layer, and metric-level findings carry human-readable reasons. The ALOHA run now correctly flags 49/50 episodes as REVIEW with the specific signals attached.

If you're building anything with a "signal producer → decision aggregator" architecture, test the wiring, not just the signals. I wrote a negative-control test for it before I trusted my own tool again.

Why this matters more than it sounds

Robot learning teams are drowning in data collection — teleop sessions, sim rollouts, fleet logs — with almost no tooling for "is this batch usable before I burn GPU hours on it." An episode with a frozen sensor or a corrupted timestamp doesn't fail loudly. It trains quietly.

RDA's philosophy: audit before train. Cheap checks first (seconds per episode, pure numpy/pandas), verdicts you can gate in CI:

rda audit ./my_dataset --format json -o report.json
# fail the pipeline if any episode comes back EXCLUDE

There's also a built-in UI (rda ui) for browsing verdicts without spelunking JSON — and as of v0.5.2 it's fully bilingual: one toggle switches the entire dashboard, backend recommendation copy included, between English and 中文.

What shipped since the first post

rda recommend — model-aware optimization advice. Tell it whether you're training a frame-wise model (MLP/BC) or a temporal one (ACT/Diffusion Policy), and it gives different answers for the same data — including an explicit DO_NOT_PRUNE guard for temporal models when valid-window ratio collapses. Every suggestion carries its experimental evidence: pruning cost our seq=10 temporal baseline +296% MSE, while trimmed ALOHA/PushT improved frame-wise baselines by 11–35%.
Privacy-first architecture — metrics compute locally; only <1KB of aggregates reach the rules API. rda audit stays 100% offline, always.
LeRobot v2.1 support — bridge-style layouts now load natively (first run on bridge data: median idle ratio 93.3%. Real robots spend a lot of time deciding.).
A bug class worth naming: silent PASS. 1,690 zero-frame episodes in a popular dataset were passing because "no evidence of problems" was treated as "no problems." Now zero-frame episodes are explicit EXCLUDEs with a diagnosis.

What's next

More behavior metrics (jitter, cycle anomalies, calibration drift)
Trend dashboards across successive audits (already in the UI's History page — feedback wanted)
Export-to-clean: one-click filtered dataset copy from surviving episodes

The project is early and hungry for real-world datasets to chew on. If you have a LeRobot-format dataset (v2.1 or v3.0), run rda audit on it and tell me what turns up — especially if it's boring. Boring results from real data are how a tool earns trust.

Issues, PRs, and "your idle-ratio threshold is wrong, here's why" comments all welcome.

(RDA is MIT-licensed. I also do paid data-quality deep dives and pipeline integration for teams that want the audit without the homework.)

Tool: https://github.com/liesliy/rda · PyPI: robot-data-audit · UI: rda ui (EN/中文)

tlabel convert: One CLI to Bridge 9 Tactile Dataset Formats

liesliy — Tue, 11 Aug 2026 04:08:43 +0000

How a single command can unify GelSight, PaXini, Daimon, ToucHD, and 5 other tactile sensor formats into training-ready data.

If you work in tactile robotics research, you've been here before:
A collaborator sends you a dataset collected with a PaXini PXCap force array. Your pipeline expects GelSight .pkl files. Your colleague's LeRobot training code needs Zarr. Someone else is publishing results on a Daimon DM-TacClaw in .parquet format.

Three sensors. Three formats. Three days of writing ad-hoc parsing scripts that you'll delete next week.

Tactile sensing is having a moment — multiple billion-dollar funding rounds in embodied AI have poured attention (and capital) into the field in 2026. But while hardware is advancing fast, data interoperability is still a mess. Every sensor vendor ships data in a proprietary format, and there's no common lingua franca for tactile manipulation datasets.

TLabel is an open-source project that tackles exactly this problem. And with the maturation of its CLI and adapter architecture through v0.18.x, the workflow for converting between tactile dataset formats has gotten dramatically simpler.

What TLabel Actually Does
Before diving into commands, let's set expectations. TLabel is a data pipeline standardization layer. It does not:

Interface with hardware or collect data
Run inference or train models
Replace your training pipeline

It does:

Define a 14-dimension semantic annotation schema (covering contact, force, slip, texture, deformation, and more)
Provide adapter implementations that translate sensor-specific formats into that schema
Export annotated data into training-ready formats (LeRobot, FTP-1 Zarr, JSON, CSV)
Think of it as the Unicode for tactile data — one standard schema, every sensor.

The Adapter Landscape
TLabel currently ships with 12 built-in adapters covering 9 dataset formats and 3 real-time sensor interfaces:

Dataset Adapters (Offline Data Loading)

<> — GelSight Mini / DIGIT, visuo-tactile, .pkl, L3
<> — PaXini PXCap, force array, .h5, L2
<> — Daimon DM-TacClaw, multimodal, .parquet, L3
<> — ToucHD, visuo-tactile, .hdf5, L3
<> — UniVTAC, visuo-tactile, .hdf5, L3
<> — VTouch, visuo-tactile, .h5, L3
<> — YCB-Slide, visuo-tactile, .npy, L3
<> — TacQuad (AnyTouch), multi-sensor, directory, L3
<> — TLabel native, meta format, .json, L1–L4

Real-Time Sensor Adapters

<> — PaXini GEN3, force array, SDK connection, L2
<> — Daimon DM-Tac, visuo-tactile, USB / .avi, L3
<> — PaXini PX6D, 6-axis force, placeholder, L2
That covers the majority of tactile sensors used in manipulation research today.

Getting Started
Install is straightforward:

pip install tlabel

# Or with sensor-specific extras:
pip install tlabel[gelsight]     # GelSight / DIGIT (.pkl)
pip install tlabel[paxini]       # PaXini PXCap (.h5)
pip install tlabel[daimon]       # Daimon DM-TacClaw (.parquet)
pip install tlabel[ftp1]         # FTP-1 export (zarr)
pip install tlabel[all]          # Everything

Exploring Adapters from the CLI
Once installed, you can inspect what's available without writing any code:

# List all registered adapters
tlabel list

This prints all dataset and real-time adapters with their type, native format, and compliance level. It's the first command I run when working with a new dataset.

For details on a specific adapter:

tlabel info gelsight

This shows the adapter's capability declaration — which of the 14 semantic dimensions it can annotate, its compliance level, and any format-specific notes. For example, GelSight outputs force_vector (L3) but not temperature (L4), while a simple resistive sensor might only declare L1 fields like contact and slip_event.
This capability declaration system is one of TLabel's key design decisions. Rather than forcing every sensor to produce all 14 dimensions (which would mean fabricating data it can't actually measure), each adapter honestly declares what it can and cannot provide.

Validating Your Data
Before converting, it's worth checking that your data passes schema validation:

tlabel validate data.json

This runs a compliance check against the 14-dimension Schema V2 and reports any issues. Catching format problems early saves debugging time downstream.

Converting Between Formats
Here's where things get practical. TLabel provides two paths for format conversion: CLI for quick checks and the Python API for full control.

Quick Conversion via CLI
For simple cases — say you have a single GelSight .pkl file and want a JSON summary:

tlabel export --input grasp_data.pkl --format json --output annotations.json

This reads the GelSight data through its adapter, applies the Schema V2 annotation, and writes a clean JSON file with all 14 semantic dimensions (at the appropriate compliance level for the sensor).

Full Conversion Pipeline via Python
For the heavy lifting — converting entire datasets into training-ready formats — the Python API gives you the most control:

import tlabel
from tlabel.converters import tlabel_to_lerobot

# Load data from any supported sensor
data = tlabel.load("path/to/paxini_data.h5")

# Inspect what you got
print(data.describe())
# -> {'num_frames': 500, 'sensor': 'paxini', 'compliance_level': 'L2', ...}

# Export to JSON/CSV for analysis
data.export("output.json")

# Export to FTP-1 Zarr for foundation model training
data.export_ftp1("output.zarr")

# Convert to LeRobot episode format
tlabel_to_lerobot("annotations.json", "lerobot_episode/")

Here's a concrete example that walks through a realistic workflow — loading PaXini force array data, validating annotations, and exporting to both LeRobot and FTP-1 formats:

import tlabel
from tlabel.converters import tlabel_to_lerobot

# Step 1: Load PaXini PXCap data
data = tlabel.load("experiment_01.h5")

# Step 2: Validate schema compliance
data.validate_annotations()
# Reports any missing or malformed fields

# Step 3: Auto-annotate events from signal patterns
data.annotate_events_auto()
# Detects: contact_onset, contact_loss, slip events, force spikes

# Step 4: Export
data.export("experiment_01.json")           # Analysis
data.export_ftp1("experiment_01.zarr")      # Foundation model training
tlabel_to_lerobot("experiment_01.json",     # LeRobot pipeline
                   "lerobot_episode/")

Batch Processing Multiple Files
When you're dealing with an entire experiment directory (which is the typical case — real manipulation datasets have hundreds of episodes), you can batch-load and convert:

import glob
import tlabel

files = glob.glob("raw_data/*.h5")

for f in files:
    data = tlabel.load(f)
    data.validate_annotations()
    data.export(f"annotated/{data.sensor}_{data.num_frames}f.json")
    print(f"Converted {f} -> {data.sensor} (L{data.compliance_level})")

Understanding the Architecture
The adapter system sits in a three-layer architecture:

┌─────────────────────────────────────────────────┐
│ Layer 1: Schema                                 │
│ 14 semantic dimensions + Compliance Level L1-L4 │
├─────────────────────────────────────────────────┤
│ Layer 2: Adapters                               │
│ DataAdapterBase │ SensorAdapterBase             │
├─────────────────────────────────────────────────┤
│ Layer 3: Downstream                             │
│ Feature derivation · Export · Augmentation      │
│ FTP-1 · LeRobot · RLDS · ROS2                  │
└─────────────────────────────────────────────────┘

Layer 1 is the schema itself — 14 semantic dimensions covering spatial perception (contact, centroid, region), mechanics (force magnitude, force vector, torque), dynamics (slip event, slip velocity, manipulation phase), surface properties (texture class), and meta-perceptions (deformation, temperature, confidence, compliance level).
Layer 2 is where adapters live. There are two base classes:

DataAdapterBase — for offline dataset files (sublcass this to add support for a new sensor format, takes ~30 minutes)
SensorAdapterBase — for real-time hardware connections (streaming data from a live sensor)

Both produce output conforming to the same Schema V2, which means Layer 3 downstream tools work identically regardless of which sensor the data came from.

Layer 3 handles everything after annotation: feature derivation, data augmentation, and export into training frameworks. This is where the LeRobot converter, FTP-1 Zarr exporter, and RLDS bridge live.

The Compliance Level System
One concept worth explaining in more detail: Compliance Levels (L1–L4). This is TLabel's answer to the question "what if my sensor can't measure temperature or 6-axis force?"

L1 — Basic Tactile: contact, centroid, slip, confidence. Examples: Single-point resistive, proximity sensors
L2 — Force-Aware: L1 + force_magnitude. Examples: PaXini, YCB-Slide, GelSight
L3 — Full-Vector: L2 + force_vector. Examples: ToucHD, calibrated DM-TAC
L4 — Rich-Semantic: L3 + all optional fields. Examples: BioTac, next-gen multimodal sensors

The key insight: a PaXini at L2 and a ToucHD at L3 both produce valid TLabel output. They just populate different subsets of the 14 dimensions. Downstream code can check compliance_level to decide what it can and cannot use, rather than writing sensor-specific branches.
This is what makes cross-sensor comparison possible. You can train a model on GelSight data (L3) and evaluate it on PaXini data (L2), knowing exactly which fields are comparable and which aren't.

What About My Sensor?
TLabel is designed for extensibility. If your sensor isn't supported yet, adding an adapter takes about 30 minutes:

Fork the adapter template from contrib/adapter-template/
Subclass DataAdapterBase (for datasets) or SensorAdapterBase (for hardware)
Implement the required methods and declare your compliance level
Submit a PR or publish as a standalone package

The project also supports register_external_adapter() and entry_points auto-discovery, so third-party adapters can be published independently.

How This Fits Into the Embodied AI Pipeline
The broader context matters. Foundation models for robotics — like those being built on top of LeRobot, Open X-Embodiment, and similar frameworks — need diverse, standardized training data. But tactile data has been a bottleneck:

Collection is sensor-specific (hardware-dependent)
Annotation has been ad-hoc (no standard schema)
Training expects uniform input formats

TLabel deliberately addresses step 2 and 3 only. It's a standardization layer that sits between your raw sensor data and your training pipeline. By providing a common schema with honest capability declarations, it enables:

Cross-sensor training: Mix data from different sensors in the same training batch
Capability-aware models: Train models that know what information is available at each compliance level
Reproducible research: Compare results across labs using different hardware

The project is also actively contributing tactile data format support upstream to LeRobot via PR #4032, which signals growing recognition that tactile data standards are needed in the broader robotics ecosystem.

Quick Reference

See all supported adapters → tlabel list
Get adapter details → tlabel info gelsight
Validate a data file → tlabel validate data.json
Export to JSON → data.export("out.json")
Export to FTP-1 Zarr → data.export_ftp1("out.zarr")
Convert to LeRobot format → tlabel_to_lerobot(src, dst)
Load any sensor data → tlabel.load("file")
Try a demo (no files needed) → tlabel.demo("gelsight")

Wrapping Up
Tactile data interoperability isn't glamorous work — it's infrastructure. But it's the kind of infrastructure that determines whether the field can scale beyond lab-specific pipelines to shared, reproducible, cross-sensor research.
TLabel's adapter architecture and CLI tools won't solve every data problem in tactile robotics. But they provide a concrete, working answer to the question: "How do I get data from sensor X into format Y without writing a custom parser?"
That question used to take a weekend. Now it takes one line.

Links:
GitHub: https://github.com/liesliy/tlabel
PyPI: https://pypi.org/project/tlabel/
TLabel Paper: PDF on GitHub
Schema V2 Spec: docs/tlabel-format.md
Adapter Template: contrib/adapter-template
Contributing Guide: CONTRIBUTING.md
LeRobot PR #4032: huggingface/lerobot#4032
Previous Dev.to post: TLabel: Unifying Tactile Data Annotation for Robotics
Open X-Embodiment: robotics-transformer-x.github.io
OpenTouch: opentouch.ai

TL;DR — TLabel is the Unicode for tactile data: one standard schema, every sensor. Install with pip install tlabel, run tlabel list to see what's supported.

How to Convert Multi-Sensor Tactile Data to LeRobot Format in One Command

liesliy — Thu, 06 Aug 2026 09:27:10 +0000

Stop writing custom data parsing scripts for every new tactile sensor. Here's a CLI tool that handles 9 sensor types and outputs training-ready LeRobot datasets.

The Problem
If you're working on robot manipulation with tactile sensing, you've hit this wall:

Every tactile sensor outputs data in a different format. GelSight gives you .pkl files. PaXini uses .h5. Daimon uses .parquet. ToucHD dumps raw .npy arrays. And your robot learning framework — whether it's LeRobot, FTP-1, or something else — expects one specific format.
So you end up writing a custom parser for each sensor. Every time you add a new sensor to your setup, you spend a day (or two) wrestling with data format conversions instead of doing actual research.
I was in the same situation. That's why I built TLabel Convert.

What TLabel Convert Does
TLabel Convert is a CLI tool that converts raw data from 9 different tactile sensors into standardized training formats:

# Convert a single file
tlabel convert --from gelsight --to lerobot --input data.pkl --output output_dir/

# Convert an entire directory
tlabel batch-convert --from univtac --to ftp1 --input-dir ./raw/ --output-dir ./converted/

That's it. Two commands. No custom parsing code.

Supported Sensors
Here's what you can convert from:

GelSight Mini / DIGIT — Vision-based tactile, .pkl input → Contact, force, optical flow, 23 dimensions
PaXini PXCap — High-res tactile array, .h5 input → Force, deformation, temporal features, 20 dimensions
Daimon DM-TacClaw — Multi-finger, .parquet input → LeRobot-compatible, force + contact
ToucHD / AnyTouch 2 — Multi-modal, .npy + directory → Force + tactile + temporal, 22 dimensions
UniVTAC — Cross-dataset, .hdf5 input → Universal cross-dataset format
VTouch — Vision-based, .h5 input → Visual tactile features
YCB-Slide (CMU DIGIT) — Sliding manipulation, .npy + directory → Sliding-specific features
TacQuad (AnyTouch) — Multi-sensor array, .csv + directory → Multi-array data
TLabel Format — Universal, .json input → Any data already in TLabel schema

Supported Output Formats

Two training-ready formats:
LeRobot Format (HuggingFace)

tlabel convert --from gelsight --to lerobot --input data.pkl --output lerobot_dataset/

Output structure:

lerobot_dataset/
├── data/
│   └── chunk-0000.parquet    # Standard LeRobot parquet
└── meta/
    └── info.json              # With observation.tactile schema

The output is directly compatible with LeRobotDataset:

from lerobot.common.datasets.lerobot_dataset import LeRobotDataset
dataset = LeRobotDataset("lerobot_dataset")
print(f"Episodes: {dataset.num_episodes}")
print(f"Tactile dims: {dataset.features['observation.tactile']}")  # 14-dim Schema V2

FTP-1/MTTS Format (Zarr)

tlabel convert --from paxini --to ftp1 --input data.h5 --output ftp1_dataset/

For tactile foundation model pretraining.

Real Example: GelSight → LeRobot
Let me walk through a complete example. Say you have GelSight Mini data and want to use it with LeRobot:

# Step 1: Install
pip install tlabel>=0.19.0

# Step 2: Convert
tlabel convert \
  --from gelsight \
  --to lerobot \
  --input gelsight_recording.pkl \
  --output my_lerobot_dataset/

# Step 3: Verify
tlabel adapter-info gelsight

The converter handles:
1.Frame extraction from the .pkl format
2.Tactile feature mapping (23 dimensions → 14-dim TLabel Schema V2)
3.Proper LeRobot parquet schema
4.Metadata generation (info.json with observation.tactile field)

What About My Sensor?
If your sensor isn't in the list, you can check what's available:

# List all supported adapters
tlabel list-adapters

# Get detailed info about a specific adapter
tlabel adapter-info paxini

The adapter architecture is extensible — if you're a sensor manufacturer or researcher, you can implement a new DataAdapter by subclassing DataAdapterBase. Check the contributing guide for details.

The Bigger Picture: TLabel Schema
TLabel Convert is part of the TLabel project, which defines a unified tactile data annotation standard.
The schema covers 14 semantic dimensions:
1.Contact state & area
2.Force magnitude & direction
3.Slip events & entropy
4.Deformation magnitude & rate
5.Optical flow direction & magnitude
6.Texture energy
7.And more...

Each dimension has a compliance level (L1-L4) indicating how much information is available. This means you can have consistent annotations across different sensors, even if they don't all measure the same things.

Quick Reference

# Install
pip install tlabel>=0.19.0

# Convert single file
tlabel convert --from <adapter> --to <format> --input <path> --output <path>

# Batch convert directory
tlabel batch-convert --from <adapter> --to <format> --input-dir <dir> --output-dir <dir>

# List supported adapters
tlabel list-adapters

# Adapter details
tlabel adapter-info <name>

Adapters (--from): gelsight, paxini, daimon, tlabel, touchd, univtac, vtouch, ycb_slide, tacquad
Formats (--to): lerobot, ftp1

Links
GitHub: https://github.com/liesliy/tlabel
PyPI: https://pypi.org/project/tlabel/
HuggingFace Dataset: https://huggingface.co/datasets/375720783jin/tlabel-convert

If you're working with tactile data and struggling with format conversions, give it a try. Issues and PRs welcome.

Your AI Agent Can See, Hear, and Talk. But Can It Feel?

liesliy — Wed, 05 Aug 2026 02:39:23 +0000

Three years ago, I couldn't have imagined building software for something you can't see, hear, or photograph.

Today, I spend my days thinking about how robots feel — and more specifically, how we make sense of the data their skin produces.

If you've been following the AI agent space lately (and honestly, who hasn't), you've probably noticed a pattern: everyone's building agents that can see (vision), hear (audio), and talk (LLM outputs). But almost nobody's asking: what about touch?

The "Babel Problem" Nobody Talks About
Here's what I found when I started working on embodied AI data:
Every robotics lab on the planet stores tactile data differently. Different column names. Different units. Different sampling rates. Different everything.

Lab A records force in Newtons, columns named fx, fy, fz
Lab B uses raw ADC counts, columns named ch_0, ch_1...
Lab C has its own JSON schema with nested objects
Robot D has a completely different coordinate system

Try to train a model across these datasets. Try to benchmark. Try to share data with a collaborator at another university. Good luck.

I call this the Babel Problem. Everyone's speaking their own language, and nobody can understand each other.
"But We Have MCP Now, Right?"
Great question. MCP standardized how LLMs call tools. A2A handles agent-to-agent communication. OpenTelemetry covers observability.

None of these touch tactile perception.
There is no standard for what a "tactile reading" actually means. Not its semantics. Not its structure. Not its units. Not how you describe contact geometry, pressure distribution, slip events, or texture features.

Think of it this way: if someone published a recipe using inches, Fahrenheit, and a custom unit called "glugs," you couldn't reproduce it — no matter how brilliant the cooking was. That's where tactile data is right now.

Why This Matters Now
Embodied AI is having its moment. Foundation models are being trained on robot demonstrations. Dexterous manipulation is no longer sci-fi. But the data pipeline underneath all of this is held together with tape.

Without a standard:

Cross-dataset training is basically impossible (you spend more time normalizing than modeling)
Hardware swaps break every downstream pipeline
Reproducibility is a joke (good luck replicating someone's tactile benchmark)
Data sharing between labs requires a custom converter... per dataset

What We Built
After months of frustration, I started building TLabel — an open standard for tactile data.
The core idea is simple: define a universal schema that describes what tactile data means, regardless of which sensor produced it.

TLabel defines 14 semantic dimensions, including:

Contact Geometry — where and how the contact happens
Normal / Shear Force — force direction and magnitude
Pressure Distribution — spatial pressure map
Texture — surface micro-features
Temperature — thermal perception
Slip — detection of movement at contact
...and 8 more (full spec in the design doc)

Each data file carries a Compliance Level (L1–L4), so consumers know exactly what's available — even if some sensors only provide basic readings.

The key design principle: TLabel doesn't tell you how to process tactile data. It tells you how to describe it. The processing is up to you.

Show Me The Code
Load a tactile file — same API regardless of which sensor produced it:

from tlabel import TLabelFile
data = TLabelFile("sample.tlabel")

Access standardized dimensions:

forces = data.get_forces()             # Newtons, always
contacts = data.get_contact_geom()     # Consistent coordinate frame
textures = data.get_texture_features()

Check what's available and inspect sensor metadata:

print(data.compliance_level)   # L1 = basic, L4 = full spec
print(data.sensor_info)        # Original hardware metadata preserved

No matter whether the original data came from a GelSight, a BioTac, a custom capacitive array, or a future sensor nobody's invented yet — the interface is the same.

The Python package is at v0.18.2 on PyPI, and we're actively building exporter plugins for popular formats.

The open-source embodied AI ecosystem is moving fast — LeRobot, Forge, various simulators and hardware platforms. But if these systems can't share and understand tactile data, we're building a tower with no common language.

I don't think TLabel is the final answer. But the conversation needs to happen, and the standard needs to be open — not owned by one company or one lab.
The project is here if you're curious: github.com/liesliy/tlabel

Hello, Dev.to — I'm Building a Data Format Nobody Asked For (But Everyone Needs)

liesliy — Sat, 01 Aug 2026 03:04:41 +0000

Hey 👋 I'm Liesliy, a developer working on tactile data infrastructure for robots.
The Problem That Brought Me Here
If you work in robotics, you've probably noticed something: nobody can agree on how to label tactile data.
Every lab has its own format. Every dataset ships in a different schema. When you try to combine data from two sources, you spend more time writing converters than doing actual research.
I've been there. So I built TLabel — an open-source unified format standard for tactile annotation data.

Think of it like this:
ROS standardized how robots communicate
ROS Bag standardized how robots record
Nobody has standardized how we label tactile data

That's the gap TLabel tries to fill.

What It Actually Does
Unified schema with 14 semantic dimensions and 4 compliance levels (L1–L4)
Adapters to convert between formats (GelSight, BioTac, digit, TacTip, and more)
PyPI-installable, plug into your existing pipeline
It's early — v0.17 just shipped — but the architecture is solid and I'm actively looking for real-world feedback.

Why Dev.to
I've been reading Dev.to for a while. What I appreciate about this place:
Engineers write for engineers — no fluff, no engagement farming
"Hello World" culture actually works — people genuinely welcome newcomers
The comment sections are better than most conferences
I'm here to:
Share what I learn building a data standard nobody asked for (yet)
Meet people working on sensor data, robotics, and embodied AI
Get honest feedback — the kind that hurts but makes the project better

What I'd Love to Hear From You
If you work with tactile sensing, robotics data pipelines, or open-source tooling — I want to talk.
Especially curious:
How do you handle format conversion in your current workflow?
What's the most painful part of working with tactile datasets?

Drop a comment or DM. I promise I'm more interested in listening than pitching. 🦞