How to Convert Multi-Sensor Tactile Data to LeRobot Format in One Command

#ai #python #opensource #robotics

Stop writing custom data parsing scripts for every new tactile sensor. Here's a CLI tool that handles 9 sensor types and outputs training-ready LeRobot datasets.

The Problem
If you're working on robot manipulation with tactile sensing, you've hit this wall:

Every tactile sensor outputs data in a different format. GelSight gives you .pkl files. PaXini uses .h5. Daimon uses .parquet. ToucHD dumps raw .npy arrays. And your robot learning framework — whether it's LeRobot, FTP-1, or something else — expects one specific format.
So you end up writing a custom parser for each sensor. Every time you add a new sensor to your setup, you spend a day (or two) wrestling with data format conversions instead of doing actual research.
I was in the same situation. That's why I built TLabel Convert.

What TLabel Convert Does
TLabel Convert is a CLI tool that converts raw data from 9 different tactile sensors into standardized training formats:

# Convert a single file
tlabel convert --from gelsight --to lerobot --input data.pkl --output output_dir/

# Convert an entire directory
tlabel batch-convert --from univtac --to ftp1 --input-dir ./raw/ --output-dir ./converted/

That's it. Two commands. No custom parsing code.

Supported Sensors
Here's what you can convert from:

GelSight Mini / DIGIT — Vision-based tactile, .pkl input → Contact, force, optical flow, 23 dimensions
PaXini PXCap — High-res tactile array, .h5 input → Force, deformation, temporal features, 20 dimensions
Daimon DM-TacClaw — Multi-finger, .parquet input → LeRobot-compatible, force + contact
ToucHD / AnyTouch 2 — Multi-modal, .npy + directory → Force + tactile + temporal, 22 dimensions
UniVTAC — Cross-dataset, .hdf5 input → Universal cross-dataset format
VTouch — Vision-based, .h5 input → Visual tactile features
YCB-Slide (CMU DIGIT) — Sliding manipulation, .npy + directory → Sliding-specific features
TacQuad (AnyTouch) — Multi-sensor array, .csv + directory → Multi-array data
TLabel Format — Universal, .json input → Any data already in TLabel schema

Supported Output Formats

Two training-ready formats:
LeRobot Format (HuggingFace)

tlabel convert --from gelsight --to lerobot --input data.pkl --output lerobot_dataset/

Output structure:

lerobot_dataset/
├── data/
│   └── chunk-0000.parquet    # Standard LeRobot parquet
└── meta/
    └── info.json              # With observation.tactile schema

The output is directly compatible with LeRobotDataset:

from lerobot.common.datasets.lerobot_dataset import LeRobotDataset
dataset = LeRobotDataset("lerobot_dataset")
print(f"Episodes: {dataset.num_episodes}")
print(f"Tactile dims: {dataset.features['observation.tactile']}")  # 14-dim Schema V2

FTP-1/MTTS Format (Zarr)

tlabel convert --from paxini --to ftp1 --input data.h5 --output ftp1_dataset/

For tactile foundation model pretraining.

Real Example: GelSight → LeRobot
Let me walk through a complete example. Say you have GelSight Mini data and want to use it with LeRobot:

# Step 1: Install
pip install tlabel>=0.19.0

# Step 2: Convert
tlabel convert \
  --from gelsight \
  --to lerobot \
  --input gelsight_recording.pkl \
  --output my_lerobot_dataset/

# Step 3: Verify
tlabel adapter-info gelsight

The converter handles:
1.Frame extraction from the .pkl format
2.Tactile feature mapping (23 dimensions → 14-dim TLabel Schema V2)
3.Proper LeRobot parquet schema
4.Metadata generation (info.json with observation.tactile field)

What About My Sensor?
If your sensor isn't in the list, you can check what's available:

# List all supported adapters
tlabel list-adapters

# Get detailed info about a specific adapter
tlabel adapter-info paxini

The adapter architecture is extensible — if you're a sensor manufacturer or researcher, you can implement a new DataAdapter by subclassing DataAdapterBase. Check the contributing guide for details.

The Bigger Picture: TLabel Schema
TLabel Convert is part of the TLabel project, which defines a unified tactile data annotation standard.
The schema covers 14 semantic dimensions:
1.Contact state & area
2.Force magnitude & direction
3.Slip events & entropy
4.Deformation magnitude & rate
5.Optical flow direction & magnitude
6.Texture energy
7.And more...

Each dimension has a compliance level (L1-L4) indicating how much information is available. This means you can have consistent annotations across different sensors, even if they don't all measure the same things.

Quick Reference

# Install
pip install tlabel>=0.19.0

# Convert single file
tlabel convert --from <adapter> --to <format> --input <path> --output <path>

# Batch convert directory
tlabel batch-convert --from <adapter> --to <format> --input-dir <dir> --output-dir <dir>

# List supported adapters
tlabel list-adapters

# Adapter details
tlabel adapter-info <name>