[memo]Droid: A large-scale in-the-wild robot manipulation dataset

Abstract

Propose distributed robot dataset

Training policies

Introduction

• 6 tasks in 4 locations, labs, offices, real households
• Training manipulation requires robot manipulation data with recorded manipulation datasets
• Collecting data at scale

Related works

Datasets in the other ML fields

• ImageNet, Kitti, Ego4D, Laion = CV
• Common crawl, The pile = NLP

Robot learning datasets

• RH20T: 33 tasks in 7 table-top scene
• BridgeV2 data in 24 scenes

DROID contains 564 scenes across 52 buildings

DROID Data collection

Difficulty: reproducible robot control access

Labeling: tasq.ai

DROID datasets environment

Viewpoint diversity: 1417 cameras, いろんな方向からカメラを設置して，ロボットの情報を取得している

Interaction location diversity

Experimental setup

• Closing waffle maker
• place chips on plate
• Put apple in pot
• Toasting
• Clean up desk
• Cook lentils

Out-of-distribution

• DROID (7k, 20 scenes)
• DROID (7k diverse scenes)

Conclusion