Grabette: an open system to record robot-manipulation data

Hugging Face introduced Grabette, an open system designed to simplify the collection of robot-manipulation data by using a handheld gripper, a camera, and a browser-based processing pipeline. The tool allows users to record tasks in minutes without requiring a robot, addressing the scarcity of large, diverse real-world manipulation datasets needed for training robot policies. The system captures demonstrations through a fisheye camera and an RGBD camera, reconstructing 6-DoF trajectories for robot learning.

Grabette is inspired by Stanford’s Universal Manipulation Interface (UMI) but aims to make the process more accessible by eliminating the need for specialized teleoperation rigs or lab setups. Users can build the device from off-the-shelf components, including a Raspberry Pi, standard Pi camera, OAK-D depth camera, and magnetic encoders. The recorded data is saved locally and processed in the browser, producing a robot-ready dataset compatible with LeRobot and shareable via the Hugging Face Hub.

The release includes a complete example to demonstrate the end-to-end workflow, though the primary focus is on the recording system itself. The data format is robot-agnostic, storing camera-local 6-DoF poses and gripper states, allowing the same dataset to train different robots or learning methods. The system synchronizes observation, tracking, and gripper data using a shared clock, ensuring accurate episode capture for training.

Hugging Face encourages community participation to expand the open dataset by contributing recorded tasks. The company plans to evolve Grabette further, including future additions like Casquette, a head-mounted device for egocentric capture. The project’s success depends on widespread adoption, with the goal of reducing barriers to robot learning by making data collection effortless and collaborative.