Overview of ARIO Dataset
Embodied AI is transforming how AI systems interact with the physical world, yet existing datasets are inadequate for developing versatile, general-purpose agents. These limitations include a lack of standardized formats, insufficient data diversity, and inadequate data volume. To address these issues, we introduce ARIO (All Robots In One), a new data standard that enhances existing datasets by offering a unified data format, comprehensive sensory modalities, and a combination of real-world and simulated data. ARIO aims to improve the training of embodied AI agents, increasing their robustness and adaptability across various tasks and environments. Building upon the proposed new standard, we present a large-scale unified ARIO dataset, comprising approximately 3 million episodes collected from 258 series and 321,064 tasks. The ARIO standard and dataset represent a significant step towards bridging the gaps of existing data resources. By providing a cohesive framework for data collection and representation, ARIO paves the way for the development of more powerful and versatile embodied AI agents, capable of navigating and interacting with the physical world in increasingly complex and diverse ways.
Comparison between ARIO dataset and other open source embodied datasets
ARIO data comes from three main sources: conversion from open source datasets, generation from simulation platforms, and acquisition of real-world robot scenarios in our hardware platform.
Statistics of the ARIO Dataset
Statistics on Scenes and Skills in the ARIO Dataset. Thanks to the uniform format design of ARIO data, we can easily perform statistical analysis of its data composition. The following figure shows the statistical distribution of ARIO's scenes (Figure a) and skills (Figure b) in three levels: series, task, and episode. It can be seen that most of the present embodied data is focused on the scenes and skills in the indoor living house environment.
In addition to scenes and skills, in the ARIO data, we can also carry out statistical analysis from the perspective of the robot itself, and learn some of the current developments in the robot industry. We provide statistical data of morphologies, motion objects, physical variables, sensors, sensor positions, camera(RGBD) numbers, proportion of control methods, proportion of operation modes, proportion of arm joint numbers, corresponding to Figure a-i below. As shown in Figure a below, it can be found that most of the current data comes from single-armed robots, while the open source data volume of humanoid robots is very small and mainly comes from our real scenario collection and simulation generation.
One part of the data from real-world scenarios is collected in Cobot Magic (AgileX Robotics) platform. We design over 30 tasks, featuring table-top manipulation in household settings. The tasks cover not only general pick and place skills, but also more complex skills like twist, insert, press, cut, etc. Some example of the tasks can be viewed in figure and video below. Please refer to this link for a complete list of tasks. Below are some example tasks in Cobot Magic, with the top row indicating the task category while the text at the botom row providing task instructions.
@misc{wang2024robotsonenewstandard,
title={All Robots in One: A New Standard and Unified Dataset for Versatile, General-Purpose Embodied Agents},
author={Zhiqiang Wang and Hao Zheng and Yunshuang Nie and Wenjun Xu and Qingwei Wang and Hua Ye and Zhe Li and Kaidong Zhang and Xuewen Cheng and Wanxi Dong and Chang Cai and Liang Lin and Feng Zheng and Xiaodan Liang},
year={2024},
eprint={2408.10899},
archivePrefix={arXiv},
primaryClass={cs.RO},
url={https://arxiv.org/abs/2408.10899},
}
This work was supported by Research Institute of Multiple Agents and Embodied Intelligence (IMAEI), Pengcheng Laboratory. Specifically, we would like to express our gratitude to researchers from IMAEI, Southern University of Science and Technology, Sun Yat-sen University and Agilex Robotics, who participated in the formulation of ARIO standard. They are Hua Ye, Wenjun Xu, Zhiqiang Xie, Junfan Lin, Lingbo Liu, Xuewen Cheng, Chang Cai, Liang Lin, Feng Zheng, Xiaodan Liang and so on. In addition, we thank all those involved in the production of the ARIO dataset, including: Hua Ye, Zhiqiang Wang, Qingwei Wang, Hao Zheng, Yunshuang Nie, Kaidong Zhang, Zhe Li, Wenjun Xu, Wanxi Dong, Zhiqiang Xie, Ruocheng Yin, Yao Mu, Xuewen Cheng, Chang Cai, Jinrui Zhang, Liang Lin, Feng Zheng, Xiaodan Liang, Junfan Lin, Lingbo Liu, Tingting Shen, Yazhan Zhang, Jichang Li, Qingyong Jia, Zhen Luo, Fangjing Wang, Zesheng Yang, Luyang Xie, Liang Xu, Yi Yan, Yixuan Yang, Jingnan Luo, Tongsheng Ding, Ziwei Chen, Guoyu Xiong, Xi Jiang, Tiantian Geng, Zhenhong Guo, Xue Jiang, Zhengyu Lin, Ziling Liu, Bingyi Xia, Jingyi Liu, Shiwei Zhang, Yun Pei, Yao Xiao. We also extend our gratitude to the various open-source, datasets and platforms, including Open X-Embodiment, RH20T, ManiWAV, JD ManiData, and the contributors from Open X-Embodiment. Their contributions were vital in creating the ARIO dataset. Special thanks to the Habita-sim simulation platform, Habitat-lab module library, Habitat-Matterport 3D Dataset (HM3D) indoor dataset, and the Habitat Challenge organized by Facebook AI Research. It is through your open-source support that we were able to collect navigation simulation data. We are grateful for the Scaling Up and Distilling Down project for the simulation framework and the MuJoCo physics engine, aiding in generating simulation manipulation data. We appreciate the ARIO Embodied Intelligence Data Open Alliance members, such as Southern University of Science and Technology, Sun Yat-sen University, Dataa Robotics, Agilex Robotics, and JD Technology, for their technical support and contributions to the ARIO dataset development. The collaborative efforts have significantly advanced embodied AI research through the creation of the ARIO dataset.
For the real robot data collected by our team or materials generated by the simulation platform developed by us, are licensed under the Creative Commons Attribution 4.0 International License (CC-BY) or MIT. For materials converted from open source datasets or generated by simulation platform developed by others, we just follow their original license while publishing. For details, pay attention to the license of each open source project.
In March 2024, An alliance organization for building open-source data for embodied intelligence has been established, and that is ARIO Alliance. The ARIO Alliance currently has 10 member units and they are Pengcheng Laboratory, Southern University of Science and Technology, Sun Yat-sen University, Agilex Robotics, University of Hong Kong, Chinese University of Hong Kong, Technical University of Munich, Dataa Robotics, D-Robotics, JD Technology. And there are still members coming in. The ARIO Alliance aims to promote the academic prosperity and industry development of embodied intelligence by building open source embodied datasets and algorithmic models, as well as industry standards.
