Hi everyone,
We are currently exploring the use of a robotic platform to collect data on robot arm manipulation in home environments. Building on this, we’re now preparing a broader effort to collect data on household indoor environments and embodied intelligence, and we’d love to hear your thoughts.
The motivation behind this is to capture interaction data between service robots and home environments to address the dependence of embodied AI service robots on large-scale, real-world datasets. However, compared to standard vision datasets, high-quality robot manipulation data in real household scenarios is still relatively scarce. We hope to contribute something useful to advance the development of embodied intelligence for home robots.
Our current plan includes:
Indoor Navigation Data: Continuous image sequences or video streams from real household indoor environments (e.g., furniture layouts, daily object placement, natural lighting changes).
Robot Manipulation Data: Video sequences from a camera mounted on the robot arm’s wrist, along with synchronized end-effector action sequences, as the arm performs various tasks with a gripper (e.g., reaching, picking, and placing).
Future Extensions: We are considering collecting manipulation data with more diverse end-effectors, such as three-finger grippers and dexterous hands, to meet the needs of different service robot platforms.
We would love to hear from the community:
Would you be willing to participate in our home service robot data collection project and share your data?
If we offer payment for each piece of robot manipulation data, what is your desired price?
Do you see a need for such datasets in your research or applications?
What kinds of data formats, task settings, or annotations/metadata would make this most useful?
Are there any best practices you’d recommend when designing a dataset of this type (e.g., reproducibility, benchmarking tasks, standards for recording actions)?
We are very open to suggestions at this stage and want to make sure the data we collect can serve as a valuable resource for others.
Looking forward to your thoughts and ideas!
2025 BEHAVIOR Challenge
