End-to-End Imitation Learning for SO-101 with ROS 2

Projects
, ,
1

Hi everyone,

I’d like to share SO-101 ROS Physical AI, an open-source ROS 2 stack for the SO-101 robot arm focused on an end-to-end imitation learning workflow on real hardware.

GitHub repo: legalaspro/so101-ros-physical-ai

The idea behind this project was to build a ROS 2-native full pipeline around a low-cost, accessible robot arm that people can actually learn on in practice.

I wanted something that could be useful for students, researchers, and hobbyists, while also serving as a practical reference for implementing similar ROS 2 concepts on other robot arms.

That full workflow was the main motivation for the project: not just isolated demos, but a ROS 2-native path from data collection to execution.

What the repo includes

  • Real robot bringup for the SO-101 arm
  • ros2_control integration with a Feetech STS3215 hardware interface
  • Leader/follower teleoperation for collecting demonstrations
  • Multi-camera support
  • Episode recording for imitation learning data collection
  • ROS-to-Rerun visualization for observations, actions, camera streams, and reviewing MCAP files
  • Rosbag / MCAP to LeRobot dataset conversion
  • Policy inference
  • sync on-device inference
  • async remote inference through a policy server for action-chunking LeRobot policies such as ACT, SmolVLA, π0/π0.5, and more

Current inference support

At the moment, the repo includes:

  • on-device sync inference for ACT and SmolVLA
  • async remote inference for action-chunking LeRobot policies via policy_server

One of the parts I especially wanted to make practical was remote inference: the robot runs the ROS 2 client locally, while the policy can run on a remote GPU server.

Typical workflow

  1. Teleoperate the robot in a leader/follower setup
  2. Record demonstration episodes as rosbags / MCAP files
  3. Convert recorded episodes into a LeRobot dataset
  4. Train a policy with LeRobot
  5. Run the learned policy back through ROS 2 on the real robot

Why I built it

There are already many strong tools in both the ROS 2 and robot learning ecosystems, but I could not find a single reusable ROS 2 example covering the full imitation learning workflow for manipulation on real hardware:

  • real robot bringup
  • teleop for demonstrations
  • dataset collection
  • conversion into LeRobot format
  • local or remote inference from ROS 2

For people who want to learn learning-based control in robotics, especially imitation learning, this felt like an important missing piece.

I also wanted it to be a practical platform that people could learn from and build on in practice.

Demo highlights

I also recorded a few short videos showing different parts of the pipeline:

Feedback welcome

I’m still actively improving the stack, and I’d love feedback from people working at the intersection of:

  • ROS 2
  • ros2_control
  • LeRobot
  • imitation learning
  • remote policy inference

I hope this can be useful both as a practical project and as a learning resource for people exploring imitation learning on real robots with ROS 2.

Thanks!

At the end, I also included a short demo showing a simple way to run async inference in ROS 2 with a remote policy server:

13 Likes
ROS News for the week of March 9th, 2026
2

Wow, this is awesome. I think you should reach out to our Physical AI working group to present this project.

1 Like
3

Thanks you so much, I really appreciate that. I’d be happy to show it to the Physical AI working group. I’m just not sure, who I should reach out to. :sweat_smile:

4

Awesome work @dmitri_manajev ! This really looks like an amazing project to get started with Imitation Learning and Physical AI with ROS 2.

1 Like
5

Thank you so much, Alberto — I really appreciate it. I’m also very grateful to The Construct Robotics Institute, since this project originally started as my final Masterclass project and a way to showcase what I learned during the course. :mechanical_arm: