📢 Announcing CRISP: Closing the Gap Between ROS 2 and Robot Learning

ROS ROS General
, , , ,
1

Hi everyone,

In our lab we’ve been working on robot learning, in particular with Vision-Language-Action models (VLAs). To make it easier for the community to experiment with these methods on real robots, we decided to open source our package: CRISP :tada:

Our main contributions are:

  • Robot-agnostic, torque/effort-based ros2_controllers for manipulators.
  • A simple Python + Gymnasium interface to collect manipulation data and deploy learned policies.

We also provide detailed documentation so that both novice ROS 2 users and researchers can get started quickly. Our hope is to lower the entry barrier to robot learning and accelerate research in this domain!

:hammer_and_wrench: Features and Notes

  • Controllers

    • Includes an operational space controller and a Cartesian impedance controller with additional useful features.
    • Uses Pinocchio for URDF parsing and all rigid-body dynamics computations.
    • We provide dockerized demos to show how to start the controllers.

  • Integration with LeRobot

    • LeRobot provides a fantastic platform for training VLAs, but it does not directly integrate with ROS 2 robots (and maybe it shouldn’t).
    • We provide a Python + Gym interface on top of our controllers, with scripts to:
      • Record data in the LeRobot format
      • Train policies using LeRobot
      • Deploy trained policies on real robots
    • Our Python and Gym interface are how
    • For the interfaces, we use pixi and robostack. This makes it easier to integrate

  • Why not MoveIt?

    • MoveIt is a powerful framework, but it has a higher entry barrier to simply get a robot moving.
    • For VLAs, which act more like path planners, all that’s needed is to stream a target pose from vision inputs. No perception or motion planning stack is strictly required for quick experimentation.

  • Limitations

    • So far we’ve mainly tested on real robots.
    • We do not provide controllers for position/velocity-based control. But cartesian controllers should be compatible with our python-gym interface so integrating them should not be a problem.

:raising_hands: How You Can Get Involved

  • Try out the controllers on your manipulator with an effort-based interface.
  • Share your feedback, ideas, or videos of your experiments!

We’d love to hear your thoughts, suggestions, and experiences.

Cheers,
Daniel San José Pro | Learning Systems and Robotics Lab @ TUM

11 Likes