Interactive SO-101 IK in ROS 2 with Viser + robokin

Hi everyone,

I wanted to share a small ROS 2 kinematics example I built for the SO-101 arm:

so101_kinematics

It is built on top of:

robokin

The main goal was to have a simple interactive IK workflow that can run on a Raspberry Pi-based server without depending on a heavy desktop GUI. For that, I’ve been using Viser, which I’ve found amazing for robot interaction and debugging from the browser.

At the moment this is mostly an IK example/integration:

  • ROS 2 + robokin

  • Viser-based interactive end-effector control

  • Placo-based solving for the SO-101

robokin itself is a small kinematics helper library I’m building around URDF-based robots, with support for tools like Placo, PyRoki, RoboPlan, Viser, and Rerun.

I’d be very interested in feedback, especially around:

  • architecture ideas

  • how people would structure this kind of ROS 2 + web UI + IK setup

  • whether others are using Viser in similar ways

This started as something I needed for my own workflow, but I thought it might be useful to share.

Thanks!

At the end, I added a short demo showing the interactive IK workflow in ROS 2. There are also more demo videos and example integrations in the robokin repository:

https://www.youtube.com/watch?v=yIVHwF1LioM

3 Likes

Interactive IK via Viser is great, but you’d get really far by demonstrating that this architecture also works with other devices (leader arm, gamepad, etc.). It seems like you are already setup with ROS topics that handle this, so it’s just a matter of showing it!

This is of great interest to me on RoboPlan (thanks for trying it!), so I’m also happy to help make that integration better.

1 Like

This is very cool! Any plans to add a ROS node that exposes the IK directly, i.e., subscribes to a topic accepting cartesian coordinates and publishes joint angles?

1 Like

Thank you so much, and also for the offer to help with RoboPlan. It really means a lot to hear that from you.

A big reason I built robokin was that I was looking for practical IK solutions for the SO-101 and could not find something that matched my use case well through the usual MoveIt path. That led me to start experimenting with Placo, RoboPlan, and Pyroki. Pyroki was also what made me notice Viser for the second time.

At the time, what mattered most to me was simply being able to give a target pose and get back a usable approximation for a 5-DOF arm. What really worked for me were numerical IK methods, especially Jacobian-based/differential approaches, because they could give a good approximation even on a 5-DOF arm. From there, I built very simple trajectories between poses, both Cartesian stepping with IK solved along the path and basic joint-space motion. The goal was to make pick-and-place sequences work first, and later possibly other tasks as well. :blush:

Viser started mainly as a debugging and visualization tool that I could run on my Raspberry Pi Ubuntu server and use from the desktop. I wanted something lightweight where I could inspect trajectories and better understand what the robot was doing. The interactive IK part came naturally from that (I thought it was nice, because it shown remote operation very easy). Later, I also started using the same setup for things like perception debugging.

Your point about demonstrating the same architecture with other devices like a leader arm or gamepad is actually very interesting to hear. I had not thought much about that because I assumed that part was already well covered by others, but if you think it is still valuable, I would definitely like to explore it.

On RoboPlan, I have also done some experiments, but I still need to clean them up and publish proper examples. I plan to share those soon. I just had to move forward with some other tasks first, including work on an autonomous/self-training robot arm environment, which is why the RoboPlan examples are not cleaned up yet. :sweat_smile:

Thanks again, this is very encouraging. :slight_smile:

Thank you so much for the kind words and the suggestion — that is genuinely very encouraging to hear.

That’s actually great suggestion, because I am still experimenting and don’t know yet correct way to make architecture reusable.

I am currently in my workflow a bit more plan-and-execute side inside each node, then exposing IK itself as a ROS2 interface.

At the moment, the flow in my code is more like:

def _go_to_pose(self, T_goal: np.ndarray): 
    q0 = self.q_measured.copy() if self.has_arm_feedback else self.q_commanded.copy() 
    ts, qs = self.planner.plan_pose_move(q0, T_goal, strategy="joint_quintic")
    self.traj_executor.start(ts, qs)

and then in control loop I execute the current sampled trajectory over time:

if self.traj_executor.is_active(): 
     q_cmd, reached_end = self.traj_executor.sample() 
     self._publish_cmd(q_cmd) 
     self.q_commanded = q_cmd.copy() 
     self.solver.set_joint_state(q_cmd) 
     if reached_end: 
         self.traj_executor.cancel() 
         motion_done = True

So right now its not really “a ROS node exposing IK directly”, but more a pipeline like:

Cartesian goal → planner / IK → trajectory → execution

Because of that, I think now your suggestion makes a lot of sense. A cleaner ROS2 integration would be separate those layers on dedicated Nodes clearly.

So if I understand correctly your point and also looking at my current workflow, next would be do:

  • an IK service node: Cartesian pose in, solved joint configuration out
  • then possibly a topic-based interface later for continuous Cartesian targets, publishing directly in ForwardCommandController
  • separately higher-level plan/execute path for trajectory generation and execution (will be interesting to solve tbh :grinning_face_with_smiling_eyes:)

So yes, I think now the direction is probably wrapping the IK solver itself as dedicated ROS2 node, instead of only using it inside the current planning/execution flow. Probably best is service-based interface first, with a topic based later for interactive tools like Viser, leader-arm, gamepad control? :thinking:

I am still exploring by experimenting, so I do not fully know yet what the cleanest architecture will be, but your suggestions very helpful and probably the right direction. :slight_smile:

1 Like

Thanks for your response @dmitri_manajev . Yes, I think now that you say it I realize what I really want is a way to control an so101 by publishing Cartesian (goal) coordinates to a ROS topic, meaning the node should do both IK and path planning + execution.

Btw, do you have a sense of the benefits of Robokin + Placo over MoveIt?

Thanks, that actually helps clarify the question a lot.

About Robokin + Placo vs MoveIt: I would not say it is “better” than MoveIt in a general sense. For me it was more that I was looking for a very practical workflow where I could say “here is a pose, send the arm there” and actually get something usable for the SO-101.

The main problem I kept running into is that the SO-101 is really a 5-DOF arm for arm kinematics, even though people many times call it 6-DOF because they count the gripper. A lot of IK tools and configs seem much more naturally designed for 6- or 7-DOF manipulators. I found a lot of MoveIt configs online for the SO-101, but most of what I saw was more about joint moves than the kind of pose-based IK workflow I wanted.

During my experiments, PickIK was actually one of the more interesting things I found, and I did manage to get it working for reachable poses, but not as well for approximate poses. I may still be missing better ways to configure that, but I also pushed an update with my own SO-101 MoveIt config experiments here:

So for me, Robokin + Placo was not “better than MoveIt,” but more a lightweight and flexible way to experiment. It let me load the URDF, try numerical IK, accept approximate solutions for a 5-DOF arm, and build simple trajectory generation around that. Viser also made that workflow much nicer, because it gave me an easy remote interactive view and debugging tool, similar to RViz+MoveIt.

In the end, I think it is becoming surprisingly simple to download a URDF, try different IK implementations in a similar spirit to how MoveIt uses plugins, and then choose whichever solver fits your workflow best, like Placo, RoboPlan, or Pyroki. You can load the same URDF into the solver, then into tools like Viser and Rerun, and very quickly build a working environment with modern tools around it. Logically, this should feel even more natural on 6- or 7-DOF arms than on the current SO-101 example.

What I have now is still more proof of concept / work in progress than final architecture, but your point about wrapping it as a proper ROS 2 Cartesian goal interface is exactly the direction that makes sense next. :slight_smile:

2 Likes

Oh, please don’t use PickIK! I worked on that some years ago and, while we got it working, it has serious performance issues compared to its inspiration, BioIK. The BioIK code is unreadable, but they do have some nice forward kinematics caching mechanism that makes it 10x faster than PickIK and therefore competitive with other solvers. So I would still recommend BioIK instead.

Reference: `pick_ik` is 10x slower than `bio_ik` · Issue #60 · PickNikRobotics/pick_ik · GitHub

1 Like

Wow, I did not know that. I wanted to try BioIK too while testing different plugins, but I ended up choosing PickIK because it was newer. Surprisingly, I was able to at least partially get the job done, so kudos to you for that :mechanical_arm:

Looks like I should have tried BioIK first after all. Thanks for the context — that is really useful to know.

1 Like

After talking with you about it, I got a better idea of what a possible design for this could look like and I ended up making something in that direction for the interactive/manual end-effector control from Viser, and I used it in some hand-eye calibration experiments. The Cartesian control part does point in the ROS 2 direction we were discussing.

And the motion node itself is here:

So I think this could later be wrapped as a more configurable ROS 2 node, with the IK backend selectable as a parameter.

The GUI I am using there is visible in the 2nd and 3rd videos in this PR comment:
That calibration workflow is still very, very experimental though, and not something I would recommend to anyone for reliable extrinsics on the SO-101
Add camera calibration tools + motion services for SO-101 by legalaspro · Pull Request #11 · legalaspro/so101-ros-physical-ai · GitHub

1 Like

I would like to add that I would be super interested in Teleoperation with Oculus Quest. We want to teach children different ways to control/program SO-ARM100.
I took some notes a while ago, but never found time to start working on it:

Other : https://github.com/lts0429/teleoperation