Real-Time Cartesian Controller for UR5/UR5e with External Sensor Feedback (ROS 2 Humble, ros2_control)

[seyiR]

Hi everyone,

My name is Oluwaseyi Afolayan. You can call me Seyi!

I’ve open-sourced a ROS 2 Humble package for real-time Cartesian position control of UR5/UR5e robots. The system uses external SE3 sensor feedback and inverse kinematics to drive precise end-effector positioning.

Demo Videos

Simulation + Visualization:

Real UR5 Robot:

Key Features

• Position-only Cartesian control with PID feedback (100 Hz control loop)
• External sensor integration via custom ros2_control hardware interface
• SVD-based damped pseudo-inverse Jacobian for numerical stability near singularities
• Multiple modes: Real robot, Gazebo simulation, fake hardware
• Dynamic target updates via TF transforms at runtime

Architecture

The interesting part is how I bridged external sensor data into ros2_control. Since hardware interfaces can’t directly subscribe to TF, I built a TF Lookup Server that:

1. Listens to TF for robot and target sensor frames
2. Serializes poses and sends them over TCP/IP sockets
3. The hardware interface reads from the socket in its read() method

This lets the Cartesian controller access real-time sensor data through the standard ros2_control state interface pattern.

SE3 Sensors → TF → TF Lookup Server (TCP) → Hardware Interface → Cartesian Controller → UR5

Quick Start

# Clone
cd ~/ros2_ws/src
git clone https://github.com/Seyi-roboticist/_controller_.git

# Build
cd ~/ros2_ws
rosdep install --from-paths src --ignore-src -r -y
colcon build --packages-select se3_sensor_driver ros2_controller_cartesian ur5e_cartesian_control

# Run (fake hardware)
ros2 launch ur5e_cartesian_control ur5e.launch.py use_fake:=true

# Set target dynamically
ros2 run tf2_ros static_transform_publisher 0.4 0.2 0.5 0 0 0 base_link target_sensor_frame

GitHub

https://github.com/Seyi-roboticist/\_controller\_

Feedback Welcome

I’d appreciate thoughts on:

1. The TCP socket approach for TF→hardware interface bridging — is there a cleaner pattern I’m missing?
2. Jacobian damping strategy — currently using fixed damping factor, considering adaptive damping near singularities
3. Any gotchas with UR robots I should be aware of for production use?

Thanks for taking a look!

[seyiR]

Apologies, I made a small mistake in the link. Here is the correct one: controller