ZED ROS 2 Wrapper 5.6: CUDA zero-copy with rosidl::Buffer for ROS 2 Lyrical and Isaac ROS 5

Projects
1

Hi everyone,

We just released ZED ROS 2 Wrapper v5.6, which adds support for the new rosidl::Buffer message fields of ROS 2 Lyrical with the CUDA buffer backend.

rosidl::Buffer and the CUDA buffer backend were contributed by NVIDIA to ROS 2 Lyrical in collaboration with the OSRA, and they are the foundation of the accelerated data path of NVIDIA Isaac ROS 5. With this release, the ZED node publishes its dense data directly from the GPU memory where the ZED SDK computes it, so GPU-accelerated subscribers no longer pay the GPU → CPU → serialize → deserialize → GPU round trip.

What’s new

  • CUDA zero-copy for all the dense data: images, depth maps, confidence maps and point clouds from the ZED stereo cameras, and images from the ZED X One monocular cameras.
  • Standard messages and topics: sensor_msgs/msg/Image, sensor_msgs/msg/PointCloud2, and so on. No custom types, no special topics, no NITROS bridge.
  • Automatic CPU fallback: subscribers that do not accept the CUDA backend (RViz2, ros2 bag, Python nodes, nodes on another machine) automatically receive standard CPU data. The ZED node does not need to know who is listening.
  • Enabled by default: the new general.buffer_backend parameter selects cuda (default) or cpu. If a requirement is missing, the node falls back to CPU and logs the reason.
  • Fast DDS or Zenoh: works with rmw_fastrtps_cpp (the Lyrical default) and with rmw_zenoh_cpp v0.10.5 or newer.

Subscribing to GPU data

An accelerated node needs only two changes: declare that the subscription accepts the CUDA backend, and read the GPU memory through the CUDA buffer API.

rclcpp::SubscriptionOptions sub_opts;
sub_opts.acceptable_buffer_backends = "cuda";
auto sub = create_subscription<sensor_msgs::msg::Image>(
  "image", qos, callback, sub_opts);

// In the callback
if (img.data.get_backend_type() == "cuda") {
  auto rh = cuda_buffer_backend::from_input_buffer(img.data, stream);
  my_cuda_kernel(rh.get_ptr());  // GPU memory of the ZED node, no copy
}

The same node keeps working with CPU data when zero-copy is not available. Note that intra-process (IPC) subscribers receive CPU data: to receive GPU buffers, disable IPC as explained in the image tutorial.

Numbers on Jetson AGX Thor

Measured with a ZED 2i on a Jetson AGX Thor Developer Kit (MAXN, jetson_clocks), Isaac ROS 5 on bare metal, latency from publish to data available on the subscriber GPU:

  • Latency reduction from 10% (compact point cloud) up to 61% (full-resolution 14.7 MB point cloud).
  • About 3x lower standard deviation of the point cloud latency.
  • All messages received as CUDA buffers at the camera frame rate.

Results depend on platform, power mode, camera, resolution and system load. The two new examples in zed-ros2-examples run the same CPU vs CUDA benchmarks, so you can reproduce them on your own hardware.

Requirements

  • ROS 2 Lyrical (as provided by NVIDIA Isaac ROS 5)
  • ZED ROS 2 Wrapper v5.6.0 or newer, ZED SDK v5.5 or newer
  • The cuda_buffer packages
  • The ZED node and the accelerated subscribers running on the same computer and GPU

Links

The wrapper supports the whole StereoLabs camera lineup, including the ZED X, ZED X One and ZED 2 families, and the ZED Box 2 series powered by Jetson Thor and Orin.

Feedback and issues are welcome on GitHub or here in this thread.