# Isaac ROS May update, 3.0 release adds more AI + robot manipulation

**URL:** <https://discourse.openrobotics.org/t/isaac-ros-may-update-3-0-release-adds-more-ai-robot-manipulation/38017>\
**Category:** ROS General\
**Tags:** ros2, moveit2, ai, wg-acceleration, gpu\
**Created:** [June 2, 2024, 3:53pm UTC](https://discourse.openrobotics.org/t/isaac-ros-may-update-3-0-release-adds-more-ai-robot-manipulation/38017 "2024-06-02T15:53:27Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![ggrigor](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/ggrigor/32/11645_2.png) [@ggrigor](https://discourse.openrobotics.org/u/ggrigor)\
**Post date:** [June 2, 2024, 3:53pm UTC](https://discourse.openrobotics.org/t/isaac-ros-may-update-3-0-release-adds-more-ai-robot-manipulation/38017/1 "2024-06-02T15:53:28Z")

</div>

We are pleased to announce at Computex Taipei 🎏 the release of Isaac ROS 3.0 with major updates including new packages, performance improvements and fixes.

The Isaac ROS update for ROS 2 Humble is available at [github.com/NVIDIA-ISAAC-ROS](http://github.com/NVIDIA-ISAAC-ROS), including packages for AI perception, image & LIDAR processing, navigation and adds:

- **New!** workflows for robot arms with [Isaac Manipulator](https://nvidia-isaac-ros.github.io/reference_workflows/isaac_manipulator/index.html) and AMR’s with [Isaac Perceptor](https://nvidia-isaac-ros.github.io/reference_workflows/isaac_perceptor/index.html).
- **New!** package [cuMotion](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_cumotion) for MoveIt 2 providing hardware accelerated motion planning with collision avoidance with obstacles.
- **New!** FoundationPose DNN for [pose\_estimation](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_pose_estimation) and tracking of unseen objects from a 3D model of the object.
- **New!** Multi-camera [visual odometry](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_visual_slam) providing robust visual tracking.
- **New!** Multi-camera [nvBlox](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_nvblox) 3D reconstruction for costmaps used in motion planning with Nav2 and MoveIt 2.
- **New!** Segformer and SAM (Segment Anything) DNN packages for transformer based [image\_segmentation](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_image_segmentation).
- **New!** [data\_recorder](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_nova/isaac_ros_data_recorder/index.html) for multi-sensor data capture to ROSBag using MCAP and [data\_replayer](https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_nova/isaac_ros_data_replayer/index.html) for AI and perception development with time synchronized real sensor data at 500+ mega samples per second.
- **New!** Software for out of the box ROS 2 development with [Nova Orin Developer Kit](https://nvidia-isaac-ros.github.io/robots/nova_developer_kit/index.html) to stop losing time on system software and drivers.
- Updated [ESS 4.0](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/isaac/models/dnn_stereo_disparity) stereo depth estimation DNN with improved accuracy, mobile robot and manipulation use cases with higher performance at \>100 frames per second.
- Performance improvements with event based scheduling for NITROS.
- **Bonus!** [tutorial](https://nvidia-isaac-ros.github.io/robots/nova_carter/demo_docking.html) for new Nav 2 docking 🅿 feature with [mission\_dispatch](https://github.com/nvidia-isaac/isaac_mission_dispatch) on the cloud connected over VDA5050 to [mission\_client](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_mission_client).
- Update to Jetpack 6.0 with Ubuntu 22.04 with real-time optimizations and CUDA 12.2
- Bug fixes

![isaac_ros_nvblox_zanker_cuboids_sm](https://us1.discourse-cdn.com/flex022/uploads/ros/original/3X/5/c/5cbcedebdb9e4c19ced036b66ca06c1d3cfbc870.gif)  
_nvBlox visualization of cuboid reconstruction with multi-camera perception to avoid obstacles missed by planar LIDAR._

![foundationpose_1.0_manipulation](https://us1.discourse-cdn.com/flex022/uploads/ros/original/3X/2/0/20e5f499c8c076acc4f36e1fbf7c34e0df8aa286.gif)  
_FoundationPose detection on challenging objects with symmetry, reflections, specular highlights, and camera motion blur._

 ![ess_v4.0.0_overview](https://us1.discourse-cdn.com/flex022/uploads/ros/original/3X/5/d/5dbcf27c4234ec8a4a0fb5f62d32271d84172059.jpeg)  
_ESS 4.0, AI based depth estimation for manipulation and AMR applications._

Isaac ROS 3.0 is available now at [github.com/NVIDIA-ISAAC-ROS](http://github.com/NVIDIA-ISAAC-ROS) and is part of our commitment to provide features and hardware acceleration for commercial deployment, development and research for autonomous robots.

Install pre-built Debian packages, leverage the [Isaac ROS Dev](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/isaac/containers/ros) container, or clone the repositories you need into your ROS workspace to build from source with colcon alongside your other ROS 2 packages. Please note that this release has been tested on the NVIDIA Jetson [AGX](https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-agx-xavier/)Orin with [JetPack](https://developer.nvidia.com/embedded/jetpack)6.0. For more details see the [release notes.](https://nvidia-isaac-ros.github.io/releases/index.html)

A minor hotfix will be released this June with the next major release in Oct at ROSCon 2024.

---

<div class="post-metadata">

**Author:** ![smac](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/smac/32/6763_2.png) [@smac](https://discourse.openrobotics.org/u/smac)\
**Post date:** [June 3, 2024, 8:07pm UTC](https://discourse.openrobotics.org/t/isaac-ros-may-update-3-0-release-adds-more-ai-robot-manipulation/38017/2 "2024-06-03T20:07:03Z")

</div>

Also make sure to check out Nav2’s new Docking Server released in conjunction with Isaac 3.0 and supported by NVIDIA’s ongoing collaborations with Open Navigation!

> **[New: Nav2 Docking Server](https://discourse.openrobotics.org/t/new-nav2-docking-server/38036)**
>
> Howdy! Its your Friendly Neighborhood Navigator here with an announcement in conjunction with NVIDIA’s Isaac 3.0 Release! We’ve been working closely with NVIDIA this year to bring new capabilities to the mobile robotics community and have a new...

---

<div class="post-metadata">

**Author:** ![amacneil](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/amacneil/32/9758_2.png) [@amacneil](https://discourse.openrobotics.org/u/amacneil)\
**Post date:** [June 5, 2024, 4:35am UTC](https://discourse.openrobotics.org/t/isaac-ros-may-update-3-0-release-adds-more-ai-robot-manipulation/38017/3 "2024-06-05T04:35:38Z")

</div>

Congrats to @ggrigor and the NVIDIA team on this release! 👏

Also, it’s exciting to see support for MCAP expanding within the industry and ROS ecosystem 💜

---

<div class="post-metadata">

**Author:** ![boba](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/boba/32/19111_2.png) [@boba](https://discourse.openrobotics.org/u/boba)\
**Post date:** [June 11, 2024, 8:13pm UTC](https://discourse.openrobotics.org/t/isaac-ros-may-update-3-0-release-adds-more-ai-robot-manipulation/38017/4 "2024-06-11T20:13:24Z")

</div>

Thankyou ggrigor and team for making this suite available, this is awesome! Are there any limitations I should immediately be aware of for Orin Nano on dev kit?

---

<div class="post-metadata">

**Author:** ![ggrigor](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/ggrigor/32/11645_2.png) [@ggrigor](https://discourse.openrobotics.org/u/ggrigor)\
**Post date:** [June 13, 2024, 7:46pm UTC](https://discourse.openrobotics.org/t/isaac-ros-may-update-3-0-release-adds-more-ai-robot-manipulation/38017/5 "2024-06-13T19:46:01Z")

</div>

> [@boba](#):
>
> Are there any limitations I should immediately be aware of for Orin Nano on dev kit?

Review the [Jetson Orin Technical Specifications (bottom of the page)](https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-orin/) to be informed on limitations for a Jetson Orin Nano vs an Jetson Orin AGX. These typically com from hardware not available in Orin Nano, lower compute, or reduced memory.

The [performance results](https://nvidia-isaac-ros.github.io/performance/index.html) have a column for Orin Nano which provides an easy metrics based comparison between platforms. For some benchmarks Orin Nano is not supported due to a lack of HW, or limitations on memory to fit AI models.

For a concrete example, Orin Nano does not have H.264 encode, so the hardware accelerated [Image compression](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_compression) to encode image data into H.264 is not present, however H.264 decode is.

Thanks

---

<div class="post-metadata">

**Author:** ![Jimi1811](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/jimi1811/32/20815_2.png) [@Jimi1811](https://discourse.openrobotics.org/u/Jimi1811)\
**Post date:** [June 13, 2024, 8:11pm UTC](https://discourse.openrobotics.org/t/isaac-ros-may-update-3-0-release-adds-more-ai-robot-manipulation/38017/6 "2024-06-13T20:11:37Z")

</div>

Hello! I would like to add that the 8GB of RAM is not enough if you want to execute ISAAC ROS VSLAM, ISAAC ROS Nvblox and the ZED wrapper.

---

<div class="post-metadata">

**Author:** ![ggrigor](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/ggrigor/32/11645_2.png) [@ggrigor](https://discourse.openrobotics.org/u/ggrigor)\
**Post date:** [June 17, 2024, 2:42pm UTC](https://discourse.openrobotics.org/t/isaac-ros-may-update-3-0-release-adds-more-ai-robot-manipulation/38017/7 "2024-06-17T14:42:24Z")

</div>

[Isaac ROS](https://github.com/NVIDIA-ISAAC-ROS) has been updated to 3.0.1 with the release of [Isaac Sim 4.0](https://developer.nvidia.com/isaac/sim) and adds sensor simulation with NITROS to improve performance.

Simulated sensor data for a robot under test typically passed using the DDS from the simulator to the ROS 2 application across processes using memory copies on the CPU. This is inefficient as simulated camera pixels are rendered on the GPU, and processed by the Isaac ROS 2 application on the GPU.

[NITROS](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_nitros) using **type adaptation (rep-2007)** keeps simulated camera pixels in GPU memory when passing image topics from the simulator to Isaac ROS application. For more details see the [tutorial](https://nvidia-isaac-ros.github.io/concepts/nitros_bridge/tutorial_isaac_sim.html).

Thanks

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex022/uploads/ros/original/3X/b/6/b607ee3cc8e908b1b613bd74a683d749fea635a1.svg) [@system](https://discourse.openrobotics.org/u/system)\
**Post date:** [September 15, 2024, 2:42pm UTC](https://discourse.openrobotics.org/t/isaac-ros-may-update-3-0-release-adds-more-ai-robot-manipulation/38017/8 "2024-09-15T14:42:40Z")

</div>

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.