While testing Open-RMF on Ubuntu 24.04 with ROS 2 Jazzy, I noticed that during the negotiation phase, CPU utilization spikes to 100% across all cores and threads. This happens consistently when multiple robots are participating in the negotiation process.
I would have expected CPU usage to remain more balanced or at least configurable to avoid full saturation of system resources.
During negotiation: All cores/threads reach 100% utilization.
After negotiation completes: CPU usage drops back to normal levels.
I’m curious whether you noticed an observable slow down in other processes running on the system or in the fleet adapter itself (besides taking a while to finish the negotiation).
The multi-threading is handled by rxcpp which I believe has a thread pool equal to the number of hardware threads detected, so it will fully utilize the CPU if has enough activity going on. I’d be very open to a PR that allows users to throttle that.
@Aaron_Chong
My System Setup is OS: Ubuntu 24.04.3 with ROS2 Jazzy CPU: AMD Ryzen 9 9950X, 16 cores / 32 threads, up to 5.76 GHz
I’m using my custom map, and in which 4 are being simulated properly, but when I increase to 10, the robots hit each other( yes, the robot models are colliding with each other)
You can refer to this video Video of a collision, and after that, all the robots stopped
The collision is surprising since the Gazebo plugin itself should prevent that as a last resort.
When dealing with 10 robots that will be intersecting each other regularly, it’s recommended to use uni-directional lanes to maintain smooth traffic flows and minimize the amount of negotiation needed. You can see some other traffic management strategies here: Graph Strategies - Programming Multiple Robots with ROS 2
Thank you for the suggestion. However, in our application, bi-directional lane operation is a core requirement, as the robots need to traverse the same paths in both directions.
For that you’ll probably need a more advanced multi-agent traffic planner than what we provide in the old generation.
As part of the next generation effort, we’re working on traffic planners that should be able to handle that degree of decision making, but we’re at least 3-6 months away from having that available, and that wouldn’t include any of the task management capabilities.
Could you provide any guidance or suggestions on how we might implement or approximate this in our current setup? For example, are there interim strategies, plugins, or approaches we could use to mitigate collisions while waiting for the next-generation planner? Any recommendations?
The only mitigations are the ones listed in the link I provided earlier. Other than that, you’d need to fork the code and change out the traffic management system.
The implementation of the current generation was a proof of concept with a narrow scope and wasn’t designed to support alternative traffic management plugins. That’s why we’re doing a rewrite based on generalized interfaces that will allow anyone to integrate any traffic management algorithm.