# \[Google Summer of Code 2024\] Zenoh Support & Benchmarking

**URL:** <https://discourse.openrobotics.org/t/google-summer-of-code-2024-zenoh-support-benchmarking/39526>\
**Category:** Manipulation\
**Tags:** ros2, zenoh, zenoh-ros\
**Created:** [September 8, 2024, 7:23pm UTC](https://discourse.openrobotics.org/t/google-summer-of-code-2024-zenoh-support-benchmarking/39526 "2024-09-08T19:23:47Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![jarbay51](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/jarbay51/32/20963_2.png) [@jarbay51](https://discourse.openrobotics.org/u/jarbay51)\
**Post date:** [September 8, 2024, 7:23pm UTC](https://discourse.openrobotics.org/t/google-summer-of-code-2024-zenoh-support-benchmarking/39526/1 "2024-09-08T19:23:47Z")

</div>

Hello everyone,

I’ve completed the project titled [Zenoh Support & Benchmarking](https://summerofcode.withgoogle.com/programs/2024/projects/sOm8rgZf) with my mentors @sjahr and @Henning_Kayser . In the scope of this GSoC project, i’ve implemented middleware benchmarking tool which aims to compare the middleware performance for [rmw\_zenoh](https://github.com/ros2/rmw_zenoh), [rmw\_fastrtps](https://github.com/ros2/rmw_fastrtps) and [rmw\_cyclonedds](https://github.com/ros2/rmw_cyclonedds), tried to find undiscovered bugs in rmw\_zenoh. we’ve shared [the blog post](https://moveit.ai/moveit/gsoc/2024/08/19/GSoC-2024-Zenoh-Support-and-Benchmarking.html) about this project. Note that it’s used [this branch](https://github.com/CihatAltiparmak/moveit_middleware_benchmark/tree/24-Aug-2024-demonstration) in bechmarking.

I also have made the presentation at [MoveIt GSoC 2024 Final Results: Show & Tell](http://discourse.ros.org/t/moveit-gsoc24-final-results-show-tell/39216) meeting. For those who want to take a look at the presentation, it’s accessible at [this link](https://www.canva.com/design/DAGOgST2nfI/sPXAAIJoXzGtMuuPoDvwfA/edit)

As you can see in the presentation, we have the middleware benchmarking tool. We’ve aimed not just to conduct benchmark for `basic topic sub pub` but also to conduct benchmarks for real robotic scenarios whereas [ApexAI’s performance\_test](https://gitlab.com/ApexAI/performance_test) tool just analyses for core ros concepts. For example `planning pipeline at robot`, `high level task planning with MoveIt` etc. With this, we’ve tried to understand if rmw\_zenoh is suitable for MoveIt project.  
Github: [moveit\_middleware\_becnhmark](https://github.com/CihatAltiparmak/moveit_middleware_benchmark)

I’ve tried the best i can do during summer and i’ve hope i’ve helped in a best way for ros2 core team to fix bugs in rmw\_zenoh. I’m wondering if there are some community which analyses ros middlewares like rmw\_cyclonedds, rmw\_fastrtps and so on. Because It was challenging for me to develop the middleware benchmarking tool in some parts like kernel configurations for benchmark reliability and so on. If there is such a community, i at least want to join their meeting and learn about their ideas. Because i couldn’t see the active tool to compare my results.

Finally we’ve seen that rmw\_zenoh can achieve same performance as rmw\_fastrtps and it has potential to use in real robotic applications in the future even though rmw\_zenoh still has some bugs and in development phase. We also have seen that [MoveIt](https://github.com/moveit/moveit2) can be used with rmw\_zenoh middleware at the moment.

Thanks goes to my mentors @sjahr and @Henning_Kayser . Other thanks goes to @JEnoch for valuable suggestions at [Zenoh Community (especially discord)](https://zenoh.io/community/). I couldn’t complete this project without their helps.

_Please test rmw\_zenoh and help for ros2 core team to fix bugs. I think that to try rmw\_zenoh in other computers makes its development phase easier_

PC specifics:

| | |
| --- | --- |
| CPU | Intel® Core™ i7-6700HQ Processor 4 core 8 thread |
| RAM | 8 GB DDR4 2133MHz |
| OS | Ubuntu 22.04 |
| ROS version | rolling |

---

<div class="post-metadata">

**Author:** ![Rayman](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/rayman/32/268_2.png) [@Rayman](https://discourse.openrobotics.org/u/Rayman)\
**Post date:** [September 9, 2024, 9:24am UTC](https://discourse.openrobotics.org/t/google-summer-of-code-2024-zenoh-support-benchmarking/39526/2 "2024-09-09T09:24:48Z")

</div>

@jarbay51 thank you for all your work. What is your current recommendation for which RMW to choose for jazzy? Is it still too early for rmw\_zenoh or can you work around its problems? And also which RMW is the runner-up? Cyclone or FastDDS?

---

<div class="post-metadata">

**Author:** ![jarbay51](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/jarbay51/32/20963_2.png) [@jarbay51](https://discourse.openrobotics.org/u/jarbay51)\
**Post date:** [September 9, 2024, 9:47am UTC](https://discourse.openrobotics.org/t/google-summer-of-code-2024-zenoh-support-benchmarking/39526/3 "2024-09-09T09:47:45Z")

</div>

@Rayman unfortunately i cannot say anything about which rmw middleware should be selected for jazzy, i conducted my benchmark tests using ros rolling. But at least i can say that rmw\_zenoh is not ready for production so that we still have open reported issues like [rmw\_subscription\_data\_t sub\_data\_handler problem · Issue #240 · ros2/rmw\_zenoh · GitHub](https://github.com/ros2/rmw_zenoh/issues/240) and still has some dataraces. However you can use rmw\_zenoh for navigation2 and moveit2 projects. I think the more reliable benchmark results can be produced using Apex AI’s performance\_test tool for rmw\_fastrtps and rmw\_cyclonedds.(I didn’t say for rmw\_zenoh because rmw\_zenoh crashes performance\_test). There are a lot of factor that affects the benchmark results like qos etc. I could test with default qos and this factors can put different middlewares to first place. (i forgot to write my PC specifics. I will update in my available time)

---

<div class="post-metadata">

**Author:** ![Timple](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/timple/32/4891_2.png) [@Timple](https://discourse.openrobotics.org/u/Timple)\
**Post date:** [September 9, 2024, 11:27am UTC](https://discourse.openrobotics.org/t/google-summer-of-code-2024-zenoh-support-benchmarking/39526/4 "2024-09-09T11:27:34Z")

</div>

Great work! It’s these kind of initiatives that prevent a lot of hard-to-debug scenarios when working with real robots!

May I ask what the origin of these numbers are?

```auto
# for rmw_zenoh
sudo sysctl -w "net.ipv4.tcp_rmem=102400 102400 102400"
sudo sysctl -w "net.ipv4.tcp_wmem=102400 102400 102400"
sudo sysctl -w "net.ipv4.tcp_mem=102400 102400 102400"

# for all middlewares
sudo sysctl -w net.ipv4.udp_mem="102400 873800 16777216"
sudo sysctl -w net.core.netdev_max_backlog="30000"
sudo sysctl -w net.core.rmem_max="20971520"
sudo sysctl -w net.core.wmem_max="20971520"
sudo sysctl -w net.core.rmem_default="20971520"
sudo sysctl -w net.core.wmem_default="20971520"

```

---

<div class="post-metadata">

**Author:** ![jarbay51](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/jarbay51/32/20963_2.png) [@jarbay51](https://discourse.openrobotics.org/u/jarbay51)\
**Post date:** [September 9, 2024, 11:44am UTC](https://discourse.openrobotics.org/t/google-summer-of-code-2024-zenoh-support-benchmarking/39526/5 "2024-09-09T11:44:24Z")

</div>

> [@Timple](#):
>
> May I ask what the origin of these numbers are?

UDP relavant kernel settings come from [Apex AI’s tuning suggestions (pdf)](https://roscon.ros.org/2018/presentations/ROSCon2018_MiddlewarePerformanceTesting.pdf)

TCP relavant kernel settings come from [the discussion at Zenoh Community’s discord server](https://discord.com/channels/914168414178779197/1260373251079536680) rmw\_zenoh uses TCP in default.

But of course i’m open to new suggestions. I will be so appreciated for new suggestions.