# Announcing the Hardware Acceleration WG, meeting #1

**URL:** <https://discourse.openrobotics.org/t/announcing-the-hardware-acceleration-wg-meeting-1/20826>\
**Category:** ROS General\
**Tags:** ros2, hardware, fpga, wg-acceleration, gpu\
**Created:** [June 8, 2021, 6:24am UTC](https://discourse.openrobotics.org/t/announcing-the-hardware-acceleration-wg-meeting-1/20826 "2021-06-08T06:24:20Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![vmayoral](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/vmayoral/32/3665_2.png) [@vmayoral](https://discourse.openrobotics.org/u/vmayoral)\
**Post date:** [June 8, 2021, 6:24am UTC](https://discourse.openrobotics.org/t/announcing-the-hardware-acceleration-wg-meeting-1/20826/1 "2021-06-08T06:24:20Z")

</div>

Dear all,

For the past month we registered interest on [Hardware Acceleration within the ROS 2](http://discourse.ros.org/t/proposal-for-ros-2-hardware-acceleration-working-group-hawg/20112) community. Thanks all those that showed support or contacted directly! We are excited to see that this topic has raised so much interest and we at [Xilinx](https://xilinx.com) are motivated to push forward hardware acceleration in robotics and contribute in the open with tools and accelerators that impact ROS 2 and Gazebo flows. For all those that asked, our reference hardware platform will initially be the [Kria K26 Adaptive SOM](https://www.xilinx.com/products/som/kria.html), with its reference carrier board for vision and perception.

If you’d like to stay more tuned to our progress and be notified of meetings, join the Google Group we’ve created. Also, you could check out the information below or simply stay tuned to the wg-acceleration tag:

> **More details about the Hardware Acceleration WG (HAWG)**
>
> - **Objective** : Drive creation, maintenance and testing of acceleration kernels on top of open standards (C++ and OpenCL) for optimized ROS 2 and Gazebo interactions over different compute substrates (including FPGAs, GPUs and ASICs).
> - **Meeting invite group** : [ROS 2 Hardware Acceleration WG Google Group](https://groups.google.com/g/ros-2-hardware-acceleration-wg)
> - **Github organization** : [ros-acceleration](https://github.com/ros-acceleration)
> - **Meeting minutes** : [HAWG minutes](https://docs.google.com/document/d/185Cy1xjpAOgJygEOnlf5OCgOQTywmF0qgSpS3GiW16Q/edit?usp=sharing)
> - **Discourse tag** : [wg-acceleration](http://discourse.ros.org/tag/wg-acceleration)
> - **Reference hardware platform** : [Kria K26 Adaptive SOM](https://www.xilinx.com/products/som/kria.html)

With all this, we’re happy to call for the first official WG meeting:

#### Meeting #1

Come join us during the first meeting of the ROS 2 Hardware Acceleration Working Group. The plan for this first meeting is to ==demonstrate current hardware acceleration flows with ROS 2==. We also would like to discuss other hardware acceleration efforts and see how all this can be brought together. For the initial demonstrations we will focus on the following preliminar architecture we’ve been working on to bring acceleration natively to the ROS 2 build system (`ament`) and meta build tools (`colcon`). This should facilitate ROS 2 centric acceleration flows using both C++ and/or OpenCL:

```auto
        ROS 2 stack HAWG @ ROS 2 stack

+------------------------+ +--------------------+
| | | xilinx_examples |
| user land | +-------------------+-----------+-------+--------------+
| | | Drivers | Libraries | Cloud |
+------------------------+ +---------------+---+--------+----------+--------------+
| | | ament_vitis | ament_rocm | | accel_fw |
| | +---------------+----------+-+----------+--------------+
| ROS tooling | | ament_acceleration | colcon_accel | accel_fw |
| | +------------------------------------------------------+
| | | build system | meta build | firmware |
+------------------------+ +--------------------------+--------------+------------+
| ROS client library |
+------------------------+
| ROS middleware iface |
+------------------------+
| middleware impl. adapt |
+------------------------+
| |
| middleware impl. |
| |
| |
+------------------------+

```

We’d love to hear about your acceleration projects so feel free to drop here a line of what you’d like to share with the group and we’ll coordinate time slots, either in this meeting or in follow up ones.

The fist meeting has been originally scheduled to be 30 minutes at a friendly time in America, late afternoon in Europe and horribly bad time in Asia (sorry for that). We’ll be rotating friendly-hours between America and Asia if anyone requests us to do so.

- Time: 2021-06-30T18:00:00Z.

- 

> **Coordinates: Zoom**
>
> - Phone one-tap: US: +17209289299,99299967182#,0#,8504098917# or +19292056099,99299967182#,0#,8504098917#
> - Meeting URL: [Launch Meeting - Zoom](https://xilinx.zoom.us/j/99299967182?pwd=S2VCTS9TWnljSFhWeTBsNnZSWjJFdz09&from=addon)
> - Meeting ID: 992 9996 7182
> - Passcode: Xk.X73&rNY

- Preliminary agenda:

---

<div class="post-metadata">

**Author:** ![peterpolidoro](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/peterpolidoro/32/2897_2.png) [@peterpolidoro](https://discourse.openrobotics.org/u/peterpolidoro)\
**Post date:** [June 8, 2021, 2:08pm UTC](https://discourse.openrobotics.org/t/announcing-the-hardware-acceleration-wg-meeting-1/20826/2 "2021-06-08T14:08:49Z")

</div>

Very exciting! That is also a nice looking board. It looks like the global shortages have caused it to have a really long lead time. I will order some now so I can try it out in a few months.

---

<div class="post-metadata">

**Author:** ![erlingrj](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/erlingrj/32/10450_2.png) [@erlingrj](https://discourse.openrobotics.org/u/erlingrj)\
**Post date:** [June 8, 2021, 2:51pm UTC](https://discourse.openrobotics.org/t/announcing-the-hardware-acceleration-wg-meeting-1/20826/3 "2021-06-08T14:51:47Z")

</div>

Cool. I am very interested in participating but I am away for summer vacation. Will you record the meeting? I have not used ROS a lot but its extensively used here at my uni and I think easy integration with ROS could mean that more projects would consider using SoC FPGAs. I am very curios to hear what you guys over at Xilinx are thinking

---

<div class="post-metadata">

**Author:** ![vmayoral](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/vmayoral/32/3665_2.png) [@vmayoral](https://discourse.openrobotics.org/u/vmayoral)\
**Post date:** [June 8, 2021, 3:12pm UTC](https://discourse.openrobotics.org/t/announcing-the-hardware-acceleration-wg-meeting-1/20826/4 "2021-06-08T15:12:13Z")

</div>

> [@erlingrj](#):
>
> Will you record the meeting?

We will @erlingrj, or at least I’ll do my best on that end 🙂 . Minutes will also be available afterwards.

> [@erlingrj](#):
>
> I think easy integration with ROS could mean that more projects would consider using SoC FPGAs. I am very curios to hear what you guys over at Xilinx are thinking

We are of the same opinion. Moreover, as discussed at [http://discourse.ros.org/t/challenges-of-gpu-acceleration-in-ros/19297/12:](http://discourse.ros.org/t/challenges-of-gpu-acceleration-in-ros/19297/12:)

> ROS 2 users have additional priorities computational graph-wise which should be considered. E.g. it appears to be quite a _hot topic_ to optimize interactions betwen nodes (inter-process, intra-process and even over the network). Accelerators on this end will add significant value to the overall ROS 2 computational graphs (and even to the underlying data layer graphs, which at the end of the day, matter as most in real deployments). In other words, while _it’d be great to have OpenCL kernels to accelerate specific computations, real impact won’t be achieved unless a holistic ROS view is applied_ (bottlenecks identified, etc.)

Real acceleration would require optimizations not just at the application layer itself, but also at the ROS underlayers so that they don’t impose bottlenecks on the computational graphs. That’s why we’ve set this as a key objective first.

As we mentioned in the past, our effort is also to fight against the _Not Invented Here syndrome_, which is a big thing in the acceleration/semiconductors world. Big silicon vendors are too self-centric and rather than contributing to expert communities, prefer to reinvent the wheel with _their yet-not-new-feature-in-acceleration-language-A_ or _a new GUI-for-robot-simulation-that-adds-nothing-but-looks-B_ . Our team at Xilinx wants to tackle this and contribute to the ROS 2 community directly, connecting to real ROS users.

I’d encourage you to participate in the future and send to us what you’re working on, so that we consider your needs.

---

<div class="post-metadata">

**Author:** ![Pedro](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/pedro/32/10455_2.png) [@Pedro](https://discourse.openrobotics.org/u/Pedro)\
**Post date:** [June 8, 2021, 4:51pm UTC](https://discourse.openrobotics.org/t/announcing-the-hardware-acceleration-wg-meeting-1/20826/5 "2021-06-08T16:51:53Z")

</div>

Hi,  
I would like to add the [Ultra96-v2](https://www.avnet.com/wps/portal/us/products/new-product-introductions/npi/aes-ultra96-v2/) board to the list of community supported boards, because is popular (so more people could contribute), not so expensive, and is readily available so we can start working with it. What do you think?

regards,  
Pedro

---

<div class="post-metadata">

**Author:** ![vmayoral](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/vmayoral/32/3665_2.png) [@vmayoral](https://discourse.openrobotics.org/u/vmayoral)\
**Post date:** [June 9, 2021, 8:49am UTC](https://discourse.openrobotics.org/t/announcing-the-hardware-acceleration-wg-meeting-1/20826/6 "2021-06-09T08:49:01Z")

</div>

Hola @Pedro 🙂 !

I think this is an awesome idea. I made a section for this at the [community repo](https://github.com/ros-acceleration/community#adding-your-board) of the WG and will add your request to the Agenda above. We’ll allocate a few minutes and discuss this during the meeting.

I don’t have an Ultra96-v2 with me so I won’t be able to help much debugging, but I purposely architected things so that extending it to a new board is all about creating another custom `acceleration_firmware` branch with the firmware artifacts for that specific board (`accel_fw` in the following diagram):

```auto
               HAWG @ ROS 2 stack

           +--------------------+
           | xilinx_examples |
+-------------------+-----------+-------+--------------+
| Drivers | Libraries | Cloud |
+---------------+---+--------+-------------------------+
| ament_vitis | ament_rocm | | accel_fw |
+---------------+----------+-+----------+-+------------+
| ament_acceleration | colcon_accel | accel_fw |
+------------------------------------------------------+
| build system | meta build | firmware |
+--------------------------+--------------+------------+

```

All of this will be detailed in the repo. If done properly, then `ament_acceleration` derived packages (e.g. `ament_vitis`) should pick all the artifacts automatically and connect it with the ROS build system and tools so that you’d only need to invoke `colcon build` with a _mixin_, that’ll be it. We sent these software components to legal for open sourcing them and they’ll be out as soon as we get the approvals. I’ll be demonstrating this in the first WG meeting. Also, I’ll make sure to detail the process of porting things to another board.

For new developments, **I’d highly encourage to stick to the official reference hardware platform ([K26](https://www.xilinx.com/products/som/kria.html))** since that’s what we’ll be testing and where all the features will be confirmed to work and available.

If there’s anyone else interested in doing something similar, please speak out!

---

<div class="post-metadata">

**Author:** ![jopequ](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/jopequ/32/18206_2.png) [@jopequ](https://discourse.openrobotics.org/u/jopequ)\
**Post date:** [June 9, 2021, 9:47am UTC](https://discourse.openrobotics.org/t/announcing-the-hardware-acceleration-wg-meeting-1/20826/7 "2021-06-09T09:47:29Z")

</div>

I will try to get a KV260 as soon as they are available but I would also like to have support for the Ultra96-v2. We have several of those that we use for teaching in a couple master courses.  
In general this looks really interesting! Thanks for the effort in pushing this forward @vmayoral

---

<div class="post-metadata">

**Author:** ![vmayoral](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/vmayoral/32/3665_2.png) [@vmayoral](https://discourse.openrobotics.org/u/vmayoral)\
**Post date:** [June 9, 2021, 10:11am UTC](https://discourse.openrobotics.org/t/announcing-the-hardware-acceleration-wg-meeting-1/20826/8 "2021-06-09T10:11:21Z")

</div>

Noted, thanks @jopequ!

---

<div class="post-metadata">

**Author:** ![Joe\_Dinius](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/joe_dinius/32/6797_2.png) [@Joe\_Dinius](https://discourse.openrobotics.org/u/Joe_Dinius)\
**Post date:** [June 21, 2021, 6:33pm UTC](https://discourse.openrobotics.org/t/announcing-the-hardware-acceleration-wg-meeting-1/20826/9 "2021-06-21T18:33:32Z")

</div>

I second @jopequ’s recommendation to support the Ultra96-v2 SoM as well. The board uses the UltraScale+ architecture (same as the KV260, I believe) and has a form-factor that can more easily be integrated with lower SWAP platforms (such as quadrotors or smaller-scale fixed wing UAVs).

---

<div class="post-metadata">

**Author:** ![victor\_sillerico](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/victor_sillerico/32/10536_2.png) [@victor\_sillerico](https://discourse.openrobotics.org/u/victor_sillerico)\
**Post date:** [June 21, 2021, 11:39pm UTC](https://discourse.openrobotics.org/t/announcing-the-hardware-acceleration-wg-meeting-1/20826/10 "2021-06-21T23:39:50Z")

</div>

Great news!. It is great to know all the possibilities that Hardware Accelerations platforms will bring to the ROS 2 community.

---

<div class="post-metadata">

**Author:** ![uayucar](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/uayucar/32/10832_2.png) [@uayucar](https://discourse.openrobotics.org/u/uayucar)\
**Post date:** [June 25, 2021, 1:23pm UTC](https://discourse.openrobotics.org/t/announcing-the-hardware-acceleration-wg-meeting-1/20826/11 "2021-06-25T13:23:07Z")

</div>

Really interesting! Accelerated hardware towards robotic system optimization will lead to a wide variety of complex computational implementation. Concepts, such as cybersecurity could clearly be benefited by this kind of technology! Looking forward to the WG.

---

<div class="post-metadata">

**Author:** ![mamerlan](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/mamerlan/32/11472_2.png) [@mamerlan](https://discourse.openrobotics.org/u/mamerlan)\
**Post date:** [June 30, 2021, 6:33am UTC](https://discourse.openrobotics.org/t/announcing-the-hardware-acceleration-wg-meeting-1/20826/12 "2021-06-30T06:33:48Z")

</div>

Hi @vmayoral,

Great news! We are very interested in knowing more about the Hardware Acceleration WG initiatives.  
We are hosting the Embedded WG focused on micro-ROS features and use cases, and for sure there will be simliar nature interests within the ecosystem.

Happy to read that the meeting will be recorded!

---

<div class="post-metadata">

**Author:** ![vmayoral](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/vmayoral/32/3665_2.png) [@vmayoral](https://discourse.openrobotics.org/u/vmayoral)\
**Post date:** [June 30, 2021, 7:26am UTC](https://discourse.openrobotics.org/t/announcing-the-hardware-acceleration-wg-meeting-1/20826/13 "2021-06-30T07:26:46Z")

</div>

Thanks @mamerlan, meeting is happening today. I know it’s a bit late in Spain but provided I don’t mess up out of being tired, recordings should be out soon after the meeting. We’ll keep minutes nevertheless 🙂 .

As for alignment with Embedded WG, I’m sure you know I’m quite tuned to micro-ROS. I look forward to cooperate. Here’re some early thoughts for alignment between the HAWG and the EWG:

- it’d be interesting to run micro-ROS Client in either the R5s or into a soft-core (e.g. in a [MicroBlaze](https://www.xilinx.com/products/design-tools/microblaze.html))
- it’d be interesting to offload the agent into Programmable Logic (the FPGA)
  - which obviously leads to consider pushing part of FastDDS also to the PL

We’ve got a few more ideas involving other architectures for mixed-criticality and link layers that allow for real-time distributed communications (e.g. TSN).

---

<div class="post-metadata">

**Author:** ![vmayoral](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/vmayoral/32/3665_2.png) [@vmayoral](https://discourse.openrobotics.org/u/vmayoral)\
**Post date:** [July 2, 2021, 4:49am UTC](https://discourse.openrobotics.org/t/announcing-the-hardware-acceleration-wg-meeting-1/20826/14 "2021-07-02T04:49:00Z")

</div>

Thanks everyone for a fantastic first meeting!

We had a successful kick-off of the ROS 2 Hardware Acceleration Working Group where we discussed how we can make **ROS 2 faster** through FPGAs and GPUs **with open technologies** , initially focusing on C++ and OpenCL. We registered **27 participants initially** and reached more than 30 (I’m being told) during the course of the meeting. This was a first for me in a WG kick-off session. We are very excited and thankful for the interest and the support messages received. Looking forward to continue contributing!

Here’s a summary of the resources discussed:

- [Minutes](https://docs.google.com/document/d/185Cy1xjpAOgJygEOnlf5OCgOQTywmF0qgSpS3GiW16Q/edit#)
- [ros-acceleration](https://github.com/ros-acceleration) Github organization
- [KV260 kit](https://www.xilinx.com/products/som/kria/kv260-vision-starter-kit.html) as reference hardware platform
- **source code** contributed:
  - [ament\_vitis](https://github.com/ros-acceleration/ament_vitis), CMake macros and utilities to include Vitis platform into the ROS 2 build system (ament) and its development flows.
  - [Xilinx Runtime (XRT)](https://github.com/Xilinx/XRT), an open-source standardized software interface that facilitates communication between the application code and the accelerated-kernels

- and finally, the video recording the session:

[![](https://us1.discourse-cdn.com/flex022/uploads/ros/original/3X/5/e/5ef33041be98befcae0f61e77139506bdbcaf15a.jpeg "ROS 2 Hardware Acceleration Working Group - meeting #1") ](https://www.youtube.com/watch?v=QfRHJgeSAOw)

For those of you that don’t have the time to watch the whole piece, see below for a chapter-ed split:

[0:00](https://www.youtube.com/watch?v=QfRHJgeSAOw&t=0s) Introduction[  
12:25](https://www.youtube.com/watch?v=QfRHJgeSAOw&t=745s) Objectives, rationale and hardware reference platforms[  
16:40](https://www.youtube.com/watch?v=QfRHJgeSAOw&t=1000s) Initial hardware acceleration architecture[  
22:10](https://www.youtube.com/watch?v=QfRHJgeSAOw&t=1330s) Short demonstration[  
26:44](https://www.youtube.com/watch?v=QfRHJgeSAOw&t=1604s) Community hardware platforms[  
29:48](https://www.youtube.com/watch?v=QfRHJgeSAOw&t=1788s) Q&A[  
43:00](https://www.youtube.com/watch?v=QfRHJgeSAOw&t=2580s) Final remarks

We’ll be following up shortly on a number of the topics discussed over the call. Stay tuned for the next meeting and enjoy summer time 🏖 🌞 !

---

<div class="post-metadata">

**Author:** ![vmayoral](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/vmayoral/32/3665_2.png) [@vmayoral](https://discourse.openrobotics.org/u/vmayoral)\
**Post date:** [July 5, 2021, 8:28pm UTC](https://discourse.openrobotics.org/t/announcing-the-hardware-acceleration-wg-meeting-1/20826/15 "2021-07-05T20:28:18Z")

</div>

As a quick update, following the community ask, I ordered today an `Ultra96-v2` board from [Farnell](https://www.farnell.com/) only to find that the lead time is 16 weeks 🙁!

@Pedro, @jopequ, @Joe_Dinius, do any of you have any advice for me on how to get my hands into one faster?

---

<div class="post-metadata">

**Author:** ![Joe\_Dinius](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/joe_dinius/32/6797_2.png) [@Joe\_Dinius](https://discourse.openrobotics.org/u/Joe_Dinius)\
**Post date:** [July 5, 2021, 10:39pm UTC](https://discourse.openrobotics.org/t/announcing-the-hardware-acceleration-wg-meeting-1/20826/16 "2021-07-05T22:39:03Z")

</div>

Looks like it is in stock at Avnet - [https://www.avnet.com/shop/us/products/avnet-engineering-services/aes-ultra96-v2-g-3074457345638646173/](https://www.avnet.com/shop/us/products/avnet-engineering-services/aes-ultra96-v2-g-3074457345638646173/)

---

<div class="post-metadata">

**Author:** ![vmayoral](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/vmayoral/32/3665_2.png) [@vmayoral](https://discourse.openrobotics.org/u/vmayoral)\
**Post date:** [July 6, 2021, 6:39am UTC](https://discourse.openrobotics.org/t/announcing-the-hardware-acceleration-wg-meeting-1/20826/17 "2021-07-06T06:39:19Z")

</div>

Thanks @Joe_Dinius, I was hoping to avoid that since I’m in Europe … but I found no better option so that’ll do I guess. Will cancell my order with Farnell and go with Avnet US.

In case someone bumps on it as well, there seem to be also some delays reported. Hopefully it’s just that:

 ![Captura de pantalla 2021-07-06 a las 8.35.39](https://us1.discourse-cdn.com/flex022/uploads/ros/original/2X/2/2cb606316145c9f1681752c041f2ad2d0aa4bc88.png)

---

<div class="post-metadata">

**Author:** ![jopequ](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/jopequ/32/18206_2.png) [@jopequ](https://discourse.openrobotics.org/u/jopequ)\
**Post date:** [July 6, 2021, 9:43am UTC](https://discourse.openrobotics.org/t/announcing-the-hardware-acceleration-wg-meeting-1/20826/18 "2021-07-06T09:43:22Z")

</div>

We got the last batch to Finland from Farnell in December last year so no experience with other sources, sorry 😕

---

<div class="post-metadata">

**Author:** ![YangZ](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/yangz/32/3341_2.png) [@YangZ](https://discourse.openrobotics.org/u/YangZ)\
**Post date:** [July 6, 2021, 10:32am UTC](https://discourse.openrobotics.org/t/announcing-the-hardware-acceleration-wg-meeting-1/20826/19 "2021-07-06T10:32:12Z")

</div>

`Ultra96-v2` is a flagship 96Boards - let me know if you need help so I can forward you to Xilinx and Avnet folks directly.

For full disclosure, I am responsible for 96Boards specifications which `Ultra96-v2` is built to be compliant.

Yang

---

<div class="post-metadata">

**Author:** ![vmayoral](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/vmayoral/32/3665_2.png) [@vmayoral](https://discourse.openrobotics.org/u/vmayoral)\
**Post date:** [July 6, 2021, 1:05pm UTC](https://discourse.openrobotics.org/t/announcing-the-hardware-acceleration-wg-meeting-1/20826/20 "2021-07-06T13:05:09Z")

</div>

@YangZ having someone from Avnet look at this would be awesome. Specially, I think they’ll benefit a lot if they could commit some engineering resources 😉 . We at Xilinx are already 🙂 doing so.

PM me if you need further details.

[Next page](https://discourse.openrobotics.org/t/announcing-the-hardware-acceleration-wg-meeting-1/20826.md?page=2)