# Edge AI Working Group

**URL:** <https://discourse.openrobotics.org/t/edge-ai-working-group/12011>\
**Category:** ROS General\
**Tags:** ros2, wg-edgeai\
**Created:** [December 19, 2019, 4:53am UTC](https://discourse.openrobotics.org/t/edge-ai-working-group/12011 "2019-12-19T04:53:43Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![joespeed](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/joespeed/32/12133_2.png) [@joespeed](https://discourse.openrobotics.org/u/joespeed)\
**Post date:** [December 19, 2019, 4:53am UTC](https://discourse.openrobotics.org/t/edge-ai-working-group/12011/1 "2019-12-19T04:53:43Z")

</div>

**Edge AI Working Group** is **launched**. We welcome your contribution this effort.

**Objective** : make Edge AI easier and ubiquitous in ROS 2

**Overview** : Edge AI, specifically ML has important applications in ROS 2. For example navigation, perception & picking, inspection, motion planning. We’ll work to integrate and support technology for ML. We’ll approach this via a process of assessing the current state and working to make it easier for ROS 2 users.

The WG seeks to do the following for ROS 2:

1. perform a survey of ML use in ROS and ROS 2
2. identify gaps and build a roadmap to close them
3. make it easier to use machine learning in ROS 2
4. enable machine learning on embedded processors found in typical mobile robots
5. enable HW acceleration of ML when present e.g. CPU SIMD, GPUs, FPGAs

Edge AI WG:

- Joe Speed @joespeed ADLINK Technology (WG leader, contributor)
- Steve Macenski @smac Samsung (contributor)
- Geoff Biggs @gbiggs (will assign) TierIV (contributor)
- Aaron Blasdel @Aaron_Blasdel Amazon (member)
- Harold Yang @hyang5, Lewis Liu @LewisLiuPub Intel (contributors)
- Katherine Scott @Katherine_Scott Open Robotics (contributor)
- Christoph Hellmann @c_h_s Fraunhofer IPA (contributor)
- Sumandeep Banerjee @sumandeepb Rapyuta Robotics, Tokyo (contributor)
- Zhen Ju @crystaldust Huawei (contributor)
- Amit Goel amitgoel NVIDIA (member)
- Bob A boba (contributor or member?)
- Alex Tyshka atyshka (contributor or member?)
- Adrian Bedford Adrian\_Bedford (contributor or member?)
- Jeremy Adams adamsj Object Computing (contributor or member?)

Edge AI WG [agenda and meeting minutes](https://docs.google.com/document/d/1z8KbqILH6ql5RkYl4jiVlphuOnmltetE0mYZcw9wYSA/edit?usp=sharing)

Please list packages that make use of ML here [ROS 2 packages using ML - ROS General - Open Robotics Discourse](http://discourse.ros.org/t/ros-2-packages-using-ml/13137)

---

<div class="post-metadata">

**Author:** ![mkhansen](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/mkhansen/32/3748_2.png) [@mkhansen](https://discourse.openrobotics.org/u/mkhansen)\
**Post date:** [December 19, 2019, 5:35pm UTC](https://discourse.openrobotics.org/t/edge-ai-working-group/12011/2 "2019-12-19T17:35:05Z")

</div>

Hi Joe,  
I’ll check internally, but I believe Intel should be able to attend / contribute also.

I’ll let you know. I assume this would kick off in January 2020?

---

<div class="post-metadata">

**Author:** ![smac](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/smac/32/6763_2.png) [@smac](https://discourse.openrobotics.org/u/smac)\
**Post date:** [December 19, 2019, 6:39pm UTC](https://discourse.openrobotics.org/t/edge-ai-working-group/12011/3 "2019-12-19T18:39:24Z")

</div>

> [@joespeed](#):
>
> CPU

Something I’m really interested by, and hopefully something Intel can help with, is an apples-to-apples comparison of a few classes of machine learning models with OpenVino-optimized model, raw CPU, and TensorRT-optimized models on a comparable Nvidia GPU (which, you know is a question of itself. But I have a Jetson Nano, a TX2, and I’m sure I could get access to a Xavier if we wanted to run the gambit).

In particular, starting from a common basis tensorflow model, instructions for how to run, instructions for how to optimize with openvino, instructions on how to optimize with TenorRT (which I can provide). This way you have a full document for “hey I trained this model, and now I want to deploy it on [all the major options]” and as the instructions we can use to reproducably get test results for different classes of models.

Now what classes of models am I interested in?

- Some robotics-capable CNN detection model. Preference for SSD mobilenet V1/2 or inception. This class of model is typical and will give a good basis for DL solutions that are generally capable of running on the edge on Nvidia hardware, but I’d love to know if I can run int on intel CPUs as well.
- Some lower dimensional detection or classification model. Think: IMU, laser scan, or narrow/shallow CNN. This class of model is interesting to test for simplier problem sets, how much the optimizations really help, and again with an eye for “can I run this on my i5 CPU tractably” without needing a separate GPU unit.
- [other] I’m sure there’s another good extreme to test here with other types of machine learning interests.

This is a really valuable investigation but perhaps not _directly_ ROS related

---

<div class="post-metadata">

**Author:** ![Katherine\_Scott](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/katherine_scott/32/4434_2.png) [@Katherine\_Scott](https://discourse.openrobotics.org/u/Katherine_Scott)\
**Post date:** [December 19, 2019, 8:54pm UTC](https://discourse.openrobotics.org/t/edge-ai-working-group/12011/4 "2019-12-19T20:54:29Z")

</div>

Could someone explain to me what the difference is between “Edge AI” and perception / computer vision? How would this be different than a perception working group? I would be more than glad to help out but I would prefer we keep the buzz words to a minimum.

---

<div class="post-metadata">

**Author:** ![smac](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/smac/32/6763_2.png) [@smac](https://discourse.openrobotics.org/u/smac)\
**Post date:** [December 19, 2019, 10:14pm UTC](https://discourse.openrobotics.org/t/edge-ai-working-group/12011/5 "2019-12-19T22:14:32Z")

</div>

> [@Katherine\_Scott](#):
>
> I would prefer we keep the buzz words to a minimum

I think its just buzz words and the specific application of AI-concepts on embedded vs cloud hardware (CPU, GPU, FPGA, AI accelerator…). I think the perception working group would be more higher level to perception tasks, which may also include some AI-concepts but also more general perception frameworks/SLAM/etc.

---

<div class="post-metadata">

**Author:** ![joespeed](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/joespeed/32/12133_2.png) [@joespeed](https://discourse.openrobotics.org/u/joespeed)\
**Post date:** [December 20, 2019, 2:31am UTC](https://discourse.openrobotics.org/t/edge-ai-working-group/12011/6 "2019-12-20T02:31:52Z")

</div>

Great! Yes, the intent is to kick off in January 2020

---

<div class="post-metadata">

**Author:** ![atyshka](https://avatars.discourse-cdn.com/v4/letter/a/7feea3/32.png) [@atyshka](https://discourse.openrobotics.org/u/atyshka)\
**Post date:** [December 21, 2019, 12:22am UTC](https://discourse.openrobotics.org/t/edge-ai-working-group/12011/7 "2019-12-21T00:22:21Z")

</div>

@joespeed One suggestion for topic of discussion would be better support for GPU builds of software. I recall frustration when trying to use OpenCV compiled with CUDA for ROS, as many packages depend on it and they all would have to be build from source. Similarly, PCL offers limited CUDA functionality but it runs into the same source-compilation issues as OpenCV. If we want Edge AI to be a big part of ROS2, we need better support for GPU acceleration, and CUDA is where a large part of the community is at right now. GPU opens up so many possibilities for ROS and it’s currently underutilized. I might like to see alternate builds of binaries with CUDA support. I understand such an endorsement of proprietary software might be a controversial step, but it would be immensely useful.

---

<div class="post-metadata">

**Author:** ![smac](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/smac/32/6763_2.png) [@smac](https://discourse.openrobotics.org/u/smac)\
**Post date:** [December 21, 2019, 2:17am UTC](https://discourse.openrobotics.org/t/edge-ai-working-group/12011/8 "2019-12-21T02:17:08Z")

</div>

@atyshka sounds like something you’re passionate about! You should join the working group and help make that a reality!

---

<div class="post-metadata">

**Author:** ![joespeed](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/joespeed/32/12133_2.png) [@joespeed](https://discourse.openrobotics.org/u/joespeed)\
**Post date:** [December 21, 2019, 2:32am UTC](https://discourse.openrobotics.org/t/edge-ai-working-group/12011/9 "2019-12-21T02:32:07Z")

</div>

@atyshka CUDA libraries are proprietary but endemic. Doing something about it maybe fits in #5. @smac is right, you can be part of the solution😁

---

<div class="post-metadata">

**Author:** ![boba](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/boba/32/19111_2.png) [@boba](https://discourse.openrobotics.org/u/boba)\
**Post date:** [December 21, 2019, 7:27am UTC](https://discourse.openrobotics.org/t/edge-ai-working-group/12011/10 "2019-12-21T07:27:10Z")

</div>

Hello, I would be interested in contributing to this effort. My background is on the FPGA embedded systems side, and I am working on a basic Ultra96 project to this end.

Thanks,

Bob Anderson

---

<div class="post-metadata">

**Author:** ![LewisLiuPub](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/lewisliupub/32/1341_2.png) [@LewisLiuPub](https://discourse.openrobotics.org/u/LewisLiuPub)\
**Post date:** [January 3, 2020, 3:11am UTC](https://discourse.openrobotics.org/t/edge-ai-working-group/12011/11 "2020-01-03T03:11:09Z")

</div>

> [@smac](#):
>
> Some robotics-capable CNN detection model. Preference for SSD mobilenet V1/2 or inception. This class of model is typical and will give a good basis for DL solutions that are generally capable of running on the edge on Nvidia hardware, but I’d love to know if I can run int on intel CPUs as well.

These features have been supported by [this project](https://github.com/intel/ros2_openvino_toolkit/tree/devel).

- It supports ROS and ROS 2 frameworks.
- It supports Intel OpenVINO and encapsules the utility of openCV.
- It supports Intel CPU / GPU / FPGA. (FPGA is not fully tested although).
- It support SSD mobilenet, YovoV2, master RCNN and other Intel trained models and public pre-trained modles.

I hope our work about ROS/ROS2 OpenVINO Toolkit projects can contribute a bit to this topic.

---

<div class="post-metadata">

**Author:** ![joespeed](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/joespeed/32/12133_2.png) [@joespeed](https://discourse.openrobotics.org/u/joespeed)\
**Post date:** [January 3, 2020, 5:53pm UTC](https://discourse.openrobotics.org/t/edge-ai-working-group/12011/12 "2020-01-03T17:53:38Z")

</div>

@LewisLiuPub loved seeing that work in [ROScon Day 2 talk](https://roscon.ros.org/2019/#day-2-november-1st) and [MoveIt Macau workshop](https://moveit.ros.org/moveit/ros/2019/09/19/moveit-workshop-macau.html), @mkhansen is working on having Intel contribute to Edge AI WG.

---

<div class="post-metadata">

**Author:** ![Adrian\_Bedford](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/adrian_bedford/32/4532_2.png) [@Adrian\_Bedford](https://discourse.openrobotics.org/u/Adrian_Bedford)\
**Post date:** [January 7, 2020, 12:39pm UTC](https://discourse.openrobotics.org/t/edge-ai-working-group/12011/13 "2020-01-07T12:39:44Z")

</div>

Hi Katherine,  
Imagine many autonomous cars all being driven by NNs. Each of these cars could gather vast data sets from all their cameras, lidars and other sensors and transmit this data back to the manufacturer. This would be a prohibitively huge amount of data, but would be useful for training and improving the next generation of NNs for the cars.

If instead, any small improvements, ie changes in the weightings of the NN, were transmitted back, these could be averaged over all the cars, and then the improved weightings could be sent back to the cars.

All the cars will be learning from each other’s experiences. This is just one example.

---

<div class="post-metadata">

**Author:** ![c\_h\_s](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/c_h_s/32/5840_2.png) [@c\_h\_s](https://discourse.openrobotics.org/u/c_h_s)\
**Post date:** [January 20, 2020, 11:52am UTC](https://discourse.openrobotics.org/t/edge-ai-working-group/12011/14 "2020-01-20T11:52:56Z")

</div>

Hi Joe,

We here at Fraunhofer IPA are also interested in the topic of how to deploy AI modules to robotics systems running ROS. Recently, we have started a research project that touches the area.  
I think this can be one major differenciator of ROS in the future compared to other industrial robot systems.

We should certainly also focus on FPGA and neuromorphic chips as this seems to be an upcoming topic.

Best regards  
Christoph

---

<div class="post-metadata">

**Author:** ![crystaldust](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/crystaldust/32/5528_2.png) [@crystaldust](https://discourse.openrobotics.org/u/crystaldust)\
**Post date:** [February 9, 2020, 11:59am UTC](https://discourse.openrobotics.org/t/edge-ai-working-group/12011/15 "2020-02-09T11:59:37Z")

</div>

Great proposal, is there some designs on architecture yet? If possible, I would like to make contributions, we have our smart device which accelerate the deep neural network, hope the arch will adapt multiple devices.

---

<div class="post-metadata">

**Author:** ![joespeed](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/joespeed/32/12133_2.png) [@joespeed](https://discourse.openrobotics.org/u/joespeed)\
**Post date:** [February 19, 2020, 4:21am UTC](https://discourse.openrobotics.org/t/edge-ai-working-group/12011/16 "2020-02-19T04:21:14Z")

</div>

Sounds lovely, happy to include you

---

<div class="post-metadata">

**Author:** ![joespeed](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/joespeed/32/12133_2.png) [@joespeed](https://discourse.openrobotics.org/u/joespeed)\
**Post date:** [February 20, 2020, 5:57am UTC](https://discourse.openrobotics.org/t/edge-ai-working-group/12011/17 "2020-02-20T05:57:45Z")

</div>

Hi Folks, [Please pick the times](http://discourse.ros.org/t/edge-ai-wg-kickoff-call-pick-a-date/12875) that can work for you next week, thanks!  
-Joe

---

<div class="post-metadata">

**Author:** ![sumandeepb](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/sumandeepb/32/5676_2.png) [@sumandeepb](https://discourse.openrobotics.org/u/sumandeepb)\
**Post date:** [February 21, 2020, 10:28am UTC](https://discourse.openrobotics.org/t/edge-ai-working-group/12011/18 "2020-02-21T10:28:16Z")

</div>

Hi @joespeed, great initiative. This in lines of something I have been ideating on in the past few months. I have also jotted down some ideas and approaches to attack this type of task. I am relatively a new comer to ROS but have extensive background in a variety of computer vision / machine learning / deep learning algorithm and product development including academic research, working on mine and other’s startups. I definitely want to contribute and be part of this.

I have worked on Intel Movidius, Jetson, Tegra GPU platforms, sensors such as RealSense, embedded boards such as RasberryPI, Tinkerboard etc. Portability and reusability of implementations and compute hardware / sensor support for widely varying hardware platforms has been a common challenge I have faced and continue to do so. Anything I can do to help the greater community and help speed up the overall pace of work in this domain is very much my intent.

I am currently working on Robotics Perception Problems at Rapyuta Robotics, Tokyo. We are a cloud robotics platform based on ROS. Kindly have a look at my profile [https://www.linkedin.com/in/sumandeep-banerjee-1436a17/](https://www.linkedin.com/in/sumandeep-banerjee-1436a17/)

---

<div class="post-metadata">

**Author:** ![joespeed](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/joespeed/32/12133_2.png) [@joespeed](https://discourse.openrobotics.org/u/joespeed)\
**Post date:** [February 22, 2020, 2:50pm UTC](https://discourse.openrobotics.org/t/edge-ai-working-group/12011/19 "2020-02-22T14:50:35Z")

</div>

[Edge AI WG](https://groups.google.com/forum/#!forum/ros-edge-ai-working-group-invites) kick off will be 2020-02-27T14:00:00Z. Excited to form a plan together and make ROS better!

Here is the [work in progress agenda](https://docs.google.com/document/d/1z8KbqILH6ql5RkYl4jiVlphuOnmltetE0mYZcw9wYSA/edit?usp=sharing), please add your interest, agenda items, what you’d like to do. If long, please give brief description and link(s) to more details.

Please [join the Edge AI WG](https://groups.google.com/forum/#!forum/ros-edge-ai-working-group-invites) (thanks Tully!) to receive the meeting invite. I’ll assume Zoom is OK a’la Real-Time WG, right? It will include a dial-in option but quality is best with Internet audio + headset. Here are two headsets with great mics:

- [$28 Mpow HC6](https://www.amazon.com/gp/product/B081Q89ZT5/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&psc=1) USB + 3.5mm, tight if big headed, isolates your ears from ambient noise
- [$42 Sennheiser SC 60 USB](https://www.amazon.com/gp/product/B00E67UV9S/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&psc=1) used for years, is quite good

 ![image](https://us1.discourse-cdn.com/flex022/uploads/ros/original/2X/6/6fd3bb0af3c0b08209cb34cba95a9579483f0d4e.png)

---

<div class="post-metadata">

**Author:** ![gbiggs](https://sea2.discourse-cdn.com/flex022/user_avatar/discourse.openrobotics.org/gbiggs/32/30361_2.png) [@gbiggs](https://discourse.openrobotics.org/u/gbiggs)\
**Post date:** [February 25, 2020, 6:05am UTC](https://discourse.openrobotics.org/t/edge-ai-working-group/12011/20 "2020-02-25T06:05:09Z")

</div>

Don’t forget to add the meeting to the ROS events calendar so we don’t miss it!

[Next page](https://discourse.openrobotics.org/t/edge-ai-working-group/12011.md?page=2)