# Embodied AI Community Group - Call of Interest
**URL:** https://discourse.openrobotics.org/t/embodied-ai-community-group-call-of-interest/39990
**Category:** ROS General
**Tags:** ros2, working-group, ai
**Created:** [October 9, 2024, 11:57am UTC](https://discourse.openrobotics.org/t/embodied-ai-community-group-call-of-interest/39990 "2024-10-09T11:57:44Z")
**Posts on this page:** 20
**Page:** 1
### Author:  [@adamdbrw](https://discourse.openrobotics.org/u/adamdbrw)
#### Post date: [October 9, 2024, 11:57am UTC](https://discourse.openrobotics.org/t/embodied-ai-community-group-call-of-interest/39990/1 "2024-10-09T11:57:44Z")
Update: the group is launched: [The launch and first meeting of Embodied AI Community Group!](http://discourse.ros.org/t/the-launch-and-first-meeting-of-embodied-ai-community-group/40420)
Dear ROS Community!
Large Language Models, Vision Language Models and AI Agent architectures are recent breakthrough technologies that most of you are familiar with. There are many gaps in Generative AI for robotics such as in spatial and temporal reasoning, dealing with imperfect perception and actuation, performance, size, liability, and so on. However, there are also important strengths and applications that are already quite reasonable, such as in human-robot interaction, high-level planning, inspections, and summaries.
This creates interest in developing robots running an AI Agent, which shows in open-source activity in embodied AI being on the rise, with multiple new projects published or updated recently, including:
- [RAI](https://github.com/RobotecAI/rai) - a novel generic AI Agent framework for ROS 2.
- [ros2ai](https://github.com/fujitatomoya/ros2ai) – command line interface extension, especially for those learning ROS by @tomoyafujita
- [ROSA](https://github.com/nasa-jpl/rosa) - to inspect, diagnose, understand, and operate robots by @RobRoyce
- [embodied-agents](https://github.com/mbodiai/embodied-agents) – to facilitate data collection and sharing for embodied AI researchers.
- [Llama\_ros](https://github.com/mgonzs13/llama_ros), [whisper\_ros](https://github.com/mgonzs13/whisper_ros), .. – ROS 2 integrated llama.cpp model, listening to speech and generating responses.
- [DORA](https://github.com/dora-rs/dora) - streamline and simplify creation of AI based robot applications.
It makes sense to me to align these developments and foster the community around Gen AI for ROS. The benefits are many, from a platform to share experiences with this novel technology and learning from other projects and applications, to collaborating on common goals, new features, and integrations.
My idea would be to lead the group personally and to organize meetings once a month to start with. I would invite speakers to present their projects, share experiences and needs. [Robotec.ai](https://www.robotec.ai/)’s contributions would involve our ongoing efforts with RAI as well as simulation projects and assets to validate embodied AI agents.
Dear ROS community! I would appreciate your expression of interest in the **Embodied AI Community Working Group** and tips on how to organize it in the best way. I would love to learn how you see your participation in the group, how you are applying AI agents to your robots, what kind of contributions you would like to make, and what your goals are!
---
### Author:  [@vmayoral](https://discourse.openrobotics.org/u/vmayoral)
#### Post date: [October 9, 2024, 1:39pm UTC](https://discourse.openrobotics.org/t/embodied-ai-community-group-call-of-interest/39990/2 "2024-10-09T13:39:51Z")
Well done @adamdbrw taking the lead and attempting to unify AI/GenAI efforts. I’d be happy to participate if the WG takes off.
---
### Author:  [@RobRoyce](https://discourse.openrobotics.org/u/RobRoyce)
#### Post date: [October 9, 2024, 2:18pm UTC](https://discourse.openrobotics.org/t/embodied-ai-community-group-call-of-interest/39990/3 "2024-10-09T14:18:09Z")
This is a great idea, and I’d be happy to help build the WG with you.
We’ve been thinking a lot about the ethical implications of embodied agents and see this as one of the major areas for improvement moving forward. It’s certainly not something that should happen in isolation. This WG seems like a good place to build on the standards we’ve established at NASA and to provide the community with solutions that are easy to adopt and grounded in an ethical framework.
Let me know what the timeline looks like and how we can help support!
---
### Author:  [@MiguelSolis](https://discourse.openrobotics.org/u/MiguelSolis)
#### Post date: [October 10, 2024, 6:03pm UTC](https://discourse.openrobotics.org/t/embodied-ai-community-group-call-of-interest/39990/4 "2024-10-10T18:03:33Z")
I am interested in participating.
---
### Author:  [@tomoyafujita](https://discourse.openrobotics.org/u/tomoyafujita)
#### Post date: [October 10, 2024, 6:05pm UTC](https://discourse.openrobotics.org/t/embodied-ai-community-group-call-of-interest/39990/5 "2024-10-10T18:05:47Z")
@adamdbrw thanks for taking a lead on this activity. i will be definitely there!
---
### Author:  [@Katherine\_Scott](https://discourse.openrobotics.org/u/Katherine_Scott)
#### Post date: [October 10, 2024, 6:15pm UTC](https://discourse.openrobotics.org/t/embodied-ai-community-group-call-of-interest/39990/6 "2024-10-10T18:15:02Z")
Hi Adam,
Please let me know when you set a time and place for your first meeting and I’ll help spread the word. I would recommend putting together a Google Form to collect e-mail address so you can notify interested parties and organize a Google Group.
---
### Author:  [@Simone\_Voto](https://discourse.openrobotics.org/u/Simone_Voto)
#### Post date: [October 10, 2024, 7:41pm UTC](https://discourse.openrobotics.org/t/embodied-ai-community-group-call-of-interest/39990/7 "2024-10-10T19:41:13Z")
Ho I’m Simone Voto from Reply. On this topic we are working from 1 year applyed on different robots. We are experimenting robot commanding with LLM. In particolar,now,we are working on using tools to give context from external system,likes RFID,in order to create a task forna robot. Here Is an example
> **[Spotlight on AI Embodied Agents | Reply](https://www.reply.com/en/robotics-and-autonomous-things/spotlight-on-ai-embodied-agents)**
>
> Explore Reply's pioneering AI-embodied agents simplifying robot control, showcased through the Spot case.
Would be great to be part of this group to share our projects and to test new feature on real robots on real cases
---
### Author:  [@azmyin12](https://discourse.openrobotics.org/u/azmyin12)
#### Post date: [October 10, 2024, 10:34pm UTC](https://discourse.openrobotics.org/t/embodied-ai-community-group-call-of-interest/39990/8 "2024-10-10T22:34:44Z")
Hi @adamdbrw interested in participating in this group.
---
### Author:  [@RobRoyce](https://discourse.openrobotics.org/u/RobRoyce)
#### Post date: [October 11, 2024, 1:41am UTC](https://discourse.openrobotics.org/t/embodied-ai-community-group-call-of-interest/39990/9 "2024-10-11T01:41:47Z")
For anyone who’s interested, we have an IEEE pre-print on arXiv that details the ROSA framework as a first of its kind embodied agent that works for both ROS1 and ROS2. We also have an entire section dedicated to ethics for embodied agents and a demonstrations section that shows example usage on multiple physical and simulated robots.
You can see it in action at the JPL Mars Yard on the NeBula-Spot robot:
[ ](https://www.youtube.com/watch?v=mZTrSg7tEsA)
> **[Enabling Novel Mission Operations and Interactions with ROSA: The Robot...](https://arxiv.org/abs/2410.06472v1)**
>
> The advancement of robotic systems has revolutionized numerous industries, yet their operation often demands specialized technical knowledge, limiting accessibility for non-expert users. This paper introduces ROSA (Robot Operating System Agent), an...
[https://github.com/nasa-jpl/rosa](https://github.com/nasa-jpl/rosa)
---
### Author:  [@vanmalleghema](https://discourse.openrobotics.org/u/vanmalleghema)
#### Post date: [October 11, 2024, 5:04pm UTC](https://discourse.openrobotics.org/t/embodied-ai-community-group-call-of-interest/39990/10 "2024-10-11T17:04:07Z")
Hello, I’m clearly interested in ! Thank you for the initiative !
---
### Author:  [@jennifer](https://discourse.openrobotics.org/u/jennifer)
#### Post date: [October 11, 2024, 10:06pm UTC](https://discourse.openrobotics.org/t/embodied-ai-community-group-call-of-interest/39990/11 "2024-10-11T22:06:00Z")
That’s such a great idea! I’m definitely onboard. I’m also happy to support it any way I can.
One idea would be to select one paper or OSS library related to embodied AI per meeting, that everyone can look into before, and which can be discussed as part of the meeting. Better even if there is one volunteer to recap the content as introduction. Just as idea.
Looking forward to it, let me know how I can support it.
Jennifer
> **[Jennifer Buehler - Ludwigsburg, Baden-Württemberg, Germany | Professional...](https://www.linkedin.com/in/jenbuehler)**
>
> Location: Ludwigsburg · 500+ connections on LinkedIn. View Jennifer Buehler’s profile on LinkedIn, a professional community of 1 billion members.
---
### Author:  [@ycheng517](https://discourse.openrobotics.org/u/ycheng517)
#### Post date: [October 13, 2024, 12:39am UTC](https://discourse.openrobotics.org/t/embodied-ai-community-group-call-of-interest/39990/12 "2024-10-13T00:39:22Z")
Hi, I’m interested in participating!
---
### Author:  [@mgonzs13](https://discourse.openrobotics.org/u/mgonzs13)
#### Post date: [October 14, 2024, 8:10am UTC](https://discourse.openrobotics.org/t/embodied-ai-community-group-call-of-interest/39990/13 "2024-10-14T08:10:00Z")
Hi @adamdbrw thanks for taking the lead, I am interested in participating 😀. I have several AI ROS 2 packages, not only [llama\_ros](https://github.com/mgonzs13/llama_ros) and [whisper\_ros](https://github.com/mgonzs13/whisper_ros), but also [tts\_ros](https://github.com/mgonzs13/tts_ros) and [yolov8\_ros](https://github.com/mgonzs13/yolov8_ros). We also have a [chatbot\_ros](https://github.com/mgonzs13/chatbot_ros), controlled by an FSM created with [YASMIN](https://github.com/uleroboticsgroup/yasmin), that uses llama-3.2-1B, whisper-large-turbo and tts. For anyone interested in running LLM locally in ROS 2, here you have a demo of LangChain RAG within ROS 2. We are launching the embedding model, to create the vector database; the reranking model; and the llama-3.2-1B model using llama\_ros.
>
---
### Author:  [@biancab](https://discourse.openrobotics.org/u/biancab)
#### Post date: [October 14, 2024, 8:37am UTC](https://discourse.openrobotics.org/t/embodied-ai-community-group-call-of-interest/39990/14 "2024-10-14T08:37:08Z")
Hi @adamdbrw. I am interested in participating in the WG. Nice initiative!
---
### Author:  [@istvan-fodor](https://discourse.openrobotics.org/u/istvan-fodor)
#### Post date: [October 14, 2024, 2:07pm UTC](https://discourse.openrobotics.org/t/embodied-ai-community-group-call-of-interest/39990/15 "2024-10-14T14:07:11Z")
Hi @adamdbrw , interested in this, keep us looped in please!
---
### Author:  [@adamdbrw](https://discourse.openrobotics.org/u/adamdbrw)
#### Post date: [October 14, 2024, 4:55pm UTC](https://discourse.openrobotics.org/t/embodied-ai-community-group-call-of-interest/39990/16 "2024-10-14T16:55:58Z")
Thanks everyone for such a great support for the idea! We are definitely moving forward with the Work Group and I am especially happy to get responses from representative of most of the projects in the space.
There is a simple registration form so that we can organize a group and notify everyone (thank you @Katherine_Scott for suggesting): [Embodied AI Working Group registration](https://docs.google.com/forms/d/e/1FAIpQLSdawrRJBTlPPMgQrnmN6DWJ-TkQHsltbRYmwhQJhQXP09ydAA/viewform). This will also allow us to determine the best time slot for calls for all of us.
I also reserved Birds of Feather session for those of us going to ROSCon in Odense, centered around Embodied AI, which will be a great kick-off for us to connect & discuss our common goals for the group. The meeting will be on the 21st at 16:00 in Room 208.
Looking forward to meeting you in person and on-line!
---
### Author:  [@adamdbrw](https://discourse.openrobotics.org/u/adamdbrw)
#### Post date: [November 5, 2024, 9:09am UTC](https://discourse.openrobotics.org/t/embodied-ai-community-group-call-of-interest/39990/17 "2024-11-05T09:09:13Z")
The group is launched!
> [@The launch and first meeting of Embodied AI Community Group!](http://discourse.ros.org/t/the-launch-and-first-meeting-of-embodied-ai-community-group/40420):
>
> I am pleased to announce that we are launching the new Embodied AI Community Group dedicated to the topic of applications of Generative AI to ROS 2 robotics. It is thrilling to see that the initiative is met with a great response from the community both through responses to a call for interest ([http://discourse.ros.org/t/embodied-ai-community-group-call-of-interest/39990](http://discourse.ros.org/t/embodied-ai-community-group-call-of-interest/39990)) and a lively Birds of Feather Embodied AI session at ROSCon 2024 in Odense, it is time to kick things off! A GitHub space,…
---
### Author:  [@Alberto\_Munoz](https://discourse.openrobotics.org/u/Alberto_Munoz)
#### Post date: [November 6, 2024, 5:44pm UTC](https://discourse.openrobotics.org/t/embodied-ai-community-group-call-of-interest/39990/18 "2024-11-06T17:44:35Z")
Great, I am in ! Cheers
---
### Author:  [@Aadi\_Mishra](https://discourse.openrobotics.org/u/Aadi_Mishra)
#### Post date: [November 6, 2024, 5:49pm UTC](https://discourse.openrobotics.org/t/embodied-ai-community-group-call-of-interest/39990/19 "2024-11-06T17:49:34Z")
@adamdbrw That’s a great initiative. I would like to be a part.
---
### Author:  [@Aayush\_Shrestha](https://discourse.openrobotics.org/u/Aayush_Shrestha)
#### Post date: [November 10, 2024, 8:57am UTC](https://discourse.openrobotics.org/t/embodied-ai-community-group-call-of-interest/39990/20 "2024-11-10T08:57:04Z")
Hello Simone, I watched the demo showcasing controlling the Spot robot using natural language. I’m interested to know more about this project in detail. How can I contact you?
[Next page](https://discourse.openrobotics.org/t/embodied-ai-community-group-call-of-interest/39990.md?page=2)