Coming from the wonderful ROS framework, I’d like to get more familiar with ROS2 through a project.
I would like to build a versatile and useful mobile platform using the ESP32-cam module, leveraging the very low cost of it, the camera it comes with, and some extra bits and pieces to have a sort of “spy-bot”. It could have many other interesting uses (RC car, camera turret, model train controller…). I fully realize that the camera resolution will be low and update rate will be probably slow, but I’d like to try. The ESP32-cam module can control an H-bridge to actuate two motors, serve the camera image over WiFi and possibly do OTA updates…
These projects have done what I aim for, but without ROS2:
I think using ROS2 framework would allow to build interesting applications by offloading the heavy computation to a computer. It would also allow to extend the hardware, such as using a game controller to control the platform.
I have researched ways to do this and I’ve found two options:
micro-ROS for Arduino would be another option, as adding the camera library should be painless, but micro-ROS for Arduino does not support ESP32. There is a branch for that, however I have not tested it.
I could test the first approach, adding my own publishers and subscribers, but I was unsuccessful in adding the camera library. I do not want to rewrite the tested and functional library as part of this project if possible. I could not find any examples that add external libraries to a micro-ROS project and the file structure is too complex for me to navigate and understand.
I have not tested the second method yet.
My question is: at this point, what is the most promising method? Are there any resources that show how to add a library to micro-ROS or is the Arduino method perhaps easier?
There’s a third and overall best option that you may have not stumbled across, which is our micro-ROS component for the ESP-IDF. This module allows you to seamlessly integrate your micro-ROS project w/ the Espressif development framework, and thus using any ESP-IDF compatible library that might come at handy, such as the GitHub - espressif/esp32-camera in your case.
Hi @FraFin, thank you very much for pointing me to the right direction.
I was really surprised not to see this kind of project done with ROS/2, in the DIY realm, considering the cheapness of the hardware and the versatility/capabilities of the board/chip.
As I was waiting for the answer I started on an easier path of using ROS Noetic with rosserial_arduino, but I realize the major issues with it (mainly connectivity) and will switch over to ROS2 in the future.
Regarding not finding any projects using ROS and ESP32-CAM module, I spoke too early. I found this project here that does exactly that. Also this one from the same author that is even closer to what I want to do, but it doesn’t have any code right now and is still ROS1.
I think the author you mentioned (Reinbert) indicated in both repo that they are both Work in Progress. I am wondering if you gave their repo a try? Does it work?
Hi, were you able to get this working in the end? I did manage to implement publishing images via microROS on an ESP32, but the framerate is terrible (about 10 FPS). Also, it’s not possible to use a best-effort publisher ( Sending ESP32CAM images on a best-effort basis · Issue #2042 · micro-ROS/micro_ros_arduino · GitHub ). I’m a bit dissappointed by these results, because with a super simple custom protocol that even works over TCP, I can transmit frames at a higher resolution at 25 FPS with ease. I originally assumed the performance would be similar with microROS.
I ended up going a different route. Instead of sending images through micro-ROS, I stream MJPEG directly from the ESP32-CAM and then bring it into ROS 2 on the robot side.
I saw similar performance issues with micro-ROS image transport, so for this project I only use the ESP32-CAM for video streaming and keep the ROS 2 communication separate.