Simulating a ROS2 Biomimetic AUV: Gazebo, Dave, or Stonefish?

[_bernardo]

Hello there,

Im working on a biomimetic auv, and im looking into simulators. Is there a go to ros2 simulator at the moment? I see Dave referenced, but it seems is not fully moved to ros2? Im looking into just fake interactions in gazebo, or go full simulation with Stonefish.

My goal is to build on the higher level stacks, like path following, computer vision related tasks, etc.

Simulating sensors like depth, imu, camera, dvl would be great. But the biggest constraint for my use case is that my auv produces thrust by flapping its wings and can produce asymetrical thrust. It also generates a significant amount of lift that i could turn into forward movement with bouyancy differentials. It turns by “banked turns” like a plane does and controls roll with the tip of the wings and pitch with the tail, which requires some level of hydrodynamics to calibrate correctly as it looses lift the more it rolls, etc

I guess i could fake the thrust generated by the wings via invisible thrusters on each side, then map certain wing movements to different thrust forces? But what about roll and pitch via control surfaces? is that something gazebo would be able to simulate?

I was thinking of going with Stonefish. But that might be overkill, and im not sure even stonefish could simulate the deforming meshes of the wing’s skin as the wings move. For the URDF i had to go with solid links and cut them along the axis of the TFs, but the real wing is a mix between solid and compliant materials. And if a sim was to realy try to simulate hydrodynamics from surfaces then it would go crazy with these solid chunks and their gaps, which are not how the actual robot behaves. So I figured as long as i can fake the hydrodynamics, bouyancy, thrust, etc that should allow me to work on path following etc.

The URDF is done, with real inertia tags, collision, etc, now i was moving into implementing into gazebo, but if i have to learn things from zero, should i try to fake it in gazebo first, or would my time be better spent starting from the get go in something like stonefish?

Thanks!!

[OlivierKermorgant]

Hi,

I am not sure Stonefish handles lift/drag surfaces, which is the core of your biomimetic system. The hydrodynamics in Stonefish use the actual collision shape to compute drag and friction, but no lift is considered.
Gazebo has a plugin for that, you have to attach it to each of the considered surface. The lift and drag coefficients have to be given explicitly, as they are not computed from the actual shape (as far as I know).

[ivandor]

@_bernardo Largely agree with the other answer.

I recommend starting in Gazebo. Get your URDF spawning with the sensors and hydrodynamic fidelity you need so you can focus on the path following and computer vision related tasks you mentioned.

If you are really interested in the simulation space, Simulation - Maritime Robotics Landscape has more information on the current state of simulation and links to a lot of the simulators available. More work is being done on the DAVE ROS2 port, which you can read more on or reach out to them directly here: https://dave-ros2.notion.site/.

[arjo129]

Call me biased but:

I think you could do a very rough model of lift generation using the available Hydrodynamic and LiftDrag plugins. Gazebo can absolutely simulate pitched aerelions. The real challenge would be calibrating your sim with real world observations. Worst case scenario, you may have to write your own custom plugin, but a flapping robot sounds super super cool and fully doable using the lift-drag plugin. In terms of sensors gazebo has the necessary sensors for everything except for good cameras.

One of the hacks Ive used in the past is to get all the data from gazebo, but leave high quality rendering as a separate task for a better external renderer (for my demo it was a gaussian splat but, same techniques can be applied). There is a proposal for such an external renderer currently being crafted by the core team.

[_bernardo]

Thank you for replying @OlivierKermorgant . I Emailed the guys at Stonefish asking about this, they said it might be doable with some tweaks on his side, which apparently are already in the works. But of course that might take time. Given the replies on this thread I’ll be putting the time on Gazebo. Thanks for the input!

[_bernardo]

Thank you @ivandor . Seems like the consensus is pretty strong towards getting to work on Gazebo. Thanks for the links, i’ll be checking them out. I’m very interested in the space. Thanks!

[_bernardo]

Thanks for your reply @arjo129 , I’m pretty convinced now to focus on Gazebo.
When you say that a flapping robot should be doable, do you mean you think simulating the thrust generated by the wings motions should be feasible? or just the other lift/Hydrodynamics like ailerons ?
Thanks for the ideas on the camera pipeline!

[arjo129]

If you use the lift drag plugin together with the joint controller it’ll automatically generate the lift forces for your robot once you start flapping its wings. This would be a poor man’s approximation but I suspect it could be a starter, as things evolve you’ll probably have to add more stuff.

[_bernardo]

Thanks, i’ll keep that in mind as I move forward. As long as it gives a rough estimation I think it will allow me to work on stuff. I will do periodic real water testings for calibration anyways. thanks for the info!

[AlePuglisi]

HI @_bernardo ,

from ocean lover to ocean lover, your project is incredible !

My answer is a bit out of topic for your current objective of a simulation for perception task, but maybe is interesting for any future development.

A friend of mine is currently working for his PhD on a manta ray inspired robot, similar to yours, but smaller.

During his master thesis he worked on the electronics, first control strategy and Jazzy-ROSify the system (manuscript here and GitHub code).

Right now he is interested in obtaining a natural bioinspired control via reinforcement learning.

He struggles a lot to find a simulator able to properly capture the behaviour of a deformable link as actuation system.

He told me about FishGym (paper here).

The first version is archived, but he told me that the guys of FishGym are working on a new version.

This simulator is mainly for the control aspect of biomimetic underwater robot, not for perception (and definitely not a ROS related project). It can be interesting for future improvements of the control stack of your awesome Manta.

Maybe their approach to simulate hydrodynamics can inspire you in the development of a soft-actuation underwater plug-in for Gazebo or extending the Stonefish library.

(I’ll try to bring him to the chat, maybe he can provide better comments)

From my side, I only worked with thrusters-based vehicles in simulation.

Currently I’m exploring Stonefish for (Opti-Acoustic) SLAM, so I’m also at the perception level ;).

Between Gazebo and Stonefish, I prefer the second for the better rendering (awesome for optical cameras, a bit less for acoustic ones) and its underwater-tailored sensors, physics and rendering.

You probably already read the documentation of Stonefish, I love it because you have the possibility of simulating different level of turbidity, the color attenuation/absorption effect, straight forward currents and waves integration in the simulation environment, and the “marine snow”.

Also, given the model description it takes care of the related underwater physics (center of buoyancy, hydrodynamic coefficients, etc).

But since you have discussed with the maintainer, you already know what is feasible and not given your specific problem.

From my experience, I tried to set up an underwater ROV in Gazebo one year ago.

It was my first attempt on underwater robot simulation, and I struggled a lot to fine tune the hydrodynamics and the inertia of the system (the one computed from CAD software make the system behave very badly and I had to simplify a lot the physical mesh).

I know that the Gazebo community is constantly improving it, since last year things can be changed.

If the others (with much more experience than me) suggest you to go for Gazebo, give it a try !

For perception related tasks also HoloOcean and OceanSim are very good, but much more restrictive for the GPU requirements (I never use it because of that..)

I can’t wait to see a first demo video of your robot in simulation !

[_bernardo]

Hey @AlePuglisi!,

Thanks a lot for the encouragement! I truly appreciate the support.

I took a look at your friend’s work, interesting stuff! Is he still working on his project? If he wanted to share ideas I’d be interested.

I had no idea about FishGym, will have to look further into it, specially if they are still working on improvements. Thanks for the tip.

I’m very new to the underwater simulation, or simulation world at all, but I was also attracted to Stonefish because it seemed like it would be the most complete tool. So for a given amount of manhours given to the task of learning, i figured I’d get more out off of getting things running on Stonefish. But from what I see, there would be value on having some sort of simulation running on Gazebo first, even if with fake hydrodynamics, thrust, or other properties to work on higher level things, so that i can start sharing the whole simulation / ros workspace so others could play with it. My idea has always been to do the software open source.

So I’m leaning towards getting the basics running on gazebo, learning and understanding how simulations work, getting the basic control loop working properly and progressively, even in parallel, work on Stonefish.

I’m honestly just winging it with ros. I do not know a lot of best practices, or how to properly structure, or divide functions, proper use of services, lifecycles, etc. So I’m a bit reticent to share the whole thing too early so others could actually get some value out of the software and not just having to struggle to make sense of it. But I’m getting there…

Thanks for taking the time to reply! My main motivation for all this is to actually build something that gets in the hands of those that care for the oceans, so if that’s something you would want to talk about I’d be happy to connect.

[AlePuglisi]

Yoo, I’m happy that my suggestions could be helpful for you

I took a look at your friend’s work, interesting stuff! Is he still working on his project? If he wanted to share ideas I’d be interested.

Yes ! He is working on that for his PhD, both on bio-inspired fins actuation and probably on sensor improvement for robust SLAM and navigation.

He would probably love to exchange ideas with you and help each other !

I invited him to join the chat, he will probably reach out and add his thought soon.

So I’m leaning towards getting the basics running on gazebo, learning and understanding how simulations work, getting the basic control loop working properly and progressively, even in parallel, work on Stonefish.

I can perfectly understand your plan, if you want to reach more people with your simulator, Gazebo is probably the best choice !

I think that Stonefish is better for marine robotics simulations, and its community is increasing, but currently it is mainly used by researchers rather than hobbyist.

If you like I would love to exchange ideas with you and even help with your project if I can do something ! This month I’m a bit busy, but from march I could try to help you to set up the basics in Stonefish.

I’m not an expert, so we will learn toghether!

I’m very curious to see how to exploit the available Stonefish actuators to simulate your bionispired manta robot (something like rudders for the pelvic fins and mapped servo-virtual thrusters for the pectoral ones).

I’m trying to set up a Stonefish+ROS2 based simulator of some MBARI’s Vehicles, if you want to take a look ! Its on my GitHub: MBARI-vehicles-sim-ros2

In that repo I have also an old attempt of a gazebo based simulation, but maybe now it is deprecated (it is from Jan. 2025)…

I’m planning to add some brief explanations of the examples and the code on the repo README! For now I’m mainly “playing” with it, and the code is not well commented and structured.

[_bernardo]

Stonefish does look very good…!! Makes me wonder if it would lend itself to apply approaches like SeaSplat.

I’ve been (mentally) exploring alternative localization / velocity approaches that wouldn’t require expensive sensors like DVLs, so optical flow came to mind, but underwater it can be a bit finicky and conditions dependent. Is that something that could be tested in Stonefish? Odometry is something I’m still trying to figure out for Gazebo (or any sim) without a DVL ( which I also dont have)

I’m still wrapping my head around what exactly I will need in a Gazebo setup and how to implement it and best practices to share it. My near term goal is to at least have a functional teleop capable sim so that people could play with other stacks on top of the “low level” basic control and movements stack. The low level stack is mostly done, but I have to work on a “goal manager” architecture that could take standard goal structures and the low level stack would take care of the orchestration of control loops and movements to achieve the goals. But from what i see right now I will need to write my own plugins/systems etc.

I will DM you so that the thread doesn’t veer off too much from the subject and we can discuss other stuff.

And pretty cool work on that MBARI simulation!

[ScudeT]

Hi @_bernardo, hi @AlePuglisi

I want to first thank @AlePuglisi for the heads up on the project, already a wile ago, but I’m sorry @_bernardo if I never reached out as I needed some time to wrap my head on the same problems you are facing.

The goal of my thesis was to develop a manta-inspired system that was completely autonomous, using only “off-the-shelf” components and open-source software so that everyone could build upon it. The control strategy was kept simple and model-free to have a starting ground for all future developments. (I’m sorry if the readme is so bad, but I lacked the time to improve it so it’s stuck to the first version I made, even before I finished the project ).

The goal of my PhD instead is to teach it to swim, and hence my current focus on Simulation environments for bio-inspired propulsion strategies. We have different prototypes in the lab so the goal would be to create a standardised approach you can apply to different bio-inspired robots.

Following recent trends and the focus on DRL (Deep Reinforcement Learning) control strategies AND espetially for future development on Vision based localization/control, I am currently focusing on a possible plugin/setup of the IsaacSim-Lab development stack. I am still very, very early in the project, and I can’t say for sure I will remain on this track.

You have to understand that, unfortunately for us, the dynamics of swimming in general are very much dependent on Turbulent Flow, which throws most lift-drag based hydrodynamic models out the window and with them all simulators that only apply those hydrodynamics.

The solutions to this problem seem to be two:

• Simulate the full Hydrodynamics through CFD: which is the FishGym track as they use a form of IB LBM (Imersed Boundary - Lattice Boltzman Method) to simulate the Fluid-Structure interaction between the mesh of the fish swimming and the surrounding fluid. This achieves very good results in simulation and i saw they applied it to one of their papers, but I still need to check it myself. They will make the latest version of the simulator available to the public after the Chinise New year and I am currently in contact with them to see how it works. (I still haven’t seen it but I suggest to see the results they obtained in tehir papers in their youtube channel Xiaopei Liu - YouTube)

• Find the correlation between actuation and movement experimentally: This is very similar to how current simulators work (StoneFish, Oceansim …) as they only simulate the hydrodynamics through lift-drag relationships from the mesh and have a separate model for thrust (a “simple” non linear relationship between motor velocity and force in the axis’s direction). Something like this could be achieved experimentally through tests on a single fin or the full robot with a 6-axis loadcell through static tests (even though I have a modelling error as the robot is not moving through water, so the forces I read are different but similar enough maybe) OR Have the robot freely move underwater and track its 3D path as a response to “random” actuation (needs a BIG swimming pool and very good localisation) [i thought of doing it at the lake but the disturbances from water currents are way too high]

Which one is best ? I am figuring it out

I would love to chat more on the project you are developing and the current state of your work !

[_bernardo]

Hey there @ScudeT,

Thanks for taking the time to write such a detailed and informative response. There’s lots of information there.

Seems like we are tackling very similar problems. My framing of the problem was mostly about building a robot that is useful and not cost a year’s worth of a team’s budget. So my goal was to aim for longer autonomy (thats how i ended up with biomimetic propulsion), depth rating, and “hackable”: Stay longer underwater, go deeper, customize. All while remaining significantly more cost effective. I saw ros2 as the standardizing language that would help speed up development and collaboration.

The robot works fairly well with an autonomy of about 6hrs on a single 5300mah lipo battery at 8.4v (for actuation, compute battery is independent). I’m planning to actually run the robot on 2 to 3 x the actuation battery capacity. Underwater speed is a bit under 1m/s today, but I think I will be able to reach >1m/s cruise with bursts above that. It seems to be quiet stall resistant, gliding very nicely at even ~0.2m/s (It is still negatively bouyant, i have not tackled that issue yet). But all this with my propulsion model so far being “ROUGH” I have not yet exhausted and tested all “kinematic knobs” the biomimetic propulsion offers, so all these observations remain just observations.

I created a map of 6 motion primitives, with 3 “thrust” profiles each. By orchestrating those movements and mixing them together I can basically make the robot do anything i need it to do. But there’s no higher level architecture or nodes that plan the motions or decide what to do yet, I do all of that myself in teleop mode. (there is a node that actually takes motion goals and mixes them together, applies servo easing, adds motion interrupts, etc). There’s plenty of room for improvement just on the software side.

This is one of the reasons I was looking into simulations, I was also exploring the feasibility of playing with DRL as you mentioned.

As I mentioned before, I emailed the guys at Stonefish and they said it should be possible to simulate the robot with a few tweaks they are planning, so I’m looking forward to see the direction that takes.

I’m re building the ROS2 workspace, to clean it up, make it more readable, add best practices etc (I’m fairly new to all this) so I can start sharing it with people that want to take a look. Would definitely appreciate to get in touch to keep the conversations going.

[woensug-choi]

@_bernardo Late joining to the thread!
The ROS2 port of DAVE(https://dave-ros2.notion.site/) does include all sensors like depth, imu, camera, dvl ready to use. We have not been able to wrap up for the release but the functionalities you find is already there. If more questions, contact me wschoi@kmou.ac.kr

[Airporal]

@AlePuglisi Hi,Your simulation project is excellent。I recently tried using Stonefish for sim2sim migration of an RL controller. To facilitate deployment on a real robot, I want to run ROS2humble on a host running Ubuntu 22.04 for verification (using the same system as the actual robot). However, I haven’t found any support for humble in the stonefish_ros2 project. I tried running the stonefish_ros2 project on Foxy but failed. Could you give me some suggestions?

[_bernardo]

Thank you for that update woensug-choi, I’m trying to keep track of the developments on this matter. Seems to be a matter of choice of fidelity levels. The tricky part i see is as @ScudeT mentioned, there’s a limit today on what the simulations can and cant simulate, and we are forced to pick and chose what matters for each task. And that is important for tasks like DRL and biomimetic propulsion models as physical properties (like coef drag, lift, or even bouyancy) can change a lot depending on what the robot is doing. I will keep an eye on the development of DAVE for sure!