I’ve been building ros2_fault_injection, a C++ ROS 2 framework for injecting controlled faults into ROS 2 systems.
The idea is to make it easier to test autonomy stacks before running on real hardware by proxying topics and services, applying configured faults, and checking expected outcomes with scenario assertions.
Current features include:
YAML-driven scenarios
topic injectors for odom, scan, IMU, TF, twist, joint states, and point clouds
service fault injection for delay/failure behavior
pluginlib-based injector discovery
scheduled and manually controlled faults
assertion events and markdown reports
an RViz panel for inspecting and controlling faults
Example use cases:
odometry bias or covariance inflation
scan dropout/noise
command delay/dropout/stale replay
PointCloud2 dust-style returns
checking topic rates and expected fault events
Repository:
Documentation:
I’d be interested in feedback on:
whether the scenario format feels natural for ROS 2 users
useful fault types I may be missing
how people would expect this to integrate with simulation, CI, or hardware-in-the-loop testing
whether the plugin interface is shaped in a sensible way
This is still evolving, but it has already been tested with a rover stack, Unity simulation, OmniSim integration experiments, and an RViz control panel.
You had me nervous for a second that this was going to be another AI pitch (“dm me if you think this is useful”), but I am pleasantly surprised to see that this looks very useful!
Do you have any cases where this type of fault injection has helped you catch something that other unit or integration tests would have missed?
Also, I think just from a quick look that this is very “ROS shaped”. Topic remapping and pluginlib is something that many users would already be familiar with, making it a comfortable pattern.
Thanks, I had the same worry about the post coming across that way, so I’m glad the repo made the intent clearer.
The most useful cases so far have been system-level behaviours rather than pure code bugs. For example, injecting delay/dropout into cmd_vel showed how important the command watchdog behaviour is: the robot should stop safely when commands stop arriving, not continue on stale velocity. That is something a unit test of the command node would not really catch in context.
The other useful area has been sensor degradation. Injecting odom bias/covariance inflation and scan/point cloud degradation makes it easier to see whether the rest of the stack notices degraded input, keeps operating, or fails silently. I’m using assertions and reports now so those behaviours can be recorded rather than just observed in RViz.
So the honest answer is: it has mostly helped catch integration assumptions and safety behaviour so far, not low-level implementation bugs. That’s the gap I’m trying to target with the framework.
yeah, I was going to say something similar as well
one of the few genuinely useful things I’ve seen announced here in recent times, even if I don’t necessarily see myself using it in the near future
Thanks for the introduction! @Shane_Hill, this looks like a useful connection. Your detector learns normal behaviour, while my framework lets you introduce known faults into ROS 2 topics and record when they’re applied.
One possible experiment would be to establish a healthy baseline, then introduce delays, dropouts, biases or noise at increasing severity. Comparing the injection events with your alerts could help measure detection latency and which faults go undetected.
I’m developing it with a rover and Unity simulation. It could provide repeatable controlled tests alongside the real-world operating hours you’re looking for. Would that be useful, and are there particular faults you’d like to test?
@reece-the-robot I’m building something similar for my application, but your repo is really extensive and useful, especially helpful for testing the whole pipeline after every merge. A very good tool for CI/CD
Thanks, really appreciate it! That’s exactly what I was aiming for — making it easier to catch issues across the whole ROS 2 pipeline in CI. Glad you’re finding it useful.