Been lurking here for a while, finally posting because this has been eating my week.
We run a small fleet of AMRs (nothing fancy, mostly ROS 2 Humble on Jetson Orins with a couple of STM32s handling motor control and safety). The high level stack is fine, we push updates over the air and roll back if something misbehaves. The pain is the low level side. Every time we touch the microcontroller firmware, someone somewhere has to physically plug in a cable. On three robots that’s annoying. On thirty it’s a full day gone.
I’ve been reading around and honestly a lot of the embedded software development advice online is either written for hobby projects (one board, one desk, one USB cable) or for automotive teams with budgets we don’t have. There’s a weird missing middle for robotics.
A few things I’ve tried or looked at:
Bootloader on the STM32 that pulls signed firmware from the Jetson over UART. Works, but recovery when a bad image lands is fiddly.
Mender for the Jetson side, which is solid, but it doesn’t really touch the MCU layer.
Just building a proper A/B partition on the MCU flash. Half the chip for the running image, half for the next one, watchdog decides who wins.
Option 3 feels right but I’d love to hear from people actually shipping this in production. A few honest questions:
How are you handling the handoff between your Linux side and the MCU during an update? Do you pause the ROS nodes, or let them keep running on the old firmware until reboot?
Anyone using something open source for MCU OTA that they actually like? MCUboot keeps coming up but examples for robotics (not IoT sensors) are thin.
For teams doing embedded software development on top of ROS 2, are you keeping firmware in the same repo as your ROS packages or splitting it out completely?
Not looking for a perfect answer, just want to hear what’s working for real robots in real warehouses. Happy to share what we land on once we stop breaking things.
Hey @hadiaali! We just announced Agency Tool Co and our first product, ATC Deploy, with this in mind as a pain point in robotics that IoT tooling often doesn’t come close to solving.
I’d love to connect sometime and hear more about your challenges. Drop me a note at jack@agencytool.com and I’d love to share more about what we’re building.
We use a small bootloader that does the A/B switching. The bootloader is normally not reflashed.
The big trick is to flash and directly boot into the new firmware but keep the default pointing to the old firmware. Only after test and verification that the firmware is not broken, we send a command to the bootloader to switch to the new firmware after power on.
Note on stm hardware you got sometimes hardware bootloader’s that simplify things. E.g. you can flash using CAN etc.
We shut down the ros stack and have external tooling for firmware flashes. This is mostly due to the fact that we do not update often. If we would update often it would be a special mode in the ros stack.
You’re closer than the thread makes it sound. MCUboot’s robotics examples are thin because the robotics case barely differs from the IoT one: the properties you need (A/B slots, image signing, automatic revert on failed boot) are identical, and the robotics-specific part lives entirely in transport and sequencing. MCUmgr/SMP is the missing transport piece. It pushes images to MCUboot over serial or CAN from the Jetson, which turns the STM32 update into a step the Orin executes instead of a cable someone drives to.
Since Mender is already working for you on the Jetson: its update-module mechanism is the clean place to hang that step. The module flashes the STM32 via MCUmgr only after the rootfs update commits, which makes rootfs plus firmware one tested, atomically deployed pair. That pairing answers the repo question too: keep firmware in its own repo, but pin it into the ROS workspace as a versioned artifact, so the pipeline builds and ships the pair, never the halves. A mixed-version fleet is behind most of the 2am mysteries this thread is really about.
The other half of the problem, reliably reaching thirty robots on customer networks to deliver any of this, is its own rabbit hole. The mechanics above are solid ground to start from.