Howdy everyone, Your Friendly Neighborhood Navigator here!
We’ve been working with AMD on something we’re really excited about: generalized semantic navigation running on the edge using Meta’s SAM3 foundation model on an AMD Strix Halo!
SAM3 is text-promptable. You tell it what to segment (“floor”, “sidewalk”, “cables”, “person”, “pallet”) and it returns per-pixel masks for each class. No retraining, no fixed class lists. The same model works across warehouses, sidewalks, parks, and construction sites on Day 1. You can even change the prompts at runtime. This is made uniquely possible by the newest generation of AI-enabled computers which are insanely cool (Thor included!).
We integrated this with Nav2’s costmap and planning stack so the robot can prefer certain surfaces, avoid others, detect small obstacles that depth cameras miss, and make behavior decisions based on what it actually sees in the scene. All running locally at 5-10 Hz on the Strix Halo’s GPU.
The AMD Strix Halo (Ryzen AI Max+ 395, X199) pairs a Radeon GPU, NPU, and 32 x86 cores with unified memory in a single package. It runs SAM3 and still has compute left over for everything else a robot needs.
Full blog post, tutorial, and open source code linked below.
Happy segmenting!
Steve Macenski
P.S. Interested in NVIDIA Jetson based workflows with a pretrained segmentation algorithm on your classes of interest? Check out this tutorial put together by Kiwibot / Robot.com!