Been building a UAV navigation stack around a Jetson Orin NX + Livox Mid-360 + Pixhawk, and wanted real numbers on the accuracy trade-offs between LiDAR-only, LiDAR-inertial odometry, and full SLAM before picking one for the platform. Ended up running two separate evaluations:
1. Public benchmark (TIERS dataset, indoor, vs OptiTrack ground truth)
| Method | APE RMSE (m) | Mean | Median | Max |
|---|---|---|---|---|
| KISS-ICP | 0.124 | 0.102 | 0.099 | 0.266 |
| FAST-LIO2 | 0.060 | 0.053 | 0.053 | 0.140 |
| GLIM (CPU) | 0.025 | 0.023 | 0.021 | 0.101 |
2. Our own UAV flight data
Livox Mid-360 + Pixhawk on the Jetson Orin NX (the actual target hardware), flown in a VICON-tracked indoor arena, ground truth from /mavros/local_position/pose (Pixhawk EKF fused with VICON). 7 flight sequences, GLIM running in GPU mode this time:
| Method | Average APE RMSE (m) |
|---|---|
| KISS-ICP | 0.123 (volatile — spikes to 0.426 on one sequence) |
| DLIO | 0.096 |
| FAST-LIO2 | 0.112 |
| GLIM (GPU) | 0.040 |
GLIM wins decisively on every single sequence in the live data too — averaging 0.040m vs 0.096–0.123m for the others, mainly from loop closure against the small, revisitable arena. KISS-ICP is the most volatile of the four; being LiDAR-only with no inertial backstop, it falls apart under more aggressive drone motion even though it’s competitive on gentler segments.
Repo (configs, scripts, both datasets’ results, trajectory/map reconstructions): GitHub - harshareddy1611/livox-mid360-slam-evaluation: Benchmarking KISS-ICP, DLIO, FAST-LIO2 and GLIM on Livox Mid360 datasets using ROS2 Humble and EVO trajectory evaluation against ground-truth data. · GitHub
A couple of things that ate real debugging time, in case they save someone else the trouble:
- ROS1→ROS2 bag conversion silently drops per-point
timestamp/linefields — breaks motion compensation until you reconvert withrosbags-convert - Mocap/LiDAR clock offsets need different handling per method
- The Ericsii FAST-LIO2 ROS2 port has its PCD-accumulation code commented out upstream, so native map export doesn’t work — had to reconstruct maps by reprojecting raw scans through each method’s estimated trajectory instead
Curious if anyone’s run a similar comparison on more dynamic/outdoor UAV flights, wondering how much of GLIM’s edge here is specific to small, loop-closure-friendly indoor spaces versus something that holds up in open environments too.