Hi everyone,
We have opened the UAV Perception Testing Track, a tool competition in the Challenge Competition Track of ICST 2027, the IEEE International Conference on Software Testing, Verification and Validation. The question it asks is a practical one for anyone flying ROS 2 drones: which scenarios make a perception-driven autonomy stack fail, in how many different ways?
Flight c0017 from the released training data: after waypoint W4 a rotor strikes a placed pole, recorded as the mechanism (pole, rotor, turn).
What you build
A scenario generator. It writes forest scenarios for a quadrotor: a patrol route, the width of the lane cleared along it, poles placed near the route, tree density, wind, cruise speed and altitude. Generators can be written in Python with the provided starter kit and are submitted as Docker images.
The stack under test (fixed for every team)
- ROS 2 Jazzy
- PX4 in software-in-the-loop, Holybro X500 airframe
- Simulated RGB-D camera with RealSense D455 intrinsics
- nvblox volumetric mapping
- EGO-Planner
- NVIDIA Isaac Sim 6.0.1 with the Pegasus Simulator, procedural forest scenes
How it is scored
Each submission writes 20 scenarios on an undisclosed seed; every scenario is flown three times. A collision is recorded as a failure mechanism: what was struck (pole, tree or terrain), with which part of the vehicle and in which route phase. Generators are ranked by how many distinct mechanisms their scenarios reach, so a generator that finds new ways to fail beats one that repeats the same crash. Scores appear on a live scoreboard and each team can submit up to five times; each team’s best submission is flown again on a new seed for the final ranking.
You do not need a GPU or a local Isaac Sim install to take part: all flights run on our side, while released flight data is available for training on request.
Dates and recognition
- Submission deadline: 15 January 2027 (AoE)
- Prizes for the three best teams in the final ranking
- Participating teams may be invited to submit a 2 to 4 page tool paper in the ICST format
- Results are presented at ICST 2027 in San Sebastián, Spain, 17 to 21 May 2027
Links
- Track site: UAV Perception Testing Track
- Repository and starter kit: GitHub - IDeaLS-Lab/uav-testing-track-2027: UAV Perception Testing Track: a tool competition on test scenario generation for autonomous UAVs, proposed for the ICST 2027 Challenge Competition Track · GitHub
- Official ICST call: ICST 2027 - Challenge Competition Track - ICST 2027
Questions are welcome here or on the repository’s issue tracker. If you have worked on drone testing, nvblox or EGO-Planner, we would especially like to see what scenarios you come up with.
Prakash Aryan, on behalf of the organizers (The Italian Institute of Artificial Intelligence (AI4I) and the University of Bern)