Nectar SDK: ROS 2 companion stack for autonomous UAVs

Projects
, , ,
1

Hi all,

I am part of Black Bee Drones, an autonomous drone team at the Federal University of Itajubá (UNIFEI) in Brazil. We developed Nectar SDK, a ROS 2 toolkit for the companion computer, so control, cameras, and perception sit behind one set of interfaces instead of being rebuilt for every mission.

It started around 2023 as shared mission code. We use it for indoor and outdoor competitions (IMAV, CBR, SAE), lab tests, and research, including IMAV indoor (3rd in 2023 and 2025) and SAE Eletroquad 2026 (2nd). We published it as open source under Apache 2.0.

competitions
competitions1149×428 135 KB

Archtecture

architecture_overview
architecture_overview1920×1241 409 KB

CI and Docker cover Humble, Jazzy, and Kilted (amd64 and arm64). We also run on Jetson Orin Nano (JetPack 6, Humble) and Raspberry Pi 4 and 5.

Control

Firmware and transport agnostic vehicle core behind one Drone protocol. Factory key and config select the backend; the flight calls stay the same.

  • ArduPilot: MAVROS or direct MAVLink
  • PX4: MAVROS, MAVLink, or uXRCE-DDS
  • Also Bebop and Crazyflie
  • Navigation, reference frames, companion PID, obstacles, indoor vision pose to the FCU, companion rangefinders
import nectar
from nectar.control import (
    CrazyflieConfig,
    DroneFactory,
    MavlinkConfig,
    MavrosConfig,
    MoveReference,
    PoseSource,
    Px4DdsConfig,
)

nectar.init()
drone = DroneFactory.create(
    "mavlink",
    MavlinkConfig(
        pose_source=PoseSource.VISION,
        connection_string="/dev/ttyTHS1",
        vision_pose_topic="/visual_slam/tracking/vo_pose_covariance",
    ),
)

# drone = DroneFactory.create(
#     "mavros",
#     MavrosConfig(
#         pose_source=PoseSource.GPS,
#         connection_string="serial:///dev/ttyUSB0:921600",
#         expect_lidar=True,
#     ),
# )
# drone = DroneFactory.create(
#     "px4_dds",
#     Px4DdsConfig(pose_source=PoseSource.GPS, agent_port=8888, offboard_rate_hz=20.0),
# )
# drone = DroneFactory.create(
#     "crazyflie",
#     CrazyflieConfig(uri="radio://0/80/2M/E7E7E7E7E7", cf_name="cf231"),
# )

drone.takeoff(altitude=2.0)
drone.move_to(
    x=5.0,
    y=0.0,
    z=1.5,
    yaw=90.0,
    reference=MoveReference.BODY,
    precision=0.12,
)
drone.land()
nectar.shutdown()

Vision

  • Cameras: USB, RealSense, OAK-D, Pi Camera, ROS topics (CameraFactory / ImageHandler)
  • Algorithms: ArUco, color, line, distance, optical flow, MediaPipe

AI

  • Tasks: detection, segmentation, classification (Detector, Segmentor, Classifier)
  • Backends: Ultralytics, Transformers, RF-DETR (task support varies)
  • Also a nectar-ai CLI for train, eval, and predict
Vision and learning 
from nectar.vision.camera import ImageHandler
from nectar.ai.detection import Detector
from nectar.ai.segmentation import Segmentor
from nectar.ai.classification import Classifier

ImageHandler("webcam", image_processing_callback=on_frame).run()
# "realsense", "oakd", or a ROS image topic

detector = Detector("yolov8n.pt")
# Detector("rfdetr-medium")
# Detector("facebook/detr-resnet-50")
detector.load()
result = detector.detect(frame)

segmentor = Segmentor("yolov8n-seg.pt")
# Segmentor("rfdetr-seg-nano")
# Segmentor("facebook/maskformer-swin-tiny-coco")
classifier = Classifier("yolo26n-cls.pt")
# Classifier("google/vit-base-patch16-224-in21k")

Interface

Optional Qt6 app for control, camera filters, and ROS 2 topics, services, parameters, and plots.

Simulation

Gazebo SITL for the same ArduPilot and PX4 backends.

GUI anf Gazebo SITL

ui_control
ui_control1280×720 79.6 KB

cbr-sim
cbr-sim1920×1080 199 KB

Documentation, examples: Nectar SDK - Nectar SDK

Repository: GitHub - Black-Bee-Drones/nectar-sdk: ROS2 SDK designed to simplify drone control and computer vision tasks · GitHub

If this is useful to other teams or labs, issues, feedback, and contributions are welcome