Drone CV Expert

Expert in robotics, drone systems, and computer vision for autonomous aerial platforms.

Decision Tree: When to Use This Skill

User mentions drones or UAVs? ├─ YES → Is it about inspection/detection of specific things (fire, roof damage, thermal)? │ ├─ YES → Use drone-inspection-specialist │ └─ NO → Is it about flight control, navigation, or general CV? │ ├─ YES → Use THIS SKILL (drone-cv-expert) │ └─ NO → Is it about GPU rendering/shaders? │ ├─ YES → Use metal-shader-expert │ └─ NO → Use THIS SKILL as default drone skill └─ NO → Is it general object detection without drone context? ├─ YES → Use clip-aware-embeddings or other CV skill └─ NO → Probably not a drone question

Core Competencies

Flight Control & Navigation

• PID Tuning: Position, velocity, attitude control loops

• SLAM: ORB-SLAM, LSD-SLAM, visual-inertial odometry (VIO)

• Path Planning: A*, RRT, RRT*, Dijkstra, potential fields

• Sensor Fusion: EKF, UKF, complementary filters

• GPS-Denied Navigation: AprilTags, visual odometry, LiDAR SLAM

Computer Vision

• Object Detection: YOLO (v5/v8/v10), EfficientDet, SSD

• Tracking: ByteTrack, DeepSORT, SORT, optical flow

• Edge Deployment: TensorRT, ONNX, OpenVINO optimization

• 3D Vision: Stereo depth, point clouds, structure-from-motion

Hardware Integration

• Flight Controllers: Pixhawk, Ardupilot, PX4, DJI

• Protocols: MAVLink, DroneKit, MAVSDK

• Edge Compute: Jetson (Nano/Xavier/Orin), Coral TPU

• Sensors: IMU, GPS, barometer, LiDAR, depth cameras

Anti-Patterns to Avoid

1. "Simulation-Only Syndrome"

Wrong: Testing only in Gazebo/AirSim, then deploying directly to real drone. Right: Simulation → Bench test → Tethered flight → Controlled environment → Field.

2. "EKF Overkill"

Wrong: Using Extended Kalman Filter when complementary filter suffices. Right: Match filter complexity to requirements:

• Complementary filter: Basic stabilization, attitude only

• EKF: Multi-sensor fusion, GPS+IMU+baro

• UKF: Highly nonlinear systems, aggressive maneuvers

3. "Max Resolution Assumption"

Wrong: Processing 4K frames at 30fps expecting real-time performance. Right: Resolution trade-offs by altitude/speed:

Altitude Speed Resolution FPS Rationale

<30m Slow 1920x1080 30 Detail needed

30-100m Medium 1280x720 30 Balance

100m Fast 640x480 60 Speed priority

4. "Single-Thread Processing"

Wrong: Sequential detect → track → control in one loop. Right: Pipeline parallelism:

Thread 1: Camera capture (async) Thread 2: Object detection (GPU) Thread 3: Tracking + state estimation Thread 4: Control commands

5. "GPS Trust"

Wrong: Assuming GPS is always accurate and available. Right: Multi-source position estimation:

• GPS: 2-5m accuracy outdoor, unavailable indoor

• Visual odometry: 0.1-1% drift, lighting dependent

• AprilTags: cm-level accuracy where deployed

• IMU: Short-term only, drift accumulates

6. "One Model Fits All"

Wrong: Using same YOLO model for all scenarios. Right: Model selection by constraint:

Constraint Model Notes

Latency critical YOLOv8n 6ms inference

Balanced YOLOv8s 15ms, better accuracy

Accuracy first YOLOv8x 50ms, highest mAP

Edge device YOLOv8n + TensorRT 3ms on Jetson

Problem-Solving Framework

1. Constraint Analysis

• Compute: What hardware? (Jetson Nano = ~5 TOPS, Xavier = 32 TOPS)

• Power: Battery capacity? Flight time impact?

• Latency: Control loop rate? Detection response time?

• Weight: Payload capacity? Center of gravity?

• Environment: Indoor/outdoor? GPS available? Lighting conditions?

2. Algorithm Selection Matrix

Problem Classical Approach Deep Learning When to Use Each

Feature tracking KLT optical flow FlowNet Classical: Real-time, limited compute. DL: Robust, more compute

Object detection HOG+SVM YOLO/SSD Classical: Simple objects, no GPU. DL: Complex, GPU available

SLAM ORB-SLAM DROID-SLAM Classical: Mature, debuggable. DL: Better in challenging scenes

Path planning A*, RRT RL-based Classical: Known environments. DL: Complex, dynamic

3. Safety Checklist

• Kill switch tested and accessible

• Geofence configured

• Return-to-home altitude set

• Low battery action defined

• Signal loss action defined

• Propeller guards (if applicable)

• Pre-flight sensor calibration

• Weather conditions checked

Quick Reference Tables

MAVLink Message Types

Message Purpose Frequency

HEARTBEAT Connection alive 1 Hz

ATTITUDE Roll/pitch/yaw 10-100 Hz

LOCAL_POSITION_NED Position 10-50 Hz

GPS_RAW_INT Raw GPS 1-10 Hz

SET_POSITION_TARGET Commands As needed

Kalman Filter Tuning

Matrix High Values Low Values

Q (process noise) Trust measurements more Trust model more

R (measurement noise) Trust model more Trust measurements more

P (initial covariance) Uncertain initial state Confident initial state

Common Coordinate Frames

Frame Origin Axes Use

NED Takeoff point North-East-Down Navigation

ENU Takeoff point East-North-Up ROS standard

Body Drone CG Forward-Right-Down Control

Camera Lens center Right-Down-Forward Vision

Reference Files

Detailed implementations in references/ :

• navigation-algorithms.md

• SLAM, path planning, localization

• sensor-fusion-ekf.md

• Kalman filters, multi-sensor fusion

• object-detection-tracking.md

• YOLO, ByteTrack, optical flow

Simulation Tools

Tool Strengths Weaknesses Best For

Gazebo ROS integration, physics Graphics quality ROS development

AirSim Photorealistic, CV-focused Windows-centric Vision algorithms

Webots Multi-robot, accessible Less drone-specific Swarm simulations

MATLAB/Simulink Control design Not real-time Controller tuning

Emerging Technologies (2024-2025)

• Event cameras: 1μs temporal resolution, no motion blur

• Neuromorphic computing: Loihi 2 for ultra-low-power inference

• 4D Radar: Velocity + 3D position, works in all weather

• Swarm autonomy: Decentralized coordination, emergent behavior

• Foundation models: SAM, CLIP for zero-shot detection

Integration Points

• drone-inspection-specialist: Domain-specific detection (fire, damage, thermal)

• metal-shader-expert: GPU-accelerated vision processing, custom shaders

• collage-layout-expert: Report generation, visual composition

Key Principle: In drone systems, reliability trumps performance. A 95% accurate system that never crashes is better than 99% accurate that fails unpredictably. Always have fallbacks.