Hoverview is a series of blogposts throughout the ICAERUS project discussing everything drone technologies. This first installment presents an overview of open source alternatives in drone tech.
The UAV (Unmanned Aerial Vehicle) or ‘drone’ has only in roughly the past decade (2012 onward) seen development outside of the traditional military usage. Due to inreased battery capacity and power, brushless motors and the shrinkage of computation with improved sensing capabilities, drones are now accesible to many more people. This technology is not specifically made for drones either, which makes the UAV a melting pot of various hardware and software developments from academia and industry.
The largest drone manufacturer DJI has in the past decade seen a very typical evolution within drones, in the first years, custom do-it-yourself kits where created for hobbyists, such as the Flame Wheel F550 and F330, merely supplying assembly instructions and some hardware, which could be replaced with whatever the user would want. With increased complexity of these systems, and improved usability too (ready-to-fly systems such as the Matrice or Mavic series), the open and do-it-yourself nature of DJI systems has gone down significantly. There is even a whole community trying to reverse engineer the firmware developed by DJI and to develop more features or repair their aircraft.
There are many reasons to go open source: such as shifting a dependency from manufacturers to communities, which tend to be more supportive or repairs and sharing knowledge; simpler adoption and replicability of your developed method for different usages or increased collaboration on the project and different insights to a problem. The closed vs open approach is an ongoing tension in drone technology: users require the accessibility, where ‘it just works’ usually only available in a closed ecosystem, versus requiring access to open systems to get started without large investments and to try things out.
Nonetheless, there are some incredible open source drone technologies out there, enabling many users an affordable and accessible start in using UAVs for whichever usage they have in mind. Covered in this post is a broad overview of some of the more prominent and interesting open source projects, related to drone technologies. An important organization at the centre of many of these initiative is DroneCode, setting standards and organizing a centralized platform.
A word on drone technologies
As previously stated, drone technology is a melting pot from a large variety of different technologies. I tend to define it as: ‘any hardware or software used in a UAV project’. Hardware could consist of the flying platform and its constituent parts and payloads mounted on a drone such as grippers, camera’s, sensors and additional computers. Software inludes, but is not limited to, flight contol software, mission planners, autopilot algorithms, obstacle avoidance algorithms, but also the analysis side of the acquired data, such as photogrammetry. Drone technology therefore covers a wide range of options, and not all are covered here or even have open source options.
Open sourced drone building:
Building your own drone can be a nice learning experience, or a necessity, as the exact features are not available commercially or just for experimenting. Luckily, the RC-hobby has embraced the drone as part of their ecosystem, and have many guides on how to make one yourself. More and more high-end professional gear is available to make reliable systems. Not every single part of a drone can be made at home, or are open-source, but the central brain has a few options in terms of software to start with. Hardware-wise, there are a few projects out there, but long-term support is lacking, and the hardware is often quickly outdated which is why most people opt to go for premade hardware, instead of soldering their own micro-control board.
Professional/Commercial grade:
The PixHawk ecosystem is open source and focuses on professional and reliable drone hardware. From flight control boards to battery management systems. These are connectors, wiring diagrams and architectural layouts. This enabled manufacturers to develop hardware that can be used interchangeably.
PX4 is the software that is usually ran on Pixhawk flight hardware. This Autopilot and flight control software is very well documented and used for many applications, from hobbyist to professional. The main limitation is the hardware it can run on, usually higher end flight controllers. Due to fast F7 or H7 based microcontrollers required for afvanced autonomy. Their documentation also covers how to assemble a drone.
Hobbyist/Racing grade:
On the other side of the spectrum are the flight controllers for hobbyist, such as racing drones. Most of these projects originally started with the MultiWii hardware and software combination, growing through BaseFlight and Cleanflight. These are similar to PX4, as it manages the various hardware components and flies the drone,.
BetaFlight is more focus on speed of flight, and acrobatic movements, while PX4 is built around autopiloting. Betaflight has exceptional documentation and also a wider range of supported hardware. More of the source code can be found on Github.
INAV is focused on autonomous waypoint navigation. With a similar wide range of flight control boards being supported.
Ardupilot is a also a long-term player in drone control software, with GitHub commits from 12 years ago. This family of softwares enables wheel-based robots, water-based, underwater-based, and fixed wing platforms to be more autonomous and regulate the sensor inputs and motor outputs.
With a search on Github many more interesting projects can be found, covering Remote Controller firmware, more flight control software, elecronic speed control firmware (regulates the speed of the motor and sits inbetween the flight controller and motor).
Open sourced autopiloting and mission planning:
Flying a drone manually is a nice exercise, but for larger areas or repeatability, an autonomous flight might be more preferable. In extension of the flight controllers are the mission planners. ). This software calculates flight paths based on a few presets, such as size of the area, number of pictures, ground sampling distance, etc.
In extension of the Ardupilot flight control, is their excellent mission planning software: Ardupilot Mission Planner. However this is not your only option, QGroundControl is also a highly used piece of kit.
For more information, check out how to choose a ground-station from the Ardupilot website.
Missing from these projects still is the massively popular Robotics Operating System. This is an open source framework for developing, testing and deploying robots. Within ROS there are many packages with advanced control options, obstacle avoidance, mapping, localization, etc. ROS is not easy to get started, but once you get the hang of it, it is an incredible tool to have in your arsenal.
PX4 has extensive documentation how the flight controller could be integrated into this robotic system for advanced movement and mapping, through WiFi or MAVlink.
Open sourced sensing:
An important task where UAVs are often used are to sense the environment, taking images and creating insights from them. Sensors are incredibly complex and expensive hardware projects, which results in not many of those to choose from. Perhaps the future will bring more interesting projects in this domain!
An honourable mention in this area is the ArduCam project. These are a family of hardware boards with affordable RGB camera’s. More important however, are their code-implementations and easy to get started with their hardware. From their website, many other manufacturers, such as Luxonis with AI-enabled USB camera’s , or RaspberryPi camera’s are showcased.
However, a multispectral, thermal or hyperspectral camera would require more patience, although it would be a valuable addition to the Drone Open Source Ecosystem. A beginning is starting thourgh the OpenHSI project, which has a software codebase to analyse Hyperspectral imagery, and a hardware implementation with Ximea sensors.
Open sourced analysis:
Drones can give fantastic insights into the environment, and have seen many promising usages using their aerial perspective. There is much interesting data to be gathered using the system, but that is half of the story, analysis is not as straightforward as it might seem from the start. Presented here are three, related and overlapping methods that are used in drone image analysis. This is also a main concern to promote, develop and optimize in WP2 of the ICAERUS project.
Developing orthomosaics/3D models/georeferencing
When an object is photographed from various perspectives, a 3-dimensional model can be calculated using various optimization techniques, from image alignment, georeferencing, depth estimation, etc. Luckily, a user does not need to know which parameters of the Scale-Invariant-Feature-Transform they should use, as this whole pipeline is often integrated into a software package, in which images, GPS points, and ground-truth observations can be combined. OpenDroneMap is the mapping software on-par with many proprietary and closed alternatives. OpenDroneMap has various implementations, such as web-based GUIs WebODM, server-side NodeODM or command-line ODM. The idea of this project is that images can be uploaded, and directly merged into a single larger orthomosaic, dense mesh, point-cloud, or any other dataset-format that is required for the analysis. Specifically aimed at drone-users, as it implements global positioning and georeferencing of the resulting dataset/map. Meshroom if you are more concerned with high-quality 3D meshes or point-clouds, Meshroom might be a good alternative.
Visualization/analysis (all built upon GDAL):
Now that you have a map or a merged dataset, further analysis still might be required. As these applications are often geographically located, Geo-Information-Software (GIS) is your way to go on. QGIS is the de-facto standard in open-source GIS, built upon the Geo-Data-Abstraction-Library (GDAL), it can calculate variables from the data, perform various geographic operations, or merge additional datasets, all through a graphic user-interface. Using a GIS-based approach really enables the most to be acquired from the drone-data. Personally, I have been using Python and Jupyter notebooks as my go-to for data analysis. This requires some programming knowledge to get started, but nothing beats the clarity and speed that GDAL-based GeoPandas, and rioxarray can provide in analyzing larger datasets.
When purely working with 3D-data, Blender is an excellent 3D program, to analyze meshes, visualize environments, create animations or even feature-films. Additionally, users have created their own library for inside Blender, specifcally made for geographic operations. Another interesting projects for 3D data is CloudCompare. Which can work with various complex 3D data structure, starting as a comparison tool, now grown with more features such as resampling, statistics and segmenting.
Machine Learning (python):
What you may think of AI or Machine Learning as a buzzword, in image-analysis it luckily has a large scientific background and information. Many see a promising way forward in combining UAV imagery with Deep Learning for new insights, and automation or precision agriculture.
The most used Machine Learning packages are all developed with a Python version too. A good starting point for statistically-based Machine Learning is Scikit-learn, this library has a collection of many algorithms and elaborate examples for various datasets. Clustering, classification, dimensionality reduction or pre-processing can all be done using this library.
Deep Learning takes the ideas of ‘learning’ approaches and applies them to large networks. These networks can be written from the ground-up, or luckily, using an abstraction library, which deals with the complexity of training these models. The most used lihbraries are Pytorch, Tensorflow, and Keras. The resulting Deep Learning models from these libraries are often interoperable.
The latest developments from OpenAI, such as their GPT familty mostly focus on Large Language Models, these are gigantic networks with text-based inputs and outputs. In drone analytics, we tend to work with imagery. Which can be subdivided into image classification, object detection and segmentation. Most usable for Drone imagery are object detection (where is object X in the image: bounding box) and image segmentation (which areas in the image are X: colouring regions). Exact architectures to get started with are the YOLO family for object detection and Mask-RCNN and U-Net for image segmentation.
When really experimenting and exploring the state-of-the-art, Huggingface is an excellent resource of datasets and models, and where most of the new developments in Deep Learning are shared. In addition they have developed their own libraries built on top of PyTorch.
This Deep Learning topic gets really deep and complex quite quickly, so expect more information, tutorials and insights on Deep Learning for UAV imagery in the future, as we identify, release and optimize more in this incredibly fast-growing field within the ICAERUS project.
Let’s begin!
It is no doubt that UAVs are built upon technology, and see much development and growth through technological improvements. It is good to see Open Source alternatives to all the closed options that are out there, as reducing the barrier to entry to this incredible flying tool should be the highest priority the coming years. This post is just an overview of some select drone technologies but are promising and useful tools for those who want use them right here, right now.
Ideally, everything is open sourced, but for now we can be extremely content with the technologies that are out there. Enabling us to test with, support or even deploy to our own drone-useages. The community-based approach also enables us to participate in the development and growth of various drone-technologies going forward. Most are based in software for now, perhaps we will see open source hardware drone initiatives grow and expand in the coming years.
Contribution by:
Jurrian Doornbos
Wageningen University and Research
