Autonomous UAV Deconfliction through Reinforcement Learning

Functie

Tasks

The assignment will include the following tasks:

• Investigation of the transformer method for (RL-based) decentralized DAA and associated challenges.
• Implementation of an RL-based DAA training setup, based on a simplified multi-vehicle simulation in Python/Pytorch. Choices with respect to features, transformer architecture, reward function and robust training setup.
• Performance and robustness analysis of the learned DAA strategy. Develop training scenarios focussed on DAA robustness in mixed traffic environments.
• Optional: Implementation of the DAA strategy on a number of real UAVs followed by flight-test validation at the NLR Drone Centre.

 

Results

The final outcome of this assignment will be:

• An RL-based UAV detect-and-avoid strategy and conclusions with respect to its robustness in complex air traffic environments.
• A technical thesis report describing the approach, results and conclusions of the work.
• Optional: a conference paper.

Duration

From 6 to 9 months.

Profiel

What do we expect from you?

• Master student aerospace engineering, mechanical engineering, control engineering or computer science
• Experience with programming (Python, Matlab)
• A good understanding of (aircraft) dynamics, simulation & control
• Preferably experience with ML/RL methods (PyTorch, Keras, Tensorflow or other)
• Any experience with Pixhawk, PX4 stack and UAV flight testing is considered a plus

Arbeidsvoorwaarden

What we offer

• Enthusiastic colleagues who are experts in their field
• A flexible working space
• An environment where you have the opportunity to develop your skills and learn new ones
• A challenging assignment in a high-tech, result orientated work environment
• A thesis assignment allowance
• An informal corporate culture where your opinion counts!

Informatie

About NLR

For more than 100 years, Royal NLR has been the ambitious knowledge organization with the will to keep innovating. From that motivation, we make the world of transportation safer, more sustainable, more efficient and more effective. We are on the threshold of ground-breaking innovations. Plans and ideas get moving when they are well fed with the right energy. Over 700 passionate professionals work on research and innovation. From aircraft engineers to psychologists and from mathematicians to application experts.

Our colleagues would love to tell you what it’s like to work at NLR.

You will be working within the AOAP (Aerospace Operations: Air Traffic Management & Airports) department. Your colleagues are focused on solving real-world problems within air traffic management, airspace design, U-Space and other exciting domains.

Background

The autonomous operation of unmanned aerial vehicles (UAVs) plays an increasingly important role in research and commercial applications with mission scenarios ranging from delivery to surveillance or search and rescue tasks. These UAV flights will likely concentrate in urban, built-up areas, leading to an increasing risk of mid-air collisions. At the same time, advances in battery technology and control systems promise a new, more affordable form of urban air mobility, with currently over 200 companies developing electrically powered ‘air-taxis’. Increasing numbers of both unmanned and manned vehicles in urban airspace, along with conventional traffic (e.g. emergency services), will lead to a highly congested airspace and require robust conflict detection and resolution.

The development of robust and autonomous detect-and-avoid (DAA) technologies is therefore seen as a critical enabler for the integration of UAVs and UAM into urban airspace.

Recent developments in machine learning, especially the development of new transformer architectures for natural language processing, have shown promise for use in conflict detection and resolution applications. Novel, attention-based transformer structures, originally developed for natural language processing, provide an efficient means for spatial encoding with variable input vector length. This allows to efficiently represent relative motion between different vehicles, learn to predict associated collision risks and train efficient and scalable policies for autonomous deconfliction.

However, this approach is yet to be tested in a realistic traffic environment. For this, it should account for mixed traffic, such as different aircraft types or traffic with and without an active DAA system.

Want to know more about this thesis assignment?

For more information about the assignment contact:

Sasha Vlaskin MSc. (NLR/TU Delft) sasha.vlaskin@nlr.nl

Solliciteren

Is this something for you?

Great! We are looking forward to hearing from you! Send your motivation and CV to Sasha Vlaskin (sasha.vlaskin@nlr.nl) and we will contact you as soon as possible!