Policy optimization in automated point merge trajectory planning: an artificial intelligence-based approach
Files
(Published version)
Date
2019
Authors
Liang, M.
LI, W.
Delahaye, D.
Notry, P.
Editors
Advisors
Journal Title
Journal ISSN
Volume Title
Type:
Conference paper
Citation
AIAA/IEEE Digital Avionics Systems Conference - Proceedings, 2019, vol.2019-September, iss.9081789, pp.1-8
Statement of Responsibility
Conference Name
IEEE/AIAA 38th Digital Avionics Systems Conference (DASC) (8 Sep 2019 - 12 Sep 2019 : San Diego)
Abstract
Air Traffic Management (ATM) is a complex decision-making process. Air traffic controllers’ decision on aircraft trajectory control actions directly leads to the efficiency of traffic flow management. In the Automated Point Merge Trajectory Planning (APMTP) problem, it aims to realize an automated routine trajectory management in Terminal Manoeuvring Area (TMA) with an intelligent decision-making agent.An Artificial Intelligence-based approach, mainly Reinforcement Learning (RL) algorithm, is applied to adaptively and smartly integrate four types of de-conflict actions for solving conflicts with fewer delays on the environment. In this paper, we will mainly discuss the policy optimization in APMTP, focus on improving the agent’s learning quality and exploration efficiency. Firstly,application of RL in adaptive trajectory planning is presented.APMTP problem is adaptively divided into several sub-problems.For each sub-problem, an online policy π is applied to guide the simulation and optimization modules to find out the conflict free and less-delay solution. The online policy π is a scale of weight distribution for choosing desirable actions. It follows the rule of Roulette-wheel selection with weighted probability. The highest desirable decision variable has the largest share of the roulette wheel, while the lowest desirable decision variable has the smallest share of the roulette wheel. The RL direct policy optimization algorithm is designed to update the online policy π.Finally, experiments are built up for validation of the proposed policy optimization algorithm for the intelligent decision-making in APMTP. The results in the test environment show that learning agent with different exploration and exploitation ability will result in different system performance in conflict resolution and delay
School/Discipline
Dissertation Note
Provenance
Description
Access Status
Rights
Copyright 2019 IEEE