UAV Maneuvering Decision-Making Algorithm Based on Twin Delayed Deep Deterministic Policy Gradient Algorithm

Authors

  • Shuangxia Bai School of Electronics and Information, Northwestern Polytechnical University, China https://orcid.org/0000-0002-3710-9152
  • Shaomei Song Beijing Electro-Mechanical Engineering Insititute, China
  • Shiyang Liang Avic Luoyang Electro-optical Equipment Research Institute, China https://orcid.org/0000-0002-6941-2404
  • Jianmei Wang School of Electronics and Information, Northwestern Polytechnical University, China
  • Bo Li School of Electronics and Information, Northwestern Polytechnical University, China https://orcid.org/0000-0002-1415-4444
  • Evgeny Neretin School of Robotic and Intelligent Systems, Moscow Aviation Institute, Russian Federation https://orcid.org/0000-0003-0174-8929

DOI:

https://doi.org/10.37965/jait.2021.12003

Keywords:

air combat, DDPG, maneuvering decision-making, TD3

Abstract

Aiming at intelligent decision-making of unmanned aerial vehicle (UAV) based on situation information in air combat, a novel maneuvering decision method based on deep reinforcement learning is proposed in this paper. The autonomous maneuvering model of UAV is established by Markov Decision Process. The Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm and the Deep Deterministic Policy Gradient (DDPG) algorithm in deep reinforcement learning are used to train the model, and the experimental results of the two algorithms are analyzed and compared. The simulation experiment results show that compared with the DDPG algorithm, the TD3 algorithm has stronger decision-making performance and faster convergence speed and is more suitable for solving combat problems. The algorithm proposed in this paper enables UAVs to autonomously make maneuvering decisions based on situation information such as position, speed, and relative azimuth, adjust their actions to approach, and successfully strike the enemy, providing a new method for UAVs to make intelligent maneuvering decisions during air combat.

Downloads

Published

2021-12-07

How to Cite

Bai, S., Song, S., Liang, S., Wang, J., Li, B., & Neretin, E. (2021). UAV Maneuvering Decision-Making Algorithm Based on Twin Delayed Deep Deterministic Policy Gradient Algorithm. Journal of Artificial Intelligence and Technology, 2(1), 16–22. https://doi.org/10.37965/jait.2021.12003

Issue

Section

Research Article