Autonomous Maneuver Strategy of UAV Wingman Air Combat Based on Hierarchical Reinforcement Learning

Jinwen Hu; Kun Guo; Zhao Xu; Gang Xu

doi:10.1007/978-981-97-1083-6_24

Autonomous Maneuver Strategy of UAV Wingman Air Combat Based on Hierarchical Reinforcement Learning

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

Unmanned aerial vehicles (UAVs) play an vital role in modern air combats, where manned fighter (leader) can better perform air combat with the assisting of UAVs. The key technology for cooperative air warfare between UAVs and manned aircraft is UAV autonomous maneuver strategy. In this paper, an autonomous maneuver strategy of UAV wingman to assist leader in air combat is proposed. Firstly, based on the mission scenario and process of cooperative air combat, the UAV-assisted leader air combat model is established. Second, a two-level maneuver strategy method of UAV based on hierarchical reinforcement learning is designed, where the low-level agents control the game with enemy and manned fighter collaboration, and they are selected by a high-level agent. The reward functions of each agent are designed according to the relative situation to leader and enemy. Finally, the simulation experiment shows that the UAV can assist leader to strike enemy, proving the effectiveness of the proposed maneuver strategy.

Original language	English
Title of host publication	Proceedings of 3rd 2023 International Conference on Autonomous Unmanned Systems (3rd ICAUS 2023) - Volume II
Editors	Yi Qu, Mancang Gu, Yifeng Niu, Wenxing Fu
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	258-267
Number of pages	10
ISBN (Print)	9789819710829
DOIs	https://doi.org/10.1007/978-981-97-1083-6_24
State	Published - 2024
Event	3rd International Conference on Autonomous Unmanned Systems, ICAUS 2023 - Nanjing, China Duration: 9 Sep 2023 → 11 Sep 2023

Publication series

Name	Lecture Notes in Electrical Engineering
Volume	1171
ISSN (Print)	1876-1100
ISSN (Electronic)	1876-1119

Conference

Conference	3rd International Conference on Autonomous Unmanned Systems, ICAUS 2023
Country/Territory	China
City	Nanjing
Period	9/09/23 → 11/09/23

Keywords

cooperative air combat
hierarchical reinforcement learning
maneuver strategy

Access to Document

10.1007/978-981-97-1083-6_24

Cite this

Hu, J., Guo, K., Xu, Z., & Xu, G. (2024). Autonomous Maneuver Strategy of UAV Wingman Air Combat Based on Hierarchical Reinforcement Learning. In Y. Qu, M. Gu, Y. Niu, & W. Fu (Eds.), Proceedings of 3rd 2023 International Conference on Autonomous Unmanned Systems (3rd ICAUS 2023) - Volume II (pp. 258-267). (Lecture Notes in Electrical Engineering; Vol. 1171). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-981-97-1083-6_24

Hu, Jinwen ; Guo, Kun ; Xu, Zhao et al. / Autonomous Maneuver Strategy of UAV Wingman Air Combat Based on Hierarchical Reinforcement Learning. Proceedings of 3rd 2023 International Conference on Autonomous Unmanned Systems (3rd ICAUS 2023) - Volume II. editor / Yi Qu ; Mancang Gu ; Yifeng Niu ; Wenxing Fu. Springer Science and Business Media Deutschland GmbH, 2024. pp. 258-267 (Lecture Notes in Electrical Engineering).

@inproceedings{ddd6c9a78477453986a078fcc8db5aee,

title = "Autonomous Maneuver Strategy of UAV Wingman Air Combat Based on Hierarchical Reinforcement Learning",

abstract = "Unmanned aerial vehicles (UAVs) play an vital role in modern air combats, where manned fighter (leader) can better perform air combat with the assisting of UAVs. The key technology for cooperative air warfare between UAVs and manned aircraft is UAV autonomous maneuver strategy. In this paper, an autonomous maneuver strategy of UAV wingman to assist leader in air combat is proposed. Firstly, based on the mission scenario and process of cooperative air combat, the UAV-assisted leader air combat model is established. Second, a two-level maneuver strategy method of UAV based on hierarchical reinforcement learning is designed, where the low-level agents control the game with enemy and manned fighter collaboration, and they are selected by a high-level agent. The reward functions of each agent are designed according to the relative situation to leader and enemy. Finally, the simulation experiment shows that the UAV can assist leader to strike enemy, proving the effectiveness of the proposed maneuver strategy.",

keywords = "cooperative air combat, hierarchical reinforcement learning, maneuver strategy",

author = "Jinwen Hu and Kun Guo and Zhao Xu and Gang Xu",

note = "Publisher Copyright: {\textcopyright} Beijing HIWING Scientific and Technological Information Institute 2024.; 3rd International Conference on Autonomous Unmanned Systems, ICAUS 2023 ; Conference date: 09-09-2023 Through 11-09-2023",

year = "2024",

doi = "10.1007/978-981-97-1083-6_24",

language = "英语",

isbn = "9789819710829",

series = "Lecture Notes in Electrical Engineering",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "258--267",

editor = "Yi Qu and Mancang Gu and Yifeng Niu and Wenxing Fu",

booktitle = "Proceedings of 3rd 2023 International Conference on Autonomous Unmanned Systems (3rd ICAUS 2023) - Volume II",

}

Hu, J, Guo, K, Xu, Z & Xu, G 2024, Autonomous Maneuver Strategy of UAV Wingman Air Combat Based on Hierarchical Reinforcement Learning. in Y Qu, M Gu, Y Niu & W Fu (eds), Proceedings of 3rd 2023 International Conference on Autonomous Unmanned Systems (3rd ICAUS 2023) - Volume II. Lecture Notes in Electrical Engineering, vol. 1171, Springer Science and Business Media Deutschland GmbH, pp. 258-267, 3rd International Conference on Autonomous Unmanned Systems, ICAUS 2023, Nanjing, China, 9/09/23. https://doi.org/10.1007/978-981-97-1083-6_24

Autonomous Maneuver Strategy of UAV Wingman Air Combat Based on Hierarchical Reinforcement Learning. / Hu, Jinwen; Guo, Kun; Xu, Zhao et al.
Proceedings of 3rd 2023 International Conference on Autonomous Unmanned Systems (3rd ICAUS 2023) - Volume II. ed. / Yi Qu; Mancang Gu; Yifeng Niu; Wenxing Fu. Springer Science and Business Media Deutschland GmbH, 2024. p. 258-267 (Lecture Notes in Electrical Engineering; Vol. 1171).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Autonomous Maneuver Strategy of UAV Wingman Air Combat Based on Hierarchical Reinforcement Learning

AU - Hu, Jinwen

AU - Guo, Kun

AU - Xu, Zhao

AU - Xu, Gang

N1 - Publisher Copyright: © Beijing HIWING Scientific and Technological Information Institute 2024.

PY - 2024

Y1 - 2024

N2 - Unmanned aerial vehicles (UAVs) play an vital role in modern air combats, where manned fighter (leader) can better perform air combat with the assisting of UAVs. The key technology for cooperative air warfare between UAVs and manned aircraft is UAV autonomous maneuver strategy. In this paper, an autonomous maneuver strategy of UAV wingman to assist leader in air combat is proposed. Firstly, based on the mission scenario and process of cooperative air combat, the UAV-assisted leader air combat model is established. Second, a two-level maneuver strategy method of UAV based on hierarchical reinforcement learning is designed, where the low-level agents control the game with enemy and manned fighter collaboration, and they are selected by a high-level agent. The reward functions of each agent are designed according to the relative situation to leader and enemy. Finally, the simulation experiment shows that the UAV can assist leader to strike enemy, proving the effectiveness of the proposed maneuver strategy.

AB - Unmanned aerial vehicles (UAVs) play an vital role in modern air combats, where manned fighter (leader) can better perform air combat with the assisting of UAVs. The key technology for cooperative air warfare between UAVs and manned aircraft is UAV autonomous maneuver strategy. In this paper, an autonomous maneuver strategy of UAV wingman to assist leader in air combat is proposed. Firstly, based on the mission scenario and process of cooperative air combat, the UAV-assisted leader air combat model is established. Second, a two-level maneuver strategy method of UAV based on hierarchical reinforcement learning is designed, where the low-level agents control the game with enemy and manned fighter collaboration, and they are selected by a high-level agent. The reward functions of each agent are designed according to the relative situation to leader and enemy. Finally, the simulation experiment shows that the UAV can assist leader to strike enemy, proving the effectiveness of the proposed maneuver strategy.

KW - cooperative air combat

KW - hierarchical reinforcement learning

KW - maneuver strategy

UR - http://www.scopus.com/inward/record.url?scp=85192559621&partnerID=8YFLogxK

U2 - 10.1007/978-981-97-1083-6_24

DO - 10.1007/978-981-97-1083-6_24

M3 - 会议稿件

AN - SCOPUS:85192559621

SN - 9789819710829

T3 - Lecture Notes in Electrical Engineering

SP - 258

EP - 267

BT - Proceedings of 3rd 2023 International Conference on Autonomous Unmanned Systems (3rd ICAUS 2023) - Volume II

A2 - Qu, Yi

A2 - Gu, Mancang

A2 - Niu, Yifeng

A2 - Fu, Wenxing

PB - Springer Science and Business Media Deutschland GmbH

T2 - 3rd International Conference on Autonomous Unmanned Systems, ICAUS 2023

Y2 - 9 September 2023 through 11 September 2023

ER -

Hu J, Guo K, Xu Z, Xu G. Autonomous Maneuver Strategy of UAV Wingman Air Combat Based on Hierarchical Reinforcement Learning. In Qu Y, Gu M, Niu Y, Fu W, editors, Proceedings of 3rd 2023 International Conference on Autonomous Unmanned Systems (3rd ICAUS 2023) - Volume II. Springer Science and Business Media Deutschland GmbH. 2024. p. 258-267. (Lecture Notes in Electrical Engineering). doi: 10.1007/978-981-97-1083-6_24