Autonomous Maneuver Strategy of UAV Wingman Air Combat Based on Hierarchical Reinforcement Learning

Unmanned aerial vehicles (UAVs) play an vital role in modern air combats, where manned fighter (leader) can better perform air combat with the assisting of UAVs. The key technology for cooperative air warfare between UAVs and manned aircraft is UAV autonomous maneuver strategy. In this paper, an autonomous maneuver strategy of UAV wingman to assist leader in air combat is proposed. Firstly, based on the mission scenario and process of cooperative air combat, the UAV-assisted leader air combat model is established. Second, a two-level maneuver strategy method of UAV based on hierarchical reinforcement learning is designed, where the low-level agents control the game with enemy and manned fighter collaboration, and they are selected by a high-level agent. The reward functions of each agent are designed according to the relative situation to leader and enemy. Finally, the simulation experiment shows that the UAV can assist leader to strike enemy, proving the effectiveness of the proposed maneuver strategy.