Dual-UAVs Maneuvering Strategy Generation Algorithm Based on Cooperative Reward Mechanism and MATD3

In order to solve the cooperative maneuvering decision problem of UAVs in dual-UAVs formations in air combat, this paper proposes an air combat maneuvering algorithm based on a cooperative reward mechanism and a distributed Multi-Agent Twin Delayed Deep Deterministic Policy Gradient (MATD3). Firstly, the reward function is designed according to the combat purpose of dual-UAVs air combat. Secondly, To address the sparse reward function problem in air combat, a cooperative reward mechanism is introduced in the reward function based on the idea of cooperative combat in real air combat, and a variable weight superposition method based on the optimal combat distance is introduced in the calculation of immediate reward to reshape the reward function. The dual-UAVs formation confrontation simulation training is conducted under the framework of MATD3 algorithm. The simulation results show that the generated dual-UAVs cooperative air combat maneuver strategy is reasonable and more effective by introducing the collaborative reward mechanism and the combat distance influence factor.