Distributed intelligent decision-making for two-stage orbital pursuit-evasion game of spacecraft swarm

With the continuous increase in the number of spacecraft and the growing complexity of the space environment, in order to conduct on-orbit operations on failed spacecraft, it is necessary to chase and follow the targets in different stages. How to deal with the non-cooperation and incomplete information of multiple targets and leverage the advantages of spacecraft swarm is a key issue, which is significant to enhance the efficiency of the on-orbit service, however, is also a core difficulty in the orbital game. In this paper, we propose a distributed intelligent decision-making approach for spacecraft swarm pursuit-evasion game to achieve rapid chasing and safe following of non-cooperative targets in elliptical orbits. Firstly, the RN-PPO (Residual Network Proximal Policy Optimization) algorithm in the frame of reinforcement learning is designed, which utilizes residual blocks to enhance the actor network of the PPO (Proximal Policy Optimization). The input of the module is directly connected to the output, enabling skip connections between layers in the neural network architecture. Residual learning accelerates neural network training and enhances network representational capabilities. Secondly, considering various constraints, a spacecraft swarm game strategy solving approach is designed based on the RN-PPO algorithm. The network architecture and training environment are devised based on the characteristics of the two-stage game and impulse maneuvering. Adaptive reward functions for different mission stages are designed to guide spacecraft swarm to chase and follow targets effectively. Finally, numerical simulations demonstrate the effectiveness and robustness of the proposed RN-PPO algorithm, which significantly enhances the training efficiency for swarm pursuit-evasion game.