Continuous-time hierarchical reinforcement learning for satellite pursuit decision

The satellite orbital pursuit game focuses on studying spacecraft maneuvering strategies in space. Traditional numerical methods often face real-time inadequacies and adaptability limitations when dealing with highly nonlinear problems. With the advancement of Deep Reinforcement Learning (DRL) technology, continuous-time orbital control capabilities have significantly improved. Despite this, the existing DRL technologies still need adjustments in action delay and discretization structure to better adapt to practical application scenarios. Combining continuous learning and model planning demonstrates the adaptability of these methods in continuous-time decision problems. Additionally, to more effectively handle action delay issues, a new scheduled action execution technique has been developed. This technique optimizes action execution timing through real-time policy adjustments, thus adapting to the dynamic changes in the orbital environment. A Hierarchical Reinforcement Learning (HRL) strategy was also adopted to simplify the decision-making process for long-distance pursuit tasks by setting phased subgoals to gradually approach the target. The effectiveness of the proposed strategy in practical satellite pursuit scenarios has been verified through simulations of two different tasks.