Enhancing Simultaneous Arrival at a Dynamic Target: A Hybrid Approach with Proximal Policy Optimization and Expert Rules

This paper addresses the complex challenge of ensuring simultaneous arrival of multiple agents at a dynamic target, a critical requirement for operations such as roundups and saturation attacks. Traditional guidance systems have faced significant hurdles due to inaccurate flight time estimates and difficulties adapting to high-speed maneuvers. To overcome these limitations, we introduce a novel framework that utilizes Proximal Policy Optimization (PPO) and expert rules. This approach leverages distributed computing to enable autonomous decision-making among agents, thereby simplifying the deep reinforcement learning model and reducing computational overhead, which enhances scalability and adaptability. Additionally, we incorporate an angular velocity reward into the reward function, improving the predictability and effectiveness of maneuvers, particularly for targets with high-speed and irregular trajectories. The proposed methods have been rigorously tested through numerous simulations and high-fidelity scenarios, confirming their robustness and superior performance over traditional and enhanced proportional guidance systems.