Deep Reinforcement Learning-Based Decision-Making for Autonomous Docking of Underwater Vehicles

Autonomous docking of Autonomous Underwater Vehicles is essential for enabling cooperative underwater operations and improving mission efficiency. This study presents a hierarchical docking framework that integrates acoustic-optical guidance, six-degree-of-freedom motion control, and a high-level decision-making module based on discrete Batch-Constrained Q-learning (BCQ), a form of offline deep reinforcement learning. Unlike online approaches, the policy is trained solely from offline data, avoiding risky real-world exploration. The BCQ-based module determines whether to proceed with docking or initiate re-docking, based on relative state information and environmental uncertainty. A tailored reward function is designed to balance docking success, operational efficiency, and safety. Lake experiments validate the proposed approach, demonstrating improved docking robustness under uncertain conditions.