Deep-Reinforcement-Learning-Based Cooperative Searching of Multi-AUV in Continuous Space With Uncertain Regions

Cooperative searching for unknown targets is a significant challenge for multiple autonomous underwater vehicles (multi-AUV) in uncertain regions. While many existing studies discretize the searching space into small grids to accelerate planning, continuous space-based algorithms generate more adaptive searching strategies in dynamic environments. However, the complexity of the underwater environment and the need for multi-AUV coordination lead to inaccurate spatial modeling and exponential growth in computation time. To address these issues, this article proposes a multiagent continuous soft actor-critic (MACSAC) algorithm for underwater cooperative searching, employing a multiagent deep reinforcement learning (MADRL) framework based on the soft actor-critic (SAC) algorithm. The multisensor cooperative detection model uses a probabilistic target representation based on Gaussian distributions, with Bayesian updating dynamically guiding the searching process. A dynamic dual-agent switching mechanism, which incorporates the MADRL framework, is designed with detection agents and detection-free agents. Moreover, the SAC algorithm is improved for continuous space to enhance cooperative searching efficiency. Simulations are conducted with four AUVs searching for an unknown target region in the underwater environment containing static obstacles. Results show that the proposed method produces smoother paths than discrete grid-based MADRL, and compared to existing continuous MADRL algorithms, reduces the average search steps by 50.70% and the sailing distance by 41.36%. These findings indicate that MACSAC enables efficient, collision-free path planning for multi-AUV systems operating in continuous, uncertain underwater environments.