Soft Constrained Reinforcement Learning for USV Dynamics System Identification

Dynamics system identification for unmanned surface vehicles is challenging due to the complex fluid-solid interactions and the requirement for labor-intensive tests. This paper presents a soft constrained reinforcement learning method to obtain stable and reliable dynamic parameters for unmanned surface vehicles using limited real-world trials. First, a simplified three degree of freedom planar model with nine unknown physical parameters is established. Second, a robust cost function with high-residual-ratio truncation is developed to mitigate the influence of outliers by discarding a small fraction of time steps associated with the largest residuals. To avoid losing physical meaning, soft constraints are applied to the unknowns through a sigmoid mapping. The means and variances of the unknown parameters are updated using a symmetric sampling policygradient scheme, which improves the consistency of the identified results. Finally, the proposed method is validated using real-world operation data. The identified parameter values exhibit improved consistency across operating conditions. Comparison against conventional least squares and Markov chain Monte Carlo methods demonstrates that the proposed approach effectively avoids physically implausible parameter estimates and excessive estimation dispersion, thereby enhancing the robustness of USV dynamic-parameter identification.