Reinforcement learning based self-triggered sliding mode fault-tolerant control for underwater vehicles

In this paper, an observer-based fault-tolerant control using reinforcement learning (RL) is proposed for an autonomous underwater vehicle (AUV) equipped with a bow rudder and four independent stern rudders arranged in an X-configuration. To enable real-time fault detection, an integrated adaptive second-order sliding mode observer is employed to estimate rudder effectiveness factors online. Based on the observer’s fault indicators, an Actor-Critic RL-based controller is trained to adaptively generate fault-compensating control inputs by minimizing tracking errors under fault conditions. To alleviate computational burden and decrease control update frequency, a self-triggered strategy is introduced, and the triggering condition is defined by both trajectory tracking errors and fault detection indicators. Finally, the fault-compensating control signals are mapped by bounded nonlinear transformation and integrated with pseudo-inverse allocation to ensure the actuator feasibility and admissibility. Results of numerical simulations and real experiments demonstrate the efficiency of the proposed method, maintaining the robust tracking performance under actuator faults. The proposed method achieves a 55.8 % reduction in lateral tracking error and a 28.1 % decrease in control energy consumption under two-rudder faults, relative to conventional pseudoinverse allocation.