Deep Reinforcement Learning-based End-to-End Navigation of Mobile Robots With Reward Shaping

This paper proposes an end-to-end autonomous navigation algorithm for unknown environments based on deep reinforcement learning (DRL), which maps the lidar data collected by the robot into control commands. The proposed LM-TD3 algorithm utilizes the Twin Delayed Deep Deterministic(TD3) policy gradient network as the backbone to generate robot action control in continuous spaces. Based on this, the Long Short-Term Memory (LSTM) neural network is introduced into the actor and critic networks, allowing the model to store long-term navigation experiences to increase its ability to perceive and handle surrounding obstacles. Furthermore, a novel reward function in DRL is designed to smooth the motion pose of the robot while controlling the robot to achieve target tracking. Finally, to enhance the early learning efficiency of the DRL network, a Hindsight Experience Replay (HER) strategy is designed specifically for the autonomous navigation system to enhance the convergence speed of the algorithm. To validate the effectiveness of the LM-TD3 algorithm with simulation experiments, scenarios of varying complexities are designed to verify the navigation ability. Compared with the TD3 algorithm, the proposed LMTD3 method can generate shorter paths with enhanced obstacle avoidance capabilities, while also maintaining more stable robot posture control.