Hierarchical Optimization Design for Autonomous Flight of Vision-Based Quadrotor Using Reinforcement Learning

Although quadrotor has been widely used in practical engineering, its autonomous flight ability needs to be improved in complex operating environment. The autonomous flight problem for quadrotor with monocular vision is investigated, which is divided into control layer and decision layer in this article. Reinforcement learning method is utilized for hierarchical optimization to ensure that quadrotor completes narrow space traversal tasks safely and efficiently. First, considering the dynamic characteristics of the quadrotor with the motor speed as the control input, a parallel policy iteration algorithm is designed for the nonaffine nonlinear system, and the proposed controller can be learned online to improve the fundamental control performance. On this basis, the autonomous decision problem with visual information as input is modeled as a Markov decision process, and a curriculum learning mechanism is introduced to overcome the difficulties caused by sparse reward. At the same time, the clipping function is optimized to improve the learning efficiency of proximal policy optimization (PPO) algorithm for autonomous flight capabilities. Finally, the effectiveness of the proposed intelligent control and decision methods are verified through simulation.