Target Recognition and Localization of UAV Passing Through Windows

This paper introduces a visual perception system of UAV which solves the perception problem of UAV entering the house through the window during street warfare reconnaissance, that is how to identify and locate the window accurately in real-time and give the maximum safe crossing radius and plane normal vector. In order to solve the above problems, we propose an airborne visual perception system composed of window detection and localization. In the detection part, YOLOv3 algorithm is used to identify the window, and Harris is used to detect the corners of the window edge. In the localization part, the depth camera and visual odometer are combined to locate the window, then the maximum safe crossing radius and the plane normal vector of the window is calculated. In order to verify the performance of the system, we set up a tent for the actual flight experiment.