UAV Autonomous Navigation Based on Multi-modal Perception: A Deep Hierarchical Reinforcement Learning Method

Kai Kou; Gang Yang; Wenqi Zhang; Chenyi Wang; Yuan Yao; Xingshe Zhou

doi:10.1007/978-981-99-0301-6_4

UAV Autonomous Navigation Based on Multi-modal Perception: A Deep Hierarchical Reinforcement Learning Method

Kai Kou, Gang Yang, Wenqi Zhang, Chenyi Wang, Yuan Yao, Xingshe Zhou

School of Computer Science

Northwestern Polytechnical University Xian

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

Autonomous navigation is a highly desirable capability for Unmanned Aerial Vehicle (UAV). In this paper, the problem of autonomous navigation of UAV in unknown dynamic environments is addressed. Specifically, we propose a visual-inertial multi-modal sensor data perception framework based on a hierarchical reinforcement learning paradigm. This model consists of a high-level behavior selection model and a low-level policy execution model. The high-level model learns a stable and reliable behavior selection strategy. The low-level model decomposes the UAV navigation task into two simpler subtasks, which respectively achieve obstacle avoidance and goal approximation, which effectively learns high-level semantic information about the scene and narrows the strategy space. Furthermore, extensive simulation results are provided to confirm the superiority of the proposed method in terms of convergence and effectiveness compared to state-of-the-art methods.

Original language	English
Title of host publication	Intelligent Robotics - 3rd China Annual Conference, CCF CIRAC 2022, Proceedings
Editors	Zhiwen Yu, Xinhong Hei, Duanling Li, Xianhua Song, Zeguang Lu
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	47-56
Number of pages	10
ISBN (Print)	9789819903009
DOIs	https://doi.org/10.1007/978-981-99-0301-6_4
State	Published - 2023
Event	3rd China Intelligent Robotics Annual Conference, CCF CIRAC 2022 - Xi'an, China Duration: 16 Dec 2022 → 18 Dec 2022

Publication series

Name	Communications in Computer and Information Science
Volume	1770 CCIS
ISSN (Print)	1865-0929
ISSN (Electronic)	1865-0937

Conference

Conference	3rd China Intelligent Robotics Annual Conference, CCF CIRAC 2022
Country/Territory	China
City	Xi'an
Period	16/12/22 → 18/12/22

Keywords

Autonomous navigation
Hierarchical reinforcement learning
Multi-modal
Unmanned Aerial Vehicles (UAV)

Access to Document

10.1007/978-981-99-0301-6_4

Cite this

Kou, K., Yang, G., Zhang, W., Wang, C., Yao, Y., & Zhou, X. (2023). UAV Autonomous Navigation Based on Multi-modal Perception: A Deep Hierarchical Reinforcement Learning Method. In Z. Yu, X. Hei, D. Li, X. Song, & Z. Lu (Eds.), Intelligent Robotics - 3rd China Annual Conference, CCF CIRAC 2022, Proceedings (pp. 47-56). (Communications in Computer and Information Science; Vol. 1770 CCIS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-981-99-0301-6_4

Kou, Kai ; Yang, Gang ; Zhang, Wenqi et al. / UAV Autonomous Navigation Based on Multi-modal Perception : A Deep Hierarchical Reinforcement Learning Method. Intelligent Robotics - 3rd China Annual Conference, CCF CIRAC 2022, Proceedings. editor / Zhiwen Yu ; Xinhong Hei ; Duanling Li ; Xianhua Song ; Zeguang Lu. Springer Science and Business Media Deutschland GmbH, 2023. pp. 47-56 (Communications in Computer and Information Science).

@inproceedings{13250ee52a3a4d3ba9d0c15fbcd36602,

title = "UAV Autonomous Navigation Based on Multi-modal Perception: A Deep Hierarchical Reinforcement Learning Method",

abstract = "Autonomous navigation is a highly desirable capability for Unmanned Aerial Vehicle (UAV). In this paper, the problem of autonomous navigation of UAV in unknown dynamic environments is addressed. Specifically, we propose a visual-inertial multi-modal sensor data perception framework based on a hierarchical reinforcement learning paradigm. This model consists of a high-level behavior selection model and a low-level policy execution model. The high-level model learns a stable and reliable behavior selection strategy. The low-level model decomposes the UAV navigation task into two simpler subtasks, which respectively achieve obstacle avoidance and goal approximation, which effectively learns high-level semantic information about the scene and narrows the strategy space. Furthermore, extensive simulation results are provided to confirm the superiority of the proposed method in terms of convergence and effectiveness compared to state-of-the-art methods.",

keywords = "Autonomous navigation, Hierarchical reinforcement learning, Multi-modal, Unmanned Aerial Vehicles (UAV)",

author = "Kai Kou and Gang Yang and Wenqi Zhang and Chenyi Wang and Yuan Yao and Xingshe Zhou",

note = "Publisher Copyright: {\textcopyright} 2023, The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.; 3rd China Intelligent Robotics Annual Conference, CCF CIRAC 2022 ; Conference date: 16-12-2022 Through 18-12-2022",

year = "2023",

doi = "10.1007/978-981-99-0301-6_4",

language = "英语",

isbn = "9789819903009",

series = "Communications in Computer and Information Science",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "47--56",

editor = "Zhiwen Yu and Xinhong Hei and Duanling Li and Xianhua Song and Zeguang Lu",

booktitle = "Intelligent Robotics - 3rd China Annual Conference, CCF CIRAC 2022, Proceedings",

}

Kou, K, Yang, G, Zhang, W, Wang, C, Yao, Y & Zhou, X 2023, UAV Autonomous Navigation Based on Multi-modal Perception: A Deep Hierarchical Reinforcement Learning Method. in Z Yu, X Hei, D Li, X Song & Z Lu (eds), Intelligent Robotics - 3rd China Annual Conference, CCF CIRAC 2022, Proceedings. Communications in Computer and Information Science, vol. 1770 CCIS, Springer Science and Business Media Deutschland GmbH, pp. 47-56, 3rd China Intelligent Robotics Annual Conference, CCF CIRAC 2022, Xi'an, China, 16/12/22. https://doi.org/10.1007/978-981-99-0301-6_4

UAV Autonomous Navigation Based on Multi-modal Perception: A Deep Hierarchical Reinforcement Learning Method. / Kou, Kai; Yang, Gang; Zhang, Wenqi et al.
Intelligent Robotics - 3rd China Annual Conference, CCF CIRAC 2022, Proceedings. ed. / Zhiwen Yu; Xinhong Hei; Duanling Li; Xianhua Song; Zeguang Lu. Springer Science and Business Media Deutschland GmbH, 2023. p. 47-56 (Communications in Computer and Information Science; Vol. 1770 CCIS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - UAV Autonomous Navigation Based on Multi-modal Perception

T2 - 3rd China Intelligent Robotics Annual Conference, CCF CIRAC 2022

AU - Kou, Kai

AU - Yang, Gang

AU - Zhang, Wenqi

AU - Wang, Chenyi

AU - Yao, Yuan

AU - Zhou, Xingshe

PY - 2023

Y1 - 2023

N2 - Autonomous navigation is a highly desirable capability for Unmanned Aerial Vehicle (UAV). In this paper, the problem of autonomous navigation of UAV in unknown dynamic environments is addressed. Specifically, we propose a visual-inertial multi-modal sensor data perception framework based on a hierarchical reinforcement learning paradigm. This model consists of a high-level behavior selection model and a low-level policy execution model. The high-level model learns a stable and reliable behavior selection strategy. The low-level model decomposes the UAV navigation task into two simpler subtasks, which respectively achieve obstacle avoidance and goal approximation, which effectively learns high-level semantic information about the scene and narrows the strategy space. Furthermore, extensive simulation results are provided to confirm the superiority of the proposed method in terms of convergence and effectiveness compared to state-of-the-art methods.

AB - Autonomous navigation is a highly desirable capability for Unmanned Aerial Vehicle (UAV). In this paper, the problem of autonomous navigation of UAV in unknown dynamic environments is addressed. Specifically, we propose a visual-inertial multi-modal sensor data perception framework based on a hierarchical reinforcement learning paradigm. This model consists of a high-level behavior selection model and a low-level policy execution model. The high-level model learns a stable and reliable behavior selection strategy. The low-level model decomposes the UAV navigation task into two simpler subtasks, which respectively achieve obstacle avoidance and goal approximation, which effectively learns high-level semantic information about the scene and narrows the strategy space. Furthermore, extensive simulation results are provided to confirm the superiority of the proposed method in terms of convergence and effectiveness compared to state-of-the-art methods.

KW - Autonomous navigation

KW - Hierarchical reinforcement learning

KW - Multi-modal

KW - Unmanned Aerial Vehicles (UAV)

UR - http://www.scopus.com/inward/record.url?scp=85151054118&partnerID=8YFLogxK

U2 - 10.1007/978-981-99-0301-6_4

DO - 10.1007/978-981-99-0301-6_4

M3 - 会议稿件

AN - SCOPUS:85151054118

SN - 9789819903009

T3 - Communications in Computer and Information Science

SP - 47

EP - 56

BT - Intelligent Robotics - 3rd China Annual Conference, CCF CIRAC 2022, Proceedings

A2 - Yu, Zhiwen

A2 - Hei, Xinhong

A2 - Li, Duanling

A2 - Song, Xianhua

A2 - Lu, Zeguang

PB - Springer Science and Business Media Deutschland GmbH

Y2 - 16 December 2022 through 18 December 2022

ER -

Kou K, Yang G, Zhang W, Wang C, Yao Y, Zhou X. UAV Autonomous Navigation Based on Multi-modal Perception: A Deep Hierarchical Reinforcement Learning Method. In Yu Z, Hei X, Li D, Song X, Lu Z, editors, Intelligent Robotics - 3rd China Annual Conference, CCF CIRAC 2022, Proceedings. Springer Science and Business Media Deutschland GmbH. 2023. p. 47-56. (Communications in Computer and Information Science). doi: 10.1007/978-981-99-0301-6_4