A Novel Framework for Adaptive Quadruped Robot Locomotion Learning in Uncertain Environments

Mengyuan Li; Bin Guo; Kaixing Zhao; Ruonan Xu; Sicong Liu; Sitong Mao; Shunbo Zhou; Qiaobo Xu; Zhiwen Yu

doi:10.1007/978-981-99-9896-8_10

A Novel Framework for Adaptive Quadruped Robot Locomotion Learning in Uncertain Environments

Mengyuan Li, Bin Guo, Kaixing Zhao, Ruonan Xu, Sicong Liu, Sitong Mao, Shunbo Zhou, Qiaobo Xu, Zhiwen Yu

School of Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

1 Scopus citations

Abstract

Learning diverse and flexible locomotion strategies in uncertain environments has been a longstanding challenge for quadruped robots. Although recent progress in domain randomization has partially tackled this difficulty by training policies on a wide range of potential factors, there is still a great need for improving efficiency. In this paper, we propose a novel framework for adaptive quadruped robot locomotion learning in uncertain environments. Our method is based on data-efficient reinforcement learning and learns simulation parameters iteratively. We also propose a novel Sampling-Interval-Adaptive Identification (SIAI) strategy that uses historical parameters to optimize sampling distribution and then improve identification accuracy. Final evaluations based on multiple robotic locomotion tasks showed superiority of our method over baselines.

Original language	English
Title of host publication	Green, Pervasive, and Cloud Computing - 18th International Conference, GPC 2023, Proceedings
Editors	Hai Jin, Zhiwen Yu, Chen Yu, Xiaokang Zhou, Zeguang Lu, Xianhua Song
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	139-154
Number of pages	16
ISBN (Print)	9789819998951
DOIs	https://doi.org/10.1007/978-981-99-9896-8_10
State	Published - 2024
Event	18th International Conference on Green, Pervasive, and Cloud Computing, GPC 2023 - Harbin, China Duration: 22 Sep 2023 → 24 Sep 2023

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	14504
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	18th International Conference on Green, Pervasive, and Cloud Computing, GPC 2023
Country/Territory	China
City	Harbin
Period	22/09/23 → 24/09/23

Keywords

Identification Strategy
Reinforcement Learning
Robot Intelligence
Transfer Learning

Access to Document

10.1007/978-981-99-9896-8_10

Cite this

Li, M., Guo, B., Zhao, K., Xu, R., Liu, S., Mao, S., Zhou, S., Xu, Q., & Yu, Z. (2024). A Novel Framework for Adaptive Quadruped Robot Locomotion Learning in Uncertain Environments. In H. Jin, Z. Yu, C. Yu, X. Zhou, Z. Lu, & X. Song (Eds.), Green, Pervasive, and Cloud Computing - 18th International Conference, GPC 2023, Proceedings (pp. 139-154). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 14504). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-981-99-9896-8_10

Li, Mengyuan ; Guo, Bin ; Zhao, Kaixing et al. / A Novel Framework for Adaptive Quadruped Robot Locomotion Learning in Uncertain Environments. Green, Pervasive, and Cloud Computing - 18th International Conference, GPC 2023, Proceedings. editor / Hai Jin ; Zhiwen Yu ; Chen Yu ; Xiaokang Zhou ; Zeguang Lu ; Xianhua Song. Springer Science and Business Media Deutschland GmbH, 2024. pp. 139-154 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{7d4fd03fcb7245a49409e91a26b853a3,

title = "A Novel Framework for Adaptive Quadruped Robot Locomotion Learning in Uncertain Environments",

abstract = "Learning diverse and flexible locomotion strategies in uncertain environments has been a longstanding challenge for quadruped robots. Although recent progress in domain randomization has partially tackled this difficulty by training policies on a wide range of potential factors, there is still a great need for improving efficiency. In this paper, we propose a novel framework for adaptive quadruped robot locomotion learning in uncertain environments. Our method is based on data-efficient reinforcement learning and learns simulation parameters iteratively. We also propose a novel Sampling-Interval-Adaptive Identification (SIAI) strategy that uses historical parameters to optimize sampling distribution and then improve identification accuracy. Final evaluations based on multiple robotic locomotion tasks showed superiority of our method over baselines.",

keywords = "Identification Strategy, Reinforcement Learning, Robot Intelligence, Transfer Learning",

author = "Mengyuan Li and Bin Guo and Kaixing Zhao and Ruonan Xu and Sicong Liu and Sitong Mao and Shunbo Zhou and Qiaobo Xu and Zhiwen Yu",

note = "Publisher Copyright: {\textcopyright} The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd 2024.; 18th International Conference on Green, Pervasive, and Cloud Computing, GPC 2023 ; Conference date: 22-09-2023 Through 24-09-2023",

year = "2024",

doi = "10.1007/978-981-99-9896-8_10",

language = "英语",

isbn = "9789819998951",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "139--154",

editor = "Hai Jin and Zhiwen Yu and Chen Yu and Xiaokang Zhou and Zeguang Lu and Xianhua Song",

booktitle = "Green, Pervasive, and Cloud Computing - 18th International Conference, GPC 2023, Proceedings",

}

Li, M, Guo, B, Zhao, K, Xu, R, Liu, S, Mao, S, Zhou, S, Xu, Q & Yu, Z 2024, A Novel Framework for Adaptive Quadruped Robot Locomotion Learning in Uncertain Environments. in H Jin, Z Yu, C Yu, X Zhou, Z Lu & X Song (eds), Green, Pervasive, and Cloud Computing - 18th International Conference, GPC 2023, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 14504, Springer Science and Business Media Deutschland GmbH, pp. 139-154, 18th International Conference on Green, Pervasive, and Cloud Computing, GPC 2023, Harbin, China, 22/09/23. https://doi.org/10.1007/978-981-99-9896-8_10

A Novel Framework for Adaptive Quadruped Robot Locomotion Learning in Uncertain Environments. / Li, Mengyuan; Guo, Bin; Zhao, Kaixing et al.
Green, Pervasive, and Cloud Computing - 18th International Conference, GPC 2023, Proceedings. ed. / Hai Jin; Zhiwen Yu; Chen Yu; Xiaokang Zhou; Zeguang Lu; Xianhua Song. Springer Science and Business Media Deutschland GmbH, 2024. p. 139-154 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 14504).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - A Novel Framework for Adaptive Quadruped Robot Locomotion Learning in Uncertain Environments

AU - Li, Mengyuan

AU - Guo, Bin

AU - Zhao, Kaixing

AU - Xu, Ruonan

AU - Liu, Sicong

AU - Mao, Sitong

AU - Zhou, Shunbo

AU - Xu, Qiaobo

AU - Yu, Zhiwen

N1 - Publisher Copyright: © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd 2024.

PY - 2024

Y1 - 2024

N2 - Learning diverse and flexible locomotion strategies in uncertain environments has been a longstanding challenge for quadruped robots. Although recent progress in domain randomization has partially tackled this difficulty by training policies on a wide range of potential factors, there is still a great need for improving efficiency. In this paper, we propose a novel framework for adaptive quadruped robot locomotion learning in uncertain environments. Our method is based on data-efficient reinforcement learning and learns simulation parameters iteratively. We also propose a novel Sampling-Interval-Adaptive Identification (SIAI) strategy that uses historical parameters to optimize sampling distribution and then improve identification accuracy. Final evaluations based on multiple robotic locomotion tasks showed superiority of our method over baselines.

AB - Learning diverse and flexible locomotion strategies in uncertain environments has been a longstanding challenge for quadruped robots. Although recent progress in domain randomization has partially tackled this difficulty by training policies on a wide range of potential factors, there is still a great need for improving efficiency. In this paper, we propose a novel framework for adaptive quadruped robot locomotion learning in uncertain environments. Our method is based on data-efficient reinforcement learning and learns simulation parameters iteratively. We also propose a novel Sampling-Interval-Adaptive Identification (SIAI) strategy that uses historical parameters to optimize sampling distribution and then improve identification accuracy. Final evaluations based on multiple robotic locomotion tasks showed superiority of our method over baselines.

KW - Identification Strategy

KW - Reinforcement Learning

KW - Robot Intelligence

KW - Transfer Learning

UR - http://www.scopus.com/inward/record.url?scp=85184293466&partnerID=8YFLogxK

U2 - 10.1007/978-981-99-9896-8_10

DO - 10.1007/978-981-99-9896-8_10

M3 - 会议稿件

AN - SCOPUS:85184293466

SN - 9789819998951

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 139

EP - 154

BT - Green, Pervasive, and Cloud Computing - 18th International Conference, GPC 2023, Proceedings

A2 - Jin, Hai

A2 - Yu, Zhiwen

A2 - Yu, Chen

A2 - Zhou, Xiaokang

A2 - Lu, Zeguang

A2 - Song, Xianhua

PB - Springer Science and Business Media Deutschland GmbH

T2 - 18th International Conference on Green, Pervasive, and Cloud Computing, GPC 2023

Y2 - 22 September 2023 through 24 September 2023

ER -

Li M, Guo B, Zhao K, Xu R, Liu S, Mao S et al. A Novel Framework for Adaptive Quadruped Robot Locomotion Learning in Uncertain Environments. In Jin H, Yu Z, Yu C, Zhou X, Lu Z, Song X, editors, Green, Pervasive, and Cloud Computing - 18th International Conference, GPC 2023, Proceedings. Springer Science and Business Media Deutschland GmbH. 2024. p. 139-154. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-981-99-9896-8_10