On the Application of Reinforcement Learning in Multi-debris Active Removal Mission Planning

Jianan Yang; Yu Hen Hu; Yong Liu; Xiaolei Hou; Quan Pan

doi:10.1109/ISIE.2019.8781167

On the Application of Reinforcement Learning in Multi-debris Active Removal Mission Planning

Jianan Yang, Yu Hen Hu, Yong Liu, Xiaolei Hou, Quan Pan

School of Automation

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

7 Scopus citations

Abstract

We formulate the active multi-debris removal mission planning task as a Reinforcement-Learning (RL) problem and developed an adjusted Deep Q-Learning (DQN) solution. We propose novel definitions of the state space, action sets, and rewards in the context of active multi-debris removal mission planning. These definitions facilitate recasting the mission planning problem into a RL problem. As such, a powerful DQN algorithm may be applied to solve the mission planning problem using an RL approach. We test this new approach using a subset of Iridium 33 debris cloud. Very encouraging results are observed. Future applications to a reactive autonomous space mission planner are also discussed.

Original language	English
Title of host publication	Proceedings - 2019 IEEE 28th International Symposium on Industrial Electronics, ISIE 2019
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	605-610
Number of pages	6
ISBN (Electronic)	9781728136660
DOIs	https://doi.org/10.1109/ISIE.2019.8781167
State	Published - Jun 2019
Event	28th IEEE International Symposium on Industrial Electronics, ISIE 2019 - Vancouver, Canada Duration: 12 Jun 2019 → 14 Jun 2019

Publication series

Name	IEEE International Symposium on Industrial Electronics
Volume	2019-June

Conference

Conference	28th IEEE International Symposium on Industrial Electronics, ISIE 2019
Country/Territory	Canada
City	Vancouver
Period	12/06/19 → 14/06/19

Keywords

Multi-debris active removal
reinforcement learning
space mission planning

Access to Document

10.1109/ISIE.2019.8781167

Cite this

Yang, J., Hu, Y. H., Liu, Y., Hou, X., & Pan, Q. (2019). On the Application of Reinforcement Learning in Multi-debris Active Removal Mission Planning. In Proceedings - 2019 IEEE 28th International Symposium on Industrial Electronics, ISIE 2019 (pp. 605-610). Article 8781167 (IEEE International Symposium on Industrial Electronics; Vol. 2019-June). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ISIE.2019.8781167

@inproceedings{6bed216854b44cf491c5b8fc6bde24da,

title = "On the Application of Reinforcement Learning in Multi-debris Active Removal Mission Planning",

abstract = "We formulate the active multi-debris removal mission planning task as a Reinforcement-Learning (RL) problem and developed an adjusted Deep Q-Learning (DQN) solution. We propose novel definitions of the state space, action sets, and rewards in the context of active multi-debris removal mission planning. These definitions facilitate recasting the mission planning problem into a RL problem. As such, a powerful DQN algorithm may be applied to solve the mission planning problem using an RL approach. We test this new approach using a subset of Iridium 33 debris cloud. Very encouraging results are observed. Future applications to a reactive autonomous space mission planner are also discussed.",

keywords = "Multi-debris active removal, reinforcement learning, space mission planning",

author = "Jianan Yang and Hu, {Yu Hen} and Yong Liu and Xiaolei Hou and Quan Pan",

note = "Publisher Copyright: {\textcopyright} 2019 IEEE.; 28th IEEE International Symposium on Industrial Electronics, ISIE 2019 ; Conference date: 12-06-2019 Through 14-06-2019",

year = "2019",

month = jun,

doi = "10.1109/ISIE.2019.8781167",

language = "英语",

series = "IEEE International Symposium on Industrial Electronics",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "605--610",

booktitle = "Proceedings - 2019 IEEE 28th International Symposium on Industrial Electronics, ISIE 2019",

}

Yang, J, Hu, YH, Liu, Y, Hou, X & Pan, Q 2019, On the Application of Reinforcement Learning in Multi-debris Active Removal Mission Planning. in Proceedings - 2019 IEEE 28th International Symposium on Industrial Electronics, ISIE 2019., 8781167, IEEE International Symposium on Industrial Electronics, vol. 2019-June, Institute of Electrical and Electronics Engineers Inc., pp. 605-610, 28th IEEE International Symposium on Industrial Electronics, ISIE 2019, Vancouver, Canada, 12/06/19. https://doi.org/10.1109/ISIE.2019.8781167

On the Application of Reinforcement Learning in Multi-debris Active Removal Mission Planning. / Yang, Jianan; Hu, Yu Hen; Liu, Yong et al.
Proceedings - 2019 IEEE 28th International Symposium on Industrial Electronics, ISIE 2019. Institute of Electrical and Electronics Engineers Inc., 2019. p. 605-610 8781167 (IEEE International Symposium on Industrial Electronics; Vol. 2019-June).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - On the Application of Reinforcement Learning in Multi-debris Active Removal Mission Planning

AU - Yang, Jianan

AU - Hu, Yu Hen

AU - Liu, Yong

AU - Hou, Xiaolei

AU - Pan, Quan

PY - 2019/6

Y1 - 2019/6

N2 - We formulate the active multi-debris removal mission planning task as a Reinforcement-Learning (RL) problem and developed an adjusted Deep Q-Learning (DQN) solution. We propose novel definitions of the state space, action sets, and rewards in the context of active multi-debris removal mission planning. These definitions facilitate recasting the mission planning problem into a RL problem. As such, a powerful DQN algorithm may be applied to solve the mission planning problem using an RL approach. We test this new approach using a subset of Iridium 33 debris cloud. Very encouraging results are observed. Future applications to a reactive autonomous space mission planner are also discussed.

AB - We formulate the active multi-debris removal mission planning task as a Reinforcement-Learning (RL) problem and developed an adjusted Deep Q-Learning (DQN) solution. We propose novel definitions of the state space, action sets, and rewards in the context of active multi-debris removal mission planning. These definitions facilitate recasting the mission planning problem into a RL problem. As such, a powerful DQN algorithm may be applied to solve the mission planning problem using an RL approach. We test this new approach using a subset of Iridium 33 debris cloud. Very encouraging results are observed. Future applications to a reactive autonomous space mission planner are also discussed.

KW - Multi-debris active removal

KW - reinforcement learning

KW - space mission planning

UR - http://www.scopus.com/inward/record.url?scp=85070641517&partnerID=8YFLogxK

U2 - 10.1109/ISIE.2019.8781167

DO - 10.1109/ISIE.2019.8781167

M3 - 会议稿件

AN - SCOPUS:85070641517

T3 - IEEE International Symposium on Industrial Electronics

SP - 605

EP - 610

BT - Proceedings - 2019 IEEE 28th International Symposium on Industrial Electronics, ISIE 2019

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 28th IEEE International Symposium on Industrial Electronics, ISIE 2019

Y2 - 12 June 2019 through 14 June 2019

ER -

Yang J, Hu YH, Liu Y, Hou X, Pan Q. On the Application of Reinforcement Learning in Multi-debris Active Removal Mission Planning. In Proceedings - 2019 IEEE 28th International Symposium on Industrial Electronics, ISIE 2019. Institute of Electrical and Electronics Engineers Inc. 2019. p. 605-610. 8781167. (IEEE International Symposium on Industrial Electronics). doi: 10.1109/ISIE.2019.8781167

On the Application of Reinforcement Learning in Multi-debris Active Removal Mission Planning

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this