Multiple-UAV Reinforcement Learning Algorithm Based on Improved PPO in Ray Framework

Guang Zhan, Xinmiao Zhang, Zhongchao Li, Lin Xu, Deyun Zhou, Zhen Yang

Research output: Contribution to journalArticlepeer-review

42 Scopus citations

Abstract

Distributed multi-agent collaborative decision-making technology is the key to general artificial intelligence. This paper takes the self-developed Unity3D collaborative combat environment as the test scenario, setting a task that requires heterogeneous unmanned aerial vehicles (UAVs) to perform a distributed decision-making and complete cooperation task. Aiming at the problem of the traditional proximal policy optimization (PPO) algorithm’s poor performance in the field of complex multi-agent collaboration scenarios based on the distributed training framework Ray, the Critic network in the PPO algorithm is improved to learn a centralized value function, and the muti-agent proximal policy optimization (MAPPO) algorithm is proposed. At the same time, the inheritance training method based on course learning is adopted to improve the generalization performance of the algorithm. In the experiment, MAPPO can obtain the highest average accumulate reward compared with other algorithms and can complete the task goal with the fewest steps after convergence, which fully demonstrates that the MAPPO algorithm outperforms the state-of-the-art.

Original languageEnglish
Article number166
JournalDrones
Volume6
Issue number7
DOIs
StatePublished - Jul 2022

Keywords

  • PPO
  • Ray
  • curriculum learning
  • deep reinforcement learning
  • multiple UAVs

Fingerprint

Dive into the research topics of 'Multiple-UAV Reinforcement Learning Algorithm Based on Improved PPO in Ray Framework'. Together they form a unique fingerprint.

Cite this