TY - JOUR
T1 - Global-and-Local Attention-Based Reinforcement Learning for Cooperative Behaviour Control of Multiple UAVs
AU - Chen, Jinchao
AU - Li, Tingyang
AU - Zhang, Ying
AU - You, Tao
AU - Lu, Yantao
AU - Tiwari, Prayag
AU - Kumar, Neeraj
N1 - Publisher Copyright:
© 2023 IEEE.
PY - 2024
Y1 - 2024
N2 - Due to the strong adaptability and high flexibility, unmanned aerial vehicles (UAVs) have been extensively studied and widely applied in both civil and military applications. Although UAVs can achieve significant cost reduction and performance enhancement in large-scale systems by taking full advantage of their cooperation and coordination, they result in a serious cooperative behaviour control problem. Especially in dynamic environments, the cooperative behaviour control problem which has to quickly produce a safe and effective behaviour decision for each UAV to achieve group missions, is NP-hard and difficult to settle. In this work, we design a global-and-local attention-based reinforcement learning algorithm for the cooperative behaviour control problem of UAVs. First, with the motion and coordination models, we analyze the collision avoidance, motion state update, and task execution constraints of multiple UAVs, and abstract the cooperative behaviour control problem as a multi-constraint decision-making one. Then, inspired from the human-learning process where more attention is devoted to the important parts of data, we design a multi-agent reinforcement learning algorithm with a global-and-local attention mechanism to cooperatively control the behaviours of UAVs and achieve the coordination. Simulation experiments in a multi-agent particle environment provided by OpenAI are conducted to verify the effectiveness and efficiency of the proposed approach. Compared with baselines, our approach shows significant advantages in mean reward, training time, and coordination effect.
AB - Due to the strong adaptability and high flexibility, unmanned aerial vehicles (UAVs) have been extensively studied and widely applied in both civil and military applications. Although UAVs can achieve significant cost reduction and performance enhancement in large-scale systems by taking full advantage of their cooperation and coordination, they result in a serious cooperative behaviour control problem. Especially in dynamic environments, the cooperative behaviour control problem which has to quickly produce a safe and effective behaviour decision for each UAV to achieve group missions, is NP-hard and difficult to settle. In this work, we design a global-and-local attention-based reinforcement learning algorithm for the cooperative behaviour control problem of UAVs. First, with the motion and coordination models, we analyze the collision avoidance, motion state update, and task execution constraints of multiple UAVs, and abstract the cooperative behaviour control problem as a multi-constraint decision-making one. Then, inspired from the human-learning process where more attention is devoted to the important parts of data, we design a multi-agent reinforcement learning algorithm with a global-and-local attention mechanism to cooperatively control the behaviours of UAVs and achieve the coordination. Simulation experiments in a multi-agent particle environment provided by OpenAI are conducted to verify the effectiveness and efficiency of the proposed approach. Compared with baselines, our approach shows significant advantages in mean reward, training time, and coordination effect.
KW - Global-and-local attention mechanism
KW - cooperative behaviour control
KW - multi-constraint decision-making
KW - multiple UAVs
KW - reinforcement learning
UR - http://www.scopus.com/inward/record.url?scp=85179001590&partnerID=8YFLogxK
U2 - 10.1109/TVT.2023.3327571
DO - 10.1109/TVT.2023.3327571
M3 - 文章
AN - SCOPUS:85179001590
SN - 0018-9545
VL - 73
SP - 4194
EP - 4206
JO - IEEE Transactions on Vehicular Technology
JF - IEEE Transactions on Vehicular Technology
IS - 3
ER -