TY - JOUR
T1 - Complex Network Optimization for Fixed-Time Continuous Action Iteration Dilemma by Using Reinforcement Learning
AU - Jia, Zhanxiao
AU - Yu, Dengxiu
AU - Wang, Zhen
AU - Chen, C. L.Philip
AU - Li, Xuelong
N1 - Publisher Copyright:
© 2013 IEEE.
PY - 2024/7/1
Y1 - 2024/7/1
N2 - In this paper, an optimization algorithm based on deep reinforcement learning is proposed to optimize complex networks in fixed-time convergence of continuous action iteration dilemmas. The field of continuous action iterative dilemmas has long been studied, with prior research primarily emphasizing the effectiveness of strategy selection and the stability of strategy evolution. However, the impact of topology on strategy evolution has remained under-explored. The present study fills this gap by examining how the structure of complex networks influences the time required for players to reach Nash Equilibrium and overall payoff. To identify the optimal complex network that ensures fixed-time convergence of continuous action iteration dilemma, achieves the shortest time, and attains the highest overall payoff in the Nash Equilibrium state, a deep reinforcement learning algorithm is designed to optimize the complex network. Firstly, the paper applies the Lyapunov stability theory to analyze the convergence of the fixed-time continuous action iteration dilemma and compute the upper bound of convergence time. Secondly, based on the fixed-time convergence of continuous action iteration dilemma, we establish evaluation criteria based on the time taken by players to reach the Nash Equilibrium and the overall payoff, subsequently designing evaluation functions for complex networks utilizing these criteria. Thirdly, this paper applies a deep reinforcement learning algorithm to resolve the optimization issue associated with the proposed evaluation function, while analyzing the convergence of complex network optimization methods. Lastly, the effectiveness of the proposed method is verified by simulating the dynamic model of snowdrift games and prisoner dilemmas.
AB - In this paper, an optimization algorithm based on deep reinforcement learning is proposed to optimize complex networks in fixed-time convergence of continuous action iteration dilemmas. The field of continuous action iterative dilemmas has long been studied, with prior research primarily emphasizing the effectiveness of strategy selection and the stability of strategy evolution. However, the impact of topology on strategy evolution has remained under-explored. The present study fills this gap by examining how the structure of complex networks influences the time required for players to reach Nash Equilibrium and overall payoff. To identify the optimal complex network that ensures fixed-time convergence of continuous action iteration dilemma, achieves the shortest time, and attains the highest overall payoff in the Nash Equilibrium state, a deep reinforcement learning algorithm is designed to optimize the complex network. Firstly, the paper applies the Lyapunov stability theory to analyze the convergence of the fixed-time continuous action iteration dilemma and compute the upper bound of convergence time. Secondly, based on the fixed-time convergence of continuous action iteration dilemma, we establish evaluation criteria based on the time taken by players to reach the Nash Equilibrium and the overall payoff, subsequently designing evaluation functions for complex networks utilizing these criteria. Thirdly, this paper applies a deep reinforcement learning algorithm to resolve the optimization issue associated with the proposed evaluation function, while analyzing the convergence of complex network optimization methods. Lastly, the effectiveness of the proposed method is verified by simulating the dynamic model of snowdrift games and prisoner dilemmas.
KW - continuous action iteration dilemma
KW - deep reinforcement learning
KW - fixed-time
KW - Optimal complex network
UR - http://www.scopus.com/inward/record.url?scp=85189820714&partnerID=8YFLogxK
U2 - 10.1109/TNSE.2024.3384509
DO - 10.1109/TNSE.2024.3384509
M3 - 文章
AN - SCOPUS:85189820714
SN - 2327-4697
VL - 11
SP - 3771
EP - 3781
JO - IEEE Transactions on Network Science and Engineering
JF - IEEE Transactions on Network Science and Engineering
IS - 4
ER -