Robust hierarchical games of linear discrete-time systems based on off-policy model-free reinforcement learning

Xiao Ma, Yuan Yuan

科研成果: 期刊稿件文章同行评审

2 引用 (Scopus)

摘要

An off-policy model-free reinforcement learning (RL) algorithm is proposed for a robust hierarchical game while considering incomplete information and input constraints. The robust hierarchical game exhibits characteristics of a Stackelberg–Nash (SN) game, where equilibrium points are designated as Stackelberg–Nash–Saddle equilibrium (SNE) points. An off-policy method is employed for the RL algorithm, addressing input constraints by using excitation input instead of real-time update polices as control inputs. Moreover, a model-free method is implemented for the off-policy RL algorithm, accounting for the challenge posed by incomplete information. The goal of this paper is to develop an off-policy model-free RL algorithm to obtain approximate SNE polices of the robust hierarchical game with incomplete information and input constraints. Furthermore, the convergence and effectiveness of the off-policy model-free RL algorithm are guaranteed by proving the equivalence of Bellman equation between nominal SNE policies and approximate SNE policies. Finally, a simulation is provided to verify the advantage of the developed algorithm.

源语言英语
文章编号106711
期刊Journal of the Franklin Institute
361
7
DOI
出版状态已出版 - 5月 2024

指纹

探究 'Robust hierarchical games of linear discrete-time systems based on off-policy model-free reinforcement learning' 的科研主题。它们共同构成独一无二的指纹。

引用此