Inducing Coordination in Multi-Agent Repeated Game through Hierarchical Gifting Policies

Mingze Lv, Jiaqi Liu, Bin Guo, Yasan Ding, Yun Zhang, Zhiwen Yu

科研成果: 书/报告/会议事项章节会议稿件同行评审

摘要

Coordination, i.e., multiple autonomous agents in a system to achieve a common goal, is critical for distributed systems since it can increase the overall reward among all agents. However, The dynamic environment and selfish agents pose challenges to learning coordination behavior from historical interaction data in a long-term interaction environment. Previous works mostly focus on one-shot or short-term distributed agent interaction environments, which often leads to selfish or lazy behavior in long-term interaction environments, i.e., prioritizing individual optimal strategies over cooperative strategies. This behavior is mainly due to the lack of historical memory or incomplete use of historical interaction data to guide the current interaction strategy. In this paper, we propose a hierarchical peer-rewarding mechanism, hierarchical gifting, that allows each agent to dynamically assign some of their rewards to other agents based on historical interaction data and guide the agents towards more coordinated behavior while ensuring that agents remain selfish and decentralized. Specifically, we first propose an auxiliary opponent modeling task so that agents can infer opponents' types through historical interaction trajectories. In addition, we design a hierarchical gifting strategy that dynamically changes during execution based on known opponents' types. We employ a theoretical framework that captures the benefit of hierarchical gifting in converging to the coordinated behavior by characterizing the equilibria's basins of attraction in a dynamical system. With hierarchical gifting, we demonstrate increased coordinated behavior of different risk, general-sum coordination games to the prosocial equilibrium both via numerical analysis and experiments.

源语言英语
主期刊名Proceedings - 2023 IEEE 20th International Conference on Mobile Ad Hoc and Smart Systems, MASS 2023
出版商Institute of Electrical and Electronics Engineers Inc.
279-287
页数9
ISBN(电子版)9798350324334
DOI
出版状态已出版 - 2023
活动20th IEEE International Conference on Mobile Ad Hoc and Smart Systems, MASS 2023 - Toronto, 加拿大
期限: 25 9月 202327 9月 2023

出版系列

姓名Proceedings - 2023 IEEE 20th International Conference on Mobile Ad Hoc and Smart Systems, MASS 2023

会议

会议20th IEEE International Conference on Mobile Ad Hoc and Smart Systems, MASS 2023
国家/地区加拿大
Toronto
时期25/09/2327/09/23

指纹

探究 'Inducing Coordination in Multi-Agent Repeated Game through Hierarchical Gifting Policies' 的科研主题。它们共同构成独一无二的指纹。

引用此