Heterogeneous Multi-Agent Reinforcement Learning for Joint Active and Passive Beamforming in IRS Assisted Communications

Ahstract-The Intelligent Reflecting Surface (IRS) has the potential to reconstruct the electromagnetic propagation environment, paving the way for a new multi-IRS assisted communications paradigm that beams scattered signals for improved spectrum efficiency (SE). However, accurate channel estimation and sharing becomes a challenge when a large number of IRS elements are involved, leading to extra hardware complexity and communication overhead. Moreover, due to the cross-interference caused by massive reflecting paths when multiple IRSs are introduced, SE optimization becomes challenging to achieve a close-formed solution because of non-convexity. This paper improves a heterogeneous based multi-agent deep deterministic policy gradient (MADDPG) approach for joint active and passive beamforming optimization without channel estimation, where base station (BS) and multiple IRSs cooperatively learn to enhance SE and suppress the interference. Due to the centralized-training and distributed-execution feature of MADDPG, the well-trained BS and IRSs can execute both the active and passive beamforming optimization independently without referring to other agents, which can greatly reduce the communication overhead and simplify the IRS deployment. Numeral simulations demonstrate the effectiveness of the proposed approach on enhancing SE and suppressing interference in the multi-IRS assisted communications system.