Massive Access in 5G and Beyond Ultra-Dense Networks: An MARL-Based NORA Scheme

Power-domain Non-Orthogonal Multiple Access (NOMA) and Ultra-Dense Network (UDN) are promising candidates to cope with the massive access challenge of Machine-Type Communications (MTC). The power level pool is crucial for NOMA to bring performance gains. The existing related literatures rarely consider the power level pool design problem, or only resolve it in the single-cell scenario. However, this problem in multi-cell scenario is more complex and difficult to solve due to the presence of inter-cell interference. Towards this end, we propose a Non-Orthogonal Random Access (NORA) scheme to enable the coexistence of Human-Type Communications (HTC) and MTC for 5G and beyond UDN, where the power level pool design problem in multi-cell scenario is our focus. In order to deal with the complexity caused by multiple optimization objectives and inter-cell interference, we present a Multi-Agent Reinforcement Learning (MARL)-based solution to solve this problem, where each small base station acts as an agent to learn a suitable gap between adjacent power levels. Extensive numerical comparisons demonstrate the superior performances of our proposed scheme in multiple perspectives.