Millimeter-Wave Networking in the Sky: A Machine Learning and Mean Field Game Approach for Joint Beamforming and Beam-Steering

In unmanned aerial vehicle (UAV)-Assisted massive multi-input multi-output (MIMO) millimeter-wave (mmWave) networks, beam-steering guarantees reliable and steady connection between flying base stations and ground users with the challenge of strict angular deviation. In this paper, we investigate a joint optimization problem of beamforming and beam-steering in the multi-UAV mmWave networks, considering line-of-sight (LoS) communication for UAVs. For the hybrid beamforming optimization of massive MIMO mmWave, we propose a hybrid beamforming scheme based on the cross-entropy estimation with the robustness algorithm inspired by machine learning, which aims to optimize the hybrid precoding matrix. For the beam-steering optimization, we propose a novel mean field game (MFG)-based massive MIMO angle control scheme to model the optimal mmWave channel optimization problem between UAVs and ground users. In addition, when dealing with the problem of initial sensitivity and difficulty to solve the partial differential equations in the MFG, we utilize reinforcement learning to achieve the mean field equilibrium, which is described as the mean field learning game algorithm. Finally, a joint beamforming and beam-steering optimization algorithm is proposed to maximize the system sum-rate. Simulation results show the significant improvements in sum-rate, energy efficiency, and spectral efficiency, which verify the effectiveness of the proposed algorithm.