Joint trajectory and power optimization in multi-type UAVs network with mean field q-learning

Unmanned aerial vehicles (UAVs) are expected to meet the requirements of diverse and efficient communication in the future, which act as aerial base stations (ABSs) with a better line-of-sight communication channels in air-to-ground communication networks. However, resource allocation, interference management and path planning of UAV ABSs have become a series of challenging problems. In this paper, trajectory design and downlink power control of multi-type UAV ABSs are jointly investigated. In order to meet the signal to interference plus noise ratio (SINR) requirements of users, each UAV ABS needs to adjust its position and transmission power. We propose a non-cooperative mean-field-type game (MFTG) model to jointly optimize the trajectory and transmission power of UAV ABS based on the interactions among multiple communication links. In order to simplify the problem, we cluster the users in the given area to get the initial deployment of the UAV ABSs. Furthermore, the discrete MFTG problem is solved by the proposed mean field Q (MFQ)-learning algorithm. Simulation results show that the proposed approach can converge to the equilibrium solution, and reduce the energy cost of each UAV ABS effectively with satisfying the SINR.