Energy Management Strategy based on Deep Q-network in the Solar-powered UAV Communications System

In this paper, we consider a general UAV-enabled wireless communication system where a solar-powered UAV is deployed to provide continuous communication services for the ground users (GUs). To get better aerodynamic effect and longer maintaining-flight time, the fixed-wing UAV with thin-film solar cells is adopted for the ground coverage. We first divide the energy component of solar-powered UAV as the aerodynamic energy consumption, communication energy consumption and solar energy harvesting from solar cells. Then, we provide the communication capacity of the GUs in our UAV communication system. In order to obtain better throughput capacity under the precondition of continuous flight, we maximize the capacity by jointly optimizing all of the energy components of UAV and three-dimensional (3-D) flight trajectory. To solve the optimization problem, we employ deep Q-Network (DQN) to simplify the decision-making processes and improve the computational efficiency. Furthermore, we compared different retained energy and intensity variations to explore the performance of communications system. The numerical results show that the DQN algorithm can receive great reward in both maintaining-flight time and the capacity.