High-Frequency Limitation Deep Reinforcement Learning Based Energy Management Strategy for Dual Fuel Cell Electric Aircraft

As an important part of hybrid power source system, energy management strategy (EMS) can be modeled by Markov decision processes and solved by reinforcement learning. A high-frequency limitation deep deterministic policy gradient (DDPG) EMS for electric aircraft is proposed, which consists of two parallel proton exchange membrane fuel cells (PEMFCs) and one battery. DDPG is a classic reinforcement learning algorithm with the advantage of making decisions in a continuous action space. However, its actions have been unsatisfactory due to volatility. By adding high-frequency limitation, high-frequency limitation DDPG can make decisions that limit the output power of the PEMFC when the high-frequency scale is large. To validate the proposed EMS, it is compared with the conventional DDPG EMS, and the simulation results show that the proposed EMS significantly reduces the PEMFC fluctuation compared to the conventional DDPG. Under the same environment, the equivalent hydrogen consumption is 8g lower than that of DDPG, and standard deviations of PEMFC1's stress and PEMFC2's stress are reduced by 68.03% and 55.17%, respectively. In addition, its generalization ability is also verified by adjusting the initial state of charge and extending the operating conditions.