Intelligent anti-jamming decision-making of frequency hopping communication based on Q-learning in complex electromagnetic environment

Frequency hopping (FH) communication has good anti-fading, anti-jamming and anti-eavesdropping capabilities, so it is one of the main ways to combat electronic jamming. In order to further improve the anti-jamming capability of FH communication, the parameters such as fixed frequency interval, hopping rate and hopping frequency in conventional FH can be assigned with time-varying characteristics. In order to set appropriate hopping parameters to improve the performance of the system in the electromagnetic environment with various types of jamming, a heuristically accelerated Q-learning (HAQL) method is proposed in this paper. Firstly, a theoretical model for the parameter decision-making of FH system is made, and the key parameters affecting the energy efficiency of the system are analyzed. Secondly, a Q-learning model in complex electromagnetic environment is proposed, which includes setting states, actions and rewards, as well as a HAQL-based decision-making algorithm is put forward. Lastly, simulations are carried out under different jamming environments, and simulation results show that the average energy efficiency of HAQL algorithm is higher than that of the SARSA algorithm, the e-greedy QL algorithm and the HQL-OSGM algorithm, respectively.