Control the Migration of Self-propelling Particles in Thermal Turbulence via Reinforcement Learning Algorithm

Planning energy-efficient trajectories can significantly benefit unmanned aerial vehicles (UAVs) by increasing their endurance. Inspired by birds that exploit warm rising atmospheric currents, flying vehicles can be controlled to utilize the updrafts to conserve energy. We adopt the reinforcement learning algorithm to train a self-propelling particle in a Rayleigh-Bénard (RB) convection cell with periodic vertical boundary conditions, where the large-scale circulation oscillates, and small-scale velocities fluctuate. The trained smart particle successfully learns to utilize the background flow structure to minimize energy consumption, enabling it to migrate with less energy consumption. Despite the complex flow structures that arise both in large-scale circulation and small-scale velocity fluctuations, an energy-efficient trajectory is identified by the particle through the strategy with the highest reward. This research has practical implications for UAVs patrolling in the convective layer of the atmosphere, where energy-efficient trajectories can enhance their endurance and cover a wider range.