AFSN: Adaptive Frame Selection Network for Efficient Human Synthesis on Smartphones

Viewing real-time human synthesis results on smart-phones provides users a convenient interactive experience. How-ever, most human synthesis solutions relay on high-performance servers and lack interaction with users. In order to reduce traffic between smartphones and cloud, this paper introduces a novel adaptive frame selection network (AFSN) on mobile devices. Besides, we propose a reinforcement-learning based agent that selects the image frames according to the pose change, that reduces communications and the amount of calculations. Based upon AFSN, we build a multi-task human synthesis system, that can deal with background changing, pose transfer, virtual try-on, and other generate tasks in a unified framework. Experimental results on one real-world dataset demonstrate that our method can reduce the number of frames to be processed by more than 50%.