Research on Projection Interaction Based on Gesture Recognition

With the continuous emergence and vigorous development of various new technologies in the information age, human-computer interaction is also changing constantly. People are increasingly connected with the virtual world, so it is necessary to study more natural ways of human-computer interaction. Projection interaction is one of the most important interaction methods. The purpose of this paper is to study how to overcome the misoperation problem caused by foreign objects (interferers) in the environment of projection interaction (Fig. 1 c). The method (named after FDGR) adopted is to use the fingertip detection algorithm based on user habits in image processing and the gesture recognition algorithm based on Leap Motion to complete the accurate and convenient natural human-computer interaction. Through user experiments, the experiences of completing the fixed assembly task in the mode of projection (P), gesture (G) and FDGR (PG) were compared, and the data was processed and analyzed. The experimental results showed that FDGR could effectively overcome the misoperation caused by foreign objects (interferers) and the non-intuitiveness of simple gestures, and effectively improved the user's operating experience.