Holoscopic 3D micro-gesture recognition based on fast preprocessing and deep learning techniques

It is a challenge to recognize holoscopic 3D (H3D) micro-gesture based on general vision techniques because images captured by H3D imaging system are unclear, i.e., the captured images include a large number of blurred grids. Many feature extraction methods can not be directly used for H3D images because the edge information of the grids will be captured. In this paper, we propose a fast and robust preprocessing method for H3D image reconstruction. The reconstructed images are clear and can be used directly for feature extraction or feature learning. Two contributions are presented in this paper. Firstly, we propose a bi-directional morphological filter used for enhancing the grids in an H3D image. Secondly, we propose a fast clustering algorithm with spatial information to extract grids from the H3D image. Because bi-directional morphological filter is able to incorporate local spatial information to the objective function of the fast clustering algorithm, the grids in H3D images are removed completely. Moreover, because the fast clustering algorithm perform clustering on gray levels of H3D images, a small computational cost is required. The proposed method is used to reconstruct H3D images to obtain multiple images with low resolution captured for 3D gesture recognition. Experiments show that the proposed preprocessing method is not only able to obtain better images that are clear and suitable for feature extraction or feature learning, but also is able to improve recognition accuracy in the micro-gesture recognition based on H3D imaging systems.