A Novel 3D Convolutional Neural Network for Action Recognition in Infrared Videos

As Infrared (IR) imaging is sensitive to objects emitting heat, it is more useful to distinguish people from the background compared with visible spectrum imaging, especially in poor light enviornment. Recently, IR images have attracted increasing attention in action recognition, for which Convolutional Neural Networks (ConvNets) have achieved great success in both aspects of performance and speed. Compared to 2D ConvNets, 3D ConvNets are more powerful tools for action recognition which can jointly extract features from temporal and spatial domains and combine these together to enhance the performance. In this paper, we propose a novel 3D ConvNet with a deep architecture to realize action recognition in IR videos. Besides, a residual fully connected (FC) module is introduced after the ConvNet backbone to improve the performance. Furthermore, we employe a transfer learning strategy, i.e., the proposed 3D ConvNet is pretrained on a large-scale visible spectrum dataset and then finetuned with false-color version of IR images to generalize well in the action recognition task. Experimental results demonstrate the superiority of the proposed method in Average Precision comparisons.