Local and global feature learning for subtle facial expression recognition from attention perspective

Subtle facial expression recognition is important for emotion analysis. In the field of subtle facial expression recognition, there are two intrinsic characters. Firstly, subtle facial expression usually exhibits very small variations in different facial areas. Secondly, those small variations are closely correlated, and they together form an expression. Inspired by these two characteristics of facial expression, a model focus on local variations and their correlations is proposed in this paper. We utilize several attention maps to automatically attend to distinct local regions and extract local features. And then, a self-attention operation is ensembled to extract global correlation feature over the whole image. The global and local features are further fused in an efficient way to classify the facial expression. Extensive experiments have been carried out on LSEMSW and CK+ datasets.