Pain-attentive network: a deep spatio-temporal attention model for pain estimation

In the video surveillance of medical institutions, pain intensity is a significant clue to the state of patients. Of late, some approaches leverage various spatio-temporal methods to capture the dynamic pain information of videos for accomplishing pain estimation automatically. However, there is still a challenge in the spatio-temporal saliency, which means pain is always reflected in some important regions of informative image frames in a video sequence. To this end, we propose a deep spatio-temporal attention model called as Pain-Attentive Network (PAN), which pays more attention on the saliency in the extraction of dynamic features. PAN consists of two subnetworks: spatial and temporal subnetwork. Especially, in spatial subnetwork, a proposed spatial attention module is embedded to make the spatial feature extraction more targeted. Also, a devised temporal attention module is inserted in temporal subnetwork, so that the temporal features focus on informative image frames. Extensive experiment results on the UNBC-McMaster Shoulder Pain database show that our proposed PAN achieves compelling performances. In addition, to evaluate the generalization, we report competitive results of our proposed method in the Remote Collaborative and Affective database.