Perceiving local relative motion and global correlations for weakly supervised group activity recognition

This paper presents a weakly supervised approach for group activity recognition by exploiting the local relative motion and global correlations among entities. Most existing approaches of group recognition characterize motions of group activities based on the coordinates of individuals or feature maps without excluding the camera motion, which is a combination of local relative motion and camera motion. To address this problem, we utilize a simple but effective Local Relative Motion Module (LRMM): a 3D-CNN-based network to exploit the local movement. We further employ a Global Correlation Module (GCM) to establish relationships among different feature patches for capturing the entire scene. We have evaluated the proposed method on sports and group activity video. The method has achieved state-of-the-art performance on three challenging datasets for weakly supervised group activity recognition. The method has also outperformed some approaches trained with much stronger supervision in the comparative evaluation.