Multi-modal Micro-gesture Classification via Multi-scale Heterogeneous Ensemble Network

Micro-gesture classification has become an important research topic in the field of emotion analysis and human-computer interaction, and recently has received more and more attention. Although certain models of action recognition for normal behaviors have demonstrated promising results in classifying micro-gestures, these models still encounter significant challenges when processing micro-gestures that occur within subtle temporal windows. To end this, we propose a multi-scale heterogeneous ensemble network for micro-gesture classification with multi-modal data. This framework combines two models with different architectures and employs multi-scale residual connections within these models to capture fine-grained features and extend the range of receptive field. Simultaneously, we employ a novel data group training strategy, which can more effectively address the class-imbalance problem for model learning over the data. Finally, our model was evaluated on the iMiGUE dataset with Top-1 accuracy of 0.7019, placing second ranking in the MiGA2024 Challenge (Track 1: Micro-gesture Classification).