Micro-gesture Classification Based on Ensemble Hypergraph-convolution Transformer

Micro-gesture classification has emerged as a significant research area within emotion analysis and human-computer interaction, garnering increasing attention. While some skeleton-based action recognition algorithms utilizing graph convolution networks have shown competence in micro-gesture classification, these deep models still face challenges in representing subtle temporal actions and handling the long-tailed distribution of samples. To address these issues, this paper proposes a deep framework with ensemble hypergraph-convolution Transformers, which fuses multiple models focused on various categories. In this model, the Transformers with hypergraph based attention are constructed and extended to enhance the representation ability of single model. Then a data grouping training and ensemble method is employed to handle imbalanced categories for micro-gestures, resulting in a significant improvement in classification accuracy of single models. Finally, our algorithm model is evaluated on the iMiGUE dataset, which achieves the Top-1 accuracy of 0.6302 and the second ranking in the MiGA2023 Challenge (Track 1: Micro-gesture Classification).