TY - JOUR
T1 - Multi-modal Micro-gesture Classification via Multi-scale Heterogeneous Ensemble Network
AU - Huang, Hexiang
AU - Wang, Yuhan
AU - Linghu, Kerui
AU - Xia, Zhaoqiang
N1 - Publisher Copyright:
© 2024 Copyright for this paper by its authors.
PY - 2024
Y1 - 2024
N2 - Micro-gesture classification has become an important research topic in the field of emotion analysis and human-computer interaction, and recently has received more and more attention. Although certain models of action recognition for normal behaviors have demonstrated promising results in classifying micro-gestures, these models still encounter significant challenges when processing micro-gestures that occur within subtle temporal windows. To end this, we propose a multi-scale heterogeneous ensemble network for micro-gesture classification with multi-modal data. This framework combines two models with different architectures and employs multi-scale residual connections within these models to capture fine-grained features and extend the range of receptive field. Simultaneously, we employ a novel data group training strategy, which can more effectively address the class-imbalance problem for model learning over the data. Finally, our model was evaluated on the iMiGUE dataset with Top-1 accuracy of 0.7019, placing second ranking in the MiGA2024 Challenge (Track 1: Micro-gesture Classification).
AB - Micro-gesture classification has become an important research topic in the field of emotion analysis and human-computer interaction, and recently has received more and more attention. Although certain models of action recognition for normal behaviors have demonstrated promising results in classifying micro-gestures, these models still encounter significant challenges when processing micro-gestures that occur within subtle temporal windows. To end this, we propose a multi-scale heterogeneous ensemble network for micro-gesture classification with multi-modal data. This framework combines two models with different architectures and employs multi-scale residual connections within these models to capture fine-grained features and extend the range of receptive field. Simultaneously, we employ a novel data group training strategy, which can more effectively address the class-imbalance problem for model learning over the data. Finally, our model was evaluated on the iMiGUE dataset with Top-1 accuracy of 0.7019, placing second ranking in the MiGA2024 Challenge (Track 1: Micro-gesture Classification).
KW - Class-imbalance
KW - Heterogeneous ensemble network
KW - Micro-gesture classification
KW - Multi-scale
UR - http://www.scopus.com/inward/record.url?scp=85212436049&partnerID=8YFLogxK
M3 - 会议文章
AN - SCOPUS:85212436049
SN - 1613-0073
VL - 3848
JO - CEUR Workshop Proceedings
JF - CEUR Workshop Proceedings
T2 - 2024 IJCAI Workshop and Challenge on Micro-Gesture Analysis for Hidden Emotion Understanding, MiGA 2024
Y2 - 4 August 2024
ER -