Multi-modal Micro-gesture Classification via Multi-scale Heterogeneous Ensemble Network

Hexiang Huang, Yuhan Wang, Kerui Linghu, Zhaoqiang Xia

科研成果: 期刊稿件会议文章同行评审

摘要

Micro-gesture classification has become an important research topic in the field of emotion analysis and human-computer interaction, and recently has received more and more attention. Although certain models of action recognition for normal behaviors have demonstrated promising results in classifying micro-gestures, these models still encounter significant challenges when processing micro-gestures that occur within subtle temporal windows. To end this, we propose a multi-scale heterogeneous ensemble network for micro-gesture classification with multi-modal data. This framework combines two models with different architectures and employs multi-scale residual connections within these models to capture fine-grained features and extend the range of receptive field. Simultaneously, we employ a novel data group training strategy, which can more effectively address the class-imbalance problem for model learning over the data. Finally, our model was evaluated on the iMiGUE dataset with Top-1 accuracy of 0.7019, placing second ranking in the MiGA2024 Challenge (Track 1: Micro-gesture Classification).

源语言英语
期刊CEUR Workshop Proceedings
3848
出版状态已出版 - 2024
活动2024 IJCAI Workshop and Challenge on Micro-Gesture Analysis for Hidden Emotion Understanding, MiGA 2024 - Jeju, 韩国
期限: 4 8月 2024 → …

指纹

探究 'Multi-modal Micro-gesture Classification via Multi-scale Heterogeneous Ensemble Network' 的科研主题。它们共同构成独一无二的指纹。

引用此