Learning Semantics-Guided Representations for Scoring Figure Skating

Zexing Du, Di He, Xue Wang, Qing Wang

科研成果: 期刊稿件文章同行评审

8 引用 (Scopus)

摘要

This paper explores semantic-aware representations for scoring figure skating videos. Most existing approaches to sports video analysis only focus on reasoning action scores based on visual input, limiting their ability to depict high-level semantic representations. Here, we propose a teacher-student-based network with an attention mechanism to realize an adaptive knowledge transfer from the semantic domain to the visual domain, which is termed semantics-guided network (SGN). Specifically, we use a set of learnable atomic queries in the student branch to mimic the semantic-aware distribution in the teacher branch, which is represented by the visual and semantic inputs. In addition, we propose three auxiliary losses to align features in different domains. With aligned feature representations, the adapted teacher is capable of transferring the semantic knowledge to the student. To verify the effectiveness of our method, we collect a new dataset OlympicFS for scoring figure skating. Besides action scores, OlympicFS also provides professional comments on actions for learning semantic representations. By evaluating four challenging datasets, our method achieves state-of-the-art performance.

源语言英语
页(从-至)4987-4997
页数11
期刊IEEE Transactions on Multimedia
26
DOI
出版状态已出版 - 2023

指纹

探究 'Learning Semantics-Guided Representations for Scoring Figure Skating' 的科研主题。它们共同构成独一无二的指纹。

引用此