Few-shot action recognition with implicit temporal alignment and pair similarity optimization

Congqi Cao, Yajuan Li, Qinyi Lv, Peng Wang, Yanning Zhang

科研成果: 期刊稿件文章同行评审

22 引用 (Scopus)

摘要

Few-shot learning aims to recognize instances from novel classes with few labeled samples, which has great value in research and application. Although there has been a lot of work in this area recently, most of the existing work is based on image classification tasks. Video-based few-shot action recognition has not been explored well and remains challenging: (1) the differences of implementation details among different papers make a fair comparison difficult; (2) the wide variations and misalignment of temporal sequences make the video-level similarity comparison difficult; (3) the scarcity of labeled data makes the optimization difficult. To solve these problems, this paper presents (1) a specific setting to evaluate the performance of few-shot action recognition algorithms; (2) an implicit sequence-alignment algorithm for better video-level similarity comparison; (3) an advanced loss for few-shot learning to optimize pair similarity with limited data. Specifically, we propose a novel few-shot action recognition framework that uses long short-term memory following 3D convolutional layers for sequence modeling and alignment. Circle loss is introduced to maximize the within-class similarity and minimize the between-class similarity flexibly towards a more definite convergence target. Instead of using random or ambiguous experimental settings, we set a concrete criterion analogous to the standard image-based few-shot learning setting for few-shot action recognition evaluation. Extensive experiments on two datasets demonstrate the effectiveness of our proposed method.

源语言英语
文章编号103250
期刊Computer Vision and Image Understanding
210
DOI
出版状态已出版 - 9月 2021

指纹

探究 'Few-shot action recognition with implicit temporal alignment and pair similarity optimization' 的科研主题。它们共同构成独一无二的指纹。

引用此