Category driven deep recurrent neural network for video summarization

Song Xinhui, Ke Chen, Jie Lei, Li Sun, Zhiyuan Wang, Lei Xie, Song Mingli

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

22 Scopus citations

Abstract

A large number of videos are generated and uploaded to video websites (like youku, youtube) every day and video websites play more and more important roles in human life. While bringing convenience, the big video data raise the difficulty of video summarization to allow users to browse a video easily. However, although there are many existing video summarization approaches, the key frames selected fail to integrate the large video contexts and the qualities of the summarized results are difficult to evaluate because of the lack of ground-truth. Inspired by the previous methods that extract key frames, we propose a deep recurrent neural network model, which learns to extract category-driven key frames. First, we sequentially extract a fixed number of key frames using time-dependent location networks. Second, we utilize recurrent neural network to integrate information of the key frames to classify the category of the video. Therefore, the quality of the extracted key frames could be evaluated by the categorization accuracy. Experiments on a 500-video dataset show that the proposed scheme extracts reasonable key frames and outperforms other methods by quantitative evaluation.

Original languageEnglish
Title of host publication2016 IEEE International Conference on Multimedia and Expo Workshop, ICMEW 2016
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781509015528
DOIs
StatePublished - 22 Sep 2016
Event2016 IEEE International Conference on Multimedia and Expo Workshop, ICMEW 2016 - Seattle, United States
Duration: 11 Jul 201615 Jul 2016

Publication series

Name2016 IEEE International Conference on Multimedia and Expo Workshop, ICMEW 2016

Conference

Conference2016 IEEE International Conference on Multimedia and Expo Workshop, ICMEW 2016
Country/TerritoryUnited States
CitySeattle
Period11/07/1615/07/16

Keywords

  • Recurrent video summarization
  • Reinforcement learning
  • Video categorization

Fingerprint

Dive into the research topics of 'Category driven deep recurrent neural network for video summarization'. Together they form a unique fingerprint.

Cite this